Friday, 2018-08-03

*** zul has quit IRC00:00
*** vinaykns has joined #oooq00:11
hubbotFAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario009-multinode-oooq, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci- (2 more messages)00:40
*** dtrainor has quit IRC01:54
hubbotFAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario009-multinode-oooq, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy-tripleo-ci- (2 more messages)02:40
*** ykarel|away has joined #oooq03:11
*** rlandy|bbl is now known as rlandy03:14
*** ykarel|away has quit IRC03:17
*** ykarel has joined #oooq03:18
*** skramaja has joined #oooq03:22
*** udesale has joined #oooq03:31
*** ykarel has quit IRC03:50
*** ratailor has joined #oooq03:59
*** rlandy has quit IRC04:28
*** vinaykns has quit IRC04:34
*** udesale_ has joined #oooq04:35
*** udesale has quit IRC04:35
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario001-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens @ https://review.openstack.org/567224, stable/ocata: legacy- (2 more messages)04:40
*** udesale has joined #oooq04:46
*** holser_ has joined #oooq04:47
*** udesale_ has quit IRC04:48
*** ykarel has joined #oooq04:52
*** udesale has quit IRC04:54
*** udesale has joined #oooq04:55
*** brault has joined #oooq05:08
*** quiquell|off has quit IRC05:36
*** quiquell has joined #oooq05:39
*** jaosorior has quit IRC05:40
*** jaosorior has joined #oooq05:41
*** holser_ has quit IRC05:48
*** jrist has quit IRC05:49
*** apetrich has joined #oooq05:57
*** jrist has joined #oooq06:04
*** Remco8 has joined #oooq06:08
Remco8With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/06:08
Remco8I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/06:08
Remco8Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate06:08
*** panda|rover sets mode: +b *!*@123.20.117.8706:08
*** Remco8 was kicked by panda|rover (Remco8)06:08
*** gvrangan has joined #oooq06:11
*** jfrancoa has joined #oooq06:18
*** Simba27 has joined #oooq06:27
*** tesseract has joined #oooq06:28
*** Simba27 has quit IRC06:31
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario001-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens @ https://review.openstack.org/567224, stable/ocata: legacy- (2 more messages)06:40
*** ccamacho has joined #oooq06:45
*** infina5 has joined #oooq07:06
infina5With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/07:06
*** panda|rover sets mode: +b *!*@cpe-45-46-166-86.buffalo.res.rr.com07:06
*** infina5 was kicked by panda|rover (infina5)07:06
*** bigpresh7 has joined #oooq07:07
bigpresh7With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/07:07
*** panda|rover sets mode: +b *!*@78.180.92.6407:07
*** bigpresh7 was kicked by panda|rover (bigpresh7)07:07
arxcruzhaha panda|rover07:08
*** amoralej|off is now known as amoralej07:09
quiquellarxcruz: panda|rover Is now the fastest gun in the saloon07:12
arxcruzquiquell: yeah, billy the kid :D07:12
quiquellarxcruz: Bots have put a price at his head :-)07:14
arxcruzlol07:14
*** mappum has joined #oooq07:15
mappumWith our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/07:15
*** panda|rover sets mode: +b *!*@host-92-1-238-216.as43234.net07:15
*** mappum was kicked by panda|rover (mappum)07:15
*** Xiti22 has joined #oooq07:15
Xiti22With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/07:15
Xiti22I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/07:15
Xiti22Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate07:15
*** panda|rover sets mode: +b *!*@pool-96-232-202-123.nycmny.fios.verizon.net07:15
*** Xiti22 was kicked by panda|rover (Xiti22)07:15
quiquellXiti22 was faster07:16
*** tosky has joined #oooq07:26
*** thomas28 has joined #oooq07:28
thomas28With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/07:28
thomas28I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/07:28
thomas28Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate07:28
*** panda|rover sets mode: +b *!*@173.245.203.11607:28
*** thomas28 was kicked by panda|rover (thomas28)07:28
*** jtomasek_ has quit IRC07:29
*** jtomasek has joined #oooq07:31
quiquellssbarnea: You there ?07:36
ssbarneamorning! yep07:37
quiquellssbarnea: Can you help me with the QE of https://trello.com/c/DdSHkgnE/884-replace-tocijobtype-variable-s17-undercloud-playbooks-environment-tags ?07:38
ssbarneasure07:39
quiquellWe have a list of playbook_executions.log that we have to check07:40
quiquellssbarnea: Let's split it07:40
quiquellssbarnea: The review to compre to is this, https://review.openstack.org/#/c/56044507:41
quiquellssbarnea: I take the first 5 ones and you take the next 5 ones07:41
quiquellssbarnea: It's ok ?07:41
ssbarneaok07:41
*** vegii25 has joined #oooq07:57
vegii25With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/07:57
*** panda|rover sets mode: +b *!*@173.245.203.4707:57
*** vegii25 was kicked by panda|rover (vegii25)07:57
*** florianf has joined #oooq08:01
*** dtantsur|afk is now known as dtantsur08:01
ssbarneaquiquell: lookinfg at queens, one I found "ERROR - Couldn't retrieve env".08:03
ssbarneaincluding a "brctl: command not found" after that.08:04
*** Krenair18 has joined #oooq08:05
Krenair18With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/08:05
*** panda|rover sets mode: +b *!*@64.145.76.3608:05
*** Krenair18 was kicked by panda|rover (Krenair18)08:05
quiquellssbarnea: Let's compare just playbook_executions.log08:07
quiquelland only for master08:08
*** holser_ has joined #oooq08:08
quiquellssbarnea: There are a lot of tooling errors in the post08:08
*** gvrangan has quit IRC08:10
*** nealshire has joined #oooq08:10
nealshireWith our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/08:10
nealshireI thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/08:10
nealshireRead what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate08:10
*** panda|rover sets mode: +b *!*@81.171.110.4808:10
*** nealshire was kicked by panda|rover (nealshire)08:10
*** saneax has joined #oooq08:12
panda|rovermorning08:12
quiquellpanda|rover: good morning08:13
quiquellpanda|rover: Have questions regarding adding more CI coverage to the leftovers patch08:13
quiquellpanda|rover: What are the implications ?08:13
quiquellpanda|rover: It will easy our sprint lifes08:13
*** e-mail has joined #oooq08:16
e-mailWith our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/08:16
*** panda|rover sets mode: +b *!*@abuq169.neoplus.adsl.tpnet.pl08:16
*** e-mail was kicked by panda|rover (e-mail)08:16
panda|roverquiquell: I agree with you, by I smell trouble from infra perspective08:20
quiquellpanda|rover: Is filtered by tripleo-ci playbook, do we need to ask them about it ?08:20
panda|roverquiquell: no, but I want to consider 1) consequences of running all the jobs all the time 2) the load we are putting on infra by starting 15 jobs for every single patch08:21
quiquellpanda|rover: Is not all the time, is only at tripleo-ci playbook changes08:24
quiquellpanda|rover: I have add a files directive in the layout.yaml to reduce it08:24
ykarelpanda|rover, hi08:33
ykarelpanda|rover, i see nova console-log bmc- is not working in te-broker(i saw logs):- https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/te-broker/destroy-env#L3508:34
ykarelpanda|rover, can u try PYTHONIOENCODING=utf8 on te-broker with nova console-log command08:35
panda|roverwho put that patch ?08:35
panda|roverI was going to work on it today ...08:35
ykarelpanda|rover, bnemec08:35
panda|rover-_-08:35
ykarelalso awk is not working08:36
ykarelnot sure what was intented, but grep can be used to debug08:36
panda|roverykarel: what do you see that is not working "08:39
panda|roverykarel: I see the bmc logs in the te-broker08:40
hubbotFAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario009-multinode-oooq, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp- (2 more messages)08:40
*** gvrangan has joined #oooq08:40
ykarelpanda|rover, i see ERROR (UnicodeEncodeError): 'ascii' codec can't encode characters in position 47647-47649: ordinal not in range(128)08:41
panda|roverykarel: where ?08:41
ykarelpanda|rover, http://38.145.34.41/testenv-worker.log, i08:41
ykarelare the bmc logs latest08:42
panda|roverok I see them, they are not fatal, I see the logs formed there08:43
ykarelpanda|rover, okk then good08:43
panda|roverykarel: I'll add what you suggested as test08:44
mariosgerrit down?08:44
mariosno well back now08:44
panda|rovermarios: review.openstack works for me08:44
Tengumarios: works for me as well08:44
ykarelpanda|rover, ack08:45
mariosthanks panda|rover Tengu works here too now08:45
Tengumarios: zombie-gerrit :)08:46
quiquellssbarnea: https://review.openstack.org/58847508:50
quiquellssbarnea: Have time for a bj session on it ?08:55
*** ykarel has quit IRC08:57
*** jaosorior has quit IRC09:03
*** Carlos061117 has joined #oooq09:06
Carlos061117With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/09:07
*** panda|rover sets mode: +b *!*@ccj48.neoplus.adsl.tpnet.pl09:07
*** Carlos061117 was kicked by panda|rover (Carlos061117)09:07
arxcruzi think panda|rover is the new bot09:11
arxcruzpanda|rover: !status09:12
TenguXD09:13
quiquellarxcruz: hubbot panda|rover Fight !!! rise of the robots09:13
panda|roverarxcruz: I don't know the command status09:16
quiquellarxcruz: Do you know where is panda|rover repo ?09:17
quiquellssbarnea: added diff to the comparing script09:17
arxcruzquiquell: nope09:17
*** Fearful has joined #oooq09:27
FearfulWith our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/09:27
FearfulI thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/09:27
FearfulRead what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate09:27
*** panda|rover sets mode: +b *!*@64.145.76.7909:27
*** Fearful was kicked by panda|rover (Fearful)09:27
*** zoli is now known as zoli|lunch09:32
quiquellssbarnea: Better here09:37
quiquellssbarnea: noop change vs TAGs change09:37
quiquellssbarnea: ./compare-reviews.py 584508 56044509:37
quiquellssbarnea: There are some differences there09:37
quiquellssbarnea: We have to check if they can break it or not09:37
ssbarneaquiquell: regarding diffs, I find it hard to read its current output, huge diff. considering starting local diff tool to do the comparison side by side, or something similar. give me 5mins to test.09:40
quiquellssbarnea: Ok let me upload a change to show log urls, to check manually09:41
ssbarneasure, i will make a new CR for my feature to avoid accidents09:43
quiquellssbarnea: Don't worry just do a pull --rebase09:44
quiquellbefore push09:44
quiquellhumm, no that's not ok09:44
quiquellimportant thing is to spot reviews to check09:44
quiquellfor that it's working09:44
*** ykarel has joined #oooq09:45
quiquellssbarnea: updated with logs url09:45
*** ykarel is now known as ykarel|travel09:48
quiquellssbarnea: Now it uses unified_diff it's similar to linux diff command09:50
*** ykarel|travel has quit IRC09:52
ssbarneaquiquell: pprint(list(..)) is not what we want, it prints really ugly python list instead of each changelist line on its own.09:53
quiquellssbarnea: Update, i  have fix that09:53
quiquellssbarnea: just "\n".join is enough, now it's a plain diff you can compre09:53
quiquellssbarnea: Found that we are adding an extra "," at tag list I think this is not a problem09:54
quiquellssbarnea: Also featursest path is different, but it's correct too09:54
ssbarneaquiquell: indedd, not the only things missing is to add colorama so the output would match  "git --no-pager diff --no-index foo bar"09:55
quiquellssbarnea: Add it if man, I am not going to update the script, it's usable now09:56
quiquellssbarnea: Directly in the review don't worry09:56
quiquellssbarnea: Also the tags list is broken for upgrades...09:57
quiquellgoing to fix09:57
quiquellssbarnea: Fixed stuff like this ---tags build,undercloud-setup,undercloud-scripts,undercloud-install,undercloud-validate,images,,undercloud-upgrade10:00
quiquell+--tags build,undercloud-setup,undercloud-scripts,undercloud-install,undercloud-validate,images,undercloud-upgrade10:00
quiquellIs the only problem I have found, did you find anything else ?10:02
*** agopi has quit IRC10:08
quiquellpanda|rover: Found that we have some residual use for TOCI_JOBTYPE, don't know if this sprint is the momento to clean this up10:14
quiquellpanda|rover: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/playbooks/tripleo-ci/templates/oooq_common_functions.sh.j2#n17310:15
ssbarneaquiquell: https://seashells.io/v/57chJeuh -- ok with results?10:15
quiquellssbarnea: Didn't know about seashells that's pretty cool10:17
quiquellssbarnea: Have upload a change to fix the ',' problem10:18
quiquellssbarnea: The other path is ok, it's just follow the zuulv3 path and that's better10:18
ssbarneaquiquell: it is cool and live streaming! but it has two major issues: server is not opensource and it does remove logs after 24 hours.10:19
ssbarneai was planning to start a personal side project that would do the same, but fully open source, so we could run our own severs without time and size limits.10:20
quiquellssbarnea: You to have extra eyes, did you find any more difference except, the tags and featureset path ?10:20
quiquellssbarnea: This is perfect to implement at logs.openstack.org10:21
ssbarneaalso i want to add line numebers (bookmarkable). looking now at the diff...10:21
ssbarneawas the yaml file relocation needed as part of this patch or is something else?10:22
ssbarneai found another place with double comman in text, but this was the removed line10:23
quiquellssbarnea: That's a good one, this patch is only about tags, don't know why that it doesn't appear in the previous one10:24
quiquellssbarnea: Ahh it also changes FEATURESET_FILE to use {{ tripleo_root }}10:25
quiquellssbarnea: Is all good10:25
ssbarneait would be useful to eliminate noise. anyway i reallu like that new tool10:25
quiquellssbarnea: Is going to help us, we have a pletora of new reviews to check with this10:25
ssbarneamaybe we should split it, to ease merges.10:25
quiquellssbarnea: It's already too small,10:26
quiquellssbarnea: Let's not add more reviews, it's a pain to merge10:26
quiquellssbarnea: What more places did you find with comma issues ?10:26
*** Xgc16 has joined #oooq10:32
Xgc16With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/10:32
*** panda|rover sets mode: +b *!*@pool-108-30-42-92.nycmny.fios.verizon.net10:32
*** Xgc16 was kicked by panda|rover (Xgc16)10:32
quiquellmarios: Are you there ?10:35
quiquellpanda|rover: Do you have a very small brain cycle for me ?10:36
panda|roverquiquell: how small ?10:36
quiquellpanda|rover: context -> upgrade tags, looks it's better if we move them to the jobs after talking yesterady with weshay10:37
quiquellpanda|rover: If you agree I move them back to the jobs10:37
panda|roverquiquell: move to the jobs ?10:37
quiquellpanda|rover: Job configuration as it was originally "overcloud-upgrade", "undercloud-upgrade", etc...10:38
panda|roverquiquell: in zuul job configuration you put extra_tags: "overcloud_upgradae" or something like that ?10:39
quiquellpanda|rover: It was like this before, we dicide to keep it how it was and wait for the refactoring of emit releases file script10:39
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario001-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens @ https://review.openstack.org/567224, stable/ocata: legacy- (2 more messages)10:40
panda|roverquiquell: I like the new EXTRA_TAGS implementation10:40
quiquellpanda|rover: Ok, then I keep it10:40
quiquellpanda|rover: Also found some residual use for TOCI_JOBTYPE, don't know if we want to fix it in this sprint10:41
panda|roverquiquell: it's simple, maintains part of the logs, which we don't want to change for now10:41
quiquellpanda|rover: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/playbooks/tripleo-ci/templates/oooq_common_functions.sh.j2#n7610:42
quiquellpanda|rover: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/playbooks/tripleo-ci/templates/oooq_common_functions.sh.j2#n7610:42
ssbarneaquiquell: only two types of comma issues: double one in original in middle of string (probably ignorable as we fixed it). b) removed suffix comma which again should not be a problem by itself.10:43
quiquellpanda|rover: Damn duplication, well all the TOCI_JOBTYPE that are at oooq_common_functions.sh.j210:43
quiquellssbarnea: I think the same man10:43
quiquellssbarnea: Let's wait for the change and compare again10:44
quiquellssbarnea: Going to try that we don't have fail possitives is a playbook breaks10:45
quiquellssbarnea: Just forcing soime playbook to fail10:45
quiquellssbarnea: Sice it's running in a loop checking just one I think it's enough10:45
quiquellssbarnea: Humm we have non voting failing jobs, so we have no fail possitives10:48
*** zoli|lunch is now known as zoli10:50
*** dtrainor has joined #oooq11:02
quiquellpanda|rover, marios: PTAL leftovers change https://review.openstack.org/#/c/58446511:07
quiquellDoD of this review is ok11:07
Tenguheya! is this already known? "Could not find a version that satisfies the requirement container-check (from versions: )\nNo matching distribution found for container-check" - makes RDO CI check fail... and unrelated to my change11:30
mariosquiquell: ack sorry was trying to chase that pike timeout issue will check it out11:32
quiquellmarios: No problem, I am also checking the playbook executions log, with a script, will paste results in the individual reviews11:34
quiquellssbarnea: Going to remove the coloring, is a pain to paste it somewhere11:46
quiquellssbarnea: and vim already paint it for you11:46
ssbarneawhat?11:47
quiquellssbarnea: the comparing script I mean11:47
ssbarneai don't understand the problem11:47
quiquellssbarnea: Dumping to a file, to put it at a pastebin, involves piping it to disable colors, maybe we can make them optional11:47
quiquellssbarnea: Or even remove coloring if the script detects that we are piping11:48
ssbarneasure, let me do the fix, i know how.11:48
ssbarneawell, I did something on purpose to keep it, or seashells will miss it.11:48
quiquellssbarnea: Nice, thanks man11:48
quiquellssbarnea: Going to use https://paste.fedoraproject.org/ it has colors and all11:48
ssbarneapiping is not always a good reason for disabling them but let me fix it.11:48
*** zul has joined #oooq11:54
ssbarneai just found bug in last version, unrelated to colors.11:54
*** zul has left #oooq11:54
quiquellssbarnea: nice11:57
quiquellssbarnea: Have compare 4 reviews with a noop in 5 minutes :-)11:58
*** zul has joined #oooq11:59
ssbarneaquiquell: test with "compare-reviews.py 584508 560445" and you will see the bug.12:02
ssbarneaafter this download the review again, i just pushed a fix for both color and this bug.12:03
quiquellssbarnea: Ahh ok missing log file12:03
ssbarneanow it will use ansi only for tty terminals (plain text on tee) unless user defines PY_COLORS=1 which is "standard" way to enforce coloring regardless tty.12:03
quiquellssbarnea: Thanks, that's what is needed12:06
*** ratailor has quit IRC12:08
*** panda|rover is now known as panda|rover|off12:17
*** rlandy has joined #oooq12:38
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario001-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens @  (2 more messages)12:40
weshayrlandy, panda|rover|off I recreated the ovb introspection issue on libvirt :)12:42
rlandyweshay: really? where is the failure - on introspection or on registration?12:42
rlandythe bug reports two diff issues12:42
weshayhttp://paste.openstack.org/show/727228/12:43
rlandyweshay: bob fournie left a message in the bug12:43
weshayrlandy, I saw :(12:43
weshayre: his message12:43
rlandyunfortunately negative12:44
rlandyweshay: have you contacted him since that? otherwise I'd like to ping him to discuss12:44
weshayrlandy, going to drink a cup a coffee, look at this a big more then will ping12:46
rlandyok12:46
*** skramaja has quit IRC12:48
*** rfolco|off is now known as rfolco|ruck12:52
*** saneax has quit IRC12:55
*** gvrangan has quit IRC12:57
quiquellrlandy, weshay: PTAL leftovers https://review.openstack.org/#/c/584465/12:59
quiquellmyoung, rfolco|ruck: ^12:59
quiquellDoD is ok13:00
rlandyk - will look in a bit13:01
quiquellrfolco|ruck: I have some tiemouts at POST, have you see them in CI ?13:03
quiquellrfolco|ruck: Takes more than 30 minutes13:03
quiquellrfolco|ruck: http://logs.openstack.org/08/584508/27/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades/287ff78/job-output.txt.gz#_2018-08-03_11_36_07_62779013:03
rfolco|ruckquiquell, is this for any particular job like upgrades ?13:07
*** amoralej is now known as amoralej|lunch13:07
quiquellrfolco|ruck: It's upgrades a non voting one13:07
*** agopi has joined #oooq13:10
*** agopi_ has joined #oooq13:11
weshayrlandy, ping when you have 10min to go through the introspection issue w/ me13:12
quiquellHave a good weekend13:13
*** quiquell is now known as quiquell|off13:13
rlandyweshay: sure, whenever you have time13:14
weshayrlandy, k.. let me get setup w/ a tmate13:14
*** agopi has quit IRC13:14
*** agopi_ is now known as agopi13:14
*** quiquell|off has quit IRC13:17
rascaweshay, rlandy, myoung hi folks, don't want to be silly about this, but since I'm not testing rdophase2 for master/pike/queens since long time I'd like to have your eyes on https://code.engineering.redhat.com/gerrit/#/c/145910/13:21
*** agopi_ has joined #oooq13:25
*** agopi has quit IRC13:28
ssbarneanewbie question:  do we have any job running pep8 on tripleo-ci repo changes? I was not able to spot one myself, maybe is embedded on another one?13:29
*** agopi_ is now known as agopi13:31
rlandyrasca: ok - will look - in meeting - ping me again if I have not done the right thing in a hour or so :)13:34
*** jaganathan has quit IRC13:38
myoungrasca: : will have a look shortly, getting thru responding to in-sprint reviews13:46
*** vinaykns has joined #oooq13:49
*** Goneri has joined #oooq13:55
weshayssbarnea, I don't think so..13:58
*** saneax has joined #oooq14:04
*** amoralej|lunch is now known as amoralej14:10
myoungrlandy: are you free to chat at noon?14:12
rlandymyoung:yes14:12
myoungrlandy: great i'll send an invite :)14:13
rlandyrasca: looking at your review now14:36
rascamyoung, rlandy, thanks!14:37
myoungrasca: ack, i got a JJB error. back in a bit, shifted lunch early today for meetings14:38
*** myoung is now known as myoung|lunch14:38
rlandyrasca: I'm fine with your change - but waiting on myoung to confirm fixed jjb14:39
rascarlandy, ack14:39
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario001-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens @  (2 more messages)14:40
rascamyoung|lunch, fixed the missing space, should be fine by now14:54
*** tcw1 has joined #oooq15:07
rlandyrfolco|ruck: hi - do you know including post-run will run twcie, rather than just override?15:10
rlandyI will create the second parent15:10
rlandyI'd just like to understand why15:10
*** tcw has quit IRC15:10
rfolco|ruckrlandy, this is not a job override, so different jobs will run their own playbooks, pre, run and post for each are independent15:12
rfolco|ruckregardless of being parent15:12
rlandyrfolco|ruck: how does pre-run differ?15:13
rlandynvm15:15
rlandythere is no pre-run in the job itself15:16
rfolco|ruckyep15:17
weshayrlandy, /me is so confused15:18
weshayhttps://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-gate-master-delorean-quick-basic-6575/undercloud/home/stack/overcloud_prep_images.log.gz15:18
weshaylatest check jobs in ci.centos15:18
weshayhttps://ci.centos.org/view/rdo/view/tripleo-gate/job/tripleo-quickstart-gate-master-delorean-quick-basic/15:18
*** tcw1 is now known as tcw15:19
rlandyweshay: that's it - the end of log?15:20
rlandylog server thinks you have seen enough for today15:20
rlandyoh nvm - passes15:20
weshayrlandy, lolz ;)15:21
weshayyou know you've been looking at too many failures .. .WHEN15:21
rlandyweshay: ok - so what is confusing then - of it passes?15:21
weshayrlandy, ya.. the consistency of the passes15:21
weshayrlandy, /me wonders if I should rebuild my centos boxes15:22
weshaythink I should15:22
rlandyok - yeah - story of our lives15:22
rlandychecking if rdocloud is back15:22
weshaythink so15:22
rlandyyay15:22
rlandy[{u'result': u'Node 8324389b-cc83-47a1-ba3f-8a36a42edaaa did not reach state "manageable", the state is "enroll", error: Failed to get power state for node 8324389b-cc83-47a1-ba3f-8a36a42edaaa. Error: IPMI call failed: power status.'},15:23
rlandyweshay: well I got that error on ovb but15:24
rlandythat was probably the reboot15:24
rlandy(undercloud) [zuul@undercloud ~]$ openstack baremetal node list15:25
rlandy+--------------------------------------+-------------------+---------------+-------------+--------------------+-------------+15:25
rlandy| UUID                                 | Name              | Instance UUID | Power State | Provisioning State | Maintenance |15:25
rlandy+--------------------------------------+-------------------+---------------+-------------+--------------------+-------------+15:25
rlandy| 8324389b-cc83-47a1-ba3f-8a36a42edaaa | baremetal-14836-0 | None          | None        | enroll             | False       |15:25
rlandy| a7ac43c9-d3df-43d6-a0a6-e1d0ab1d81b3 | baremetal-14836-1 | None          | None        | enroll             | False       |15:25
rlandy| b86270e6-8c94-4964-a7fd-90767962acbb | baremetal-14836-2 | None          | None        | enroll             | False       |15:25
rlandy| e0bdb5a5-2d82-48b5-965f-6076022fadd0 | baremetal-14836-3 | None          | None        | enroll             | False       |15:25
rlandy+--------------------------------------+-------------------+---------------+-------------+--------------------+-------------+15:25
rlandystuck in enroll15:25
rlandylet's try again15:27
weshayhrm. cool15:29
*** zoli is now known as zoli|gone15:38
*** zoli|gone is now known as zoli15:38
*** myoung|lunch is now known as myoung15:39
mariosweshay: woo green pike job at https://review.openstack.org/#/c/583195/15:41
mariosrlandy: ^^15:41
mariosweshay: (it didn't update yet am watching zuul)15:41
rlandynice - what fixed it?15:41
mariosrlandy: https://review.openstack.org/#/c/588527/15:41
weshayya.. what fixed?15:41
* weshay looks15:41
mariosrlandy: explanation at comment #2 https://bugs.launchpad.net/tripleo/+bug/178506715:41
openstackLaunchpad bug 1785067 in tripleo "without bootstrap-subnodes pike deployment fails for missing os-collect-config" [High,Triaged] - Assigned to Marios Andreou (marios-b)15:41
mariosrlandy: i need to update the commit messsage of that DNM :)15:42
rlandymy rdocloud access is gone?15:42
weshayheh15:42
rlandyok back15:42
mariosweshay: rlandy looks like we could get away with a 'cheeky' repo_cmd_after15:42
mariosweshay: rlandy assuming that sits well with folks15:42
marioslemme update the commit message sec15:42
myoungis the issue of the log server not having linked timestamps already known/understood?  in looking at patches this morning i noticed the missing links15:44
marioshttps://review.openstack.org/#/c/588527/ refresh commit message weshay rlandy but the real test is the one that has this as depends-on, i.e. where pike was failing at https://review.openstack.org/#/c/583195/ ... zuul there says 'depend on a change that failed to merge' but from console you can see the overcloud was deployed OK (previously hanging) and job completed  http://zuul.openstack.org/stream.html?uuid=961d859bccc344c5adddb2171d27ebd1&logfi15:45
myoungahh or it's the browser issue15:45
myoungrasca: ack, will have another look thx15:45
weshaymarios, I don't think that closes the bug15:46
weshaymarios, that is a workaround no?15:47
mariosweshay: :( why15:47
weshaymarios, bah.. how much time is left in your day?15:48
* weshay chatting w/ rfolco|ruck 15:48
weshaymarios, would like to talk to you about that for 5 min15:48
mariosweshay: not alot :) i'm meant to be finalising slides but this thing sucked me in bigtime15:48
mariosand i should status15:48
weshayI don't think that repo setup is run on the overcloud nodes15:48
mariosweshay: it is15:48
weshaymaybe it is in multinode15:48
mariosweshay: sec15:48
weshaybut it wouldn't be on a customer site.. so deployed server config15:49
weshayshould do this afaik15:49
marios " so on the timeout job does subnode 2 has repos? " yes http://logs.openstack.org/95/583195/15/check/tripleo-ci-centos-7-containers-multinode-pike/0618bfd/logs/subnode-2/etc/yum.repos.d/delorean-current.repo.txt.gz15:49
mariosweshay: ^^15:49
mariosweshay: http://logs.openstack.org/95/583195/15/check/tripleo-ci-centos-7-containers-multinode-pike/0618bfd/logs/subnode-2/home/zuul/  here in fact you can see the repo-setup scipts15:50
mariosweshay: now repo setup is good, but what was missing was the bootstrap installation of jq and oepnstack-heat-agents normally done with https://github.com/openstack/tripleo-common/blob/master/roles/tripleo-bootstrap/defaults/main.yml but it doesn't exist on Pike15:51
mariosweshay: didn't decide about jq yet, but we definitely want openstack-heat-agents on the subnodes... may as well use the repo-setup to do that (even if only for Pike, or we need another way)15:52
mariosweshay: yeah well re deployed-server15:52
mariosweshay: that is also true15:52
mariosweshay: this repo-setup stuff is executed by/invoked in the tripleo-heat-templates in fact15:53
weshaysec. need 515:53
mariosweshay: in the deploy_steps_playbook (see comment 2 in the bug it has all these pointers already ) https://github.com/openstack/tripleo-heat-templates/blob/21dadd4ef37393ef738eca050b130eb7ef558241/common/deploy-steps.j2#L42015:53
rlandyrdocloud keeps kicking me out now15:54
*** ccamacho has quit IRC15:57
*** jfrancoa has quit IRC16:00
-openstackstatus- NOTICE: The infra team is renaming projects in Gerrit. There will be a short ~10 minute Gerrit downtime in a few minutes as a result.16:01
*** vinaykns has quit IRC16:03
myoungrasca: confirmed, merged16:05
*** rlandy is now known as rlandy|mtg16:07
rascamyoung++16:07
hubbotrasca: myoung's karma is now 416:07
*** ccamacho has joined #oooq16:09
*** holser_ has quit IRC16:10
*** holser_ has joined #oooq16:15
*** udesale has quit IRC16:20
rlandy|mtgrfolco|ruck: can you join us for a moment?16:20
rlandy|mtgmyoung'sbj16:21
myoung^^ https://bluejeans.com/705085945516:21
weshaymarios, ok16:21
weshaymarios, can you join my blue for 5min16:21
weshaymarios, bah.. did I loose you?16:24
rfolco|ruckrlandy|mtg, yes, one sec16:25
*** holser_ has quit IRC16:32
rascahave a great weekend folks!16:39
hubbotFAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario009-multinode-oooq, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp- (2 more messages)16:40
*** tesseract has quit IRC16:44
*** agopi_ has joined #oooq16:51
*** agopi has quit IRC16:54
*** agopi_ is now known as agopi16:54
*** vinaykns has joined #oooq17:10
*** florianf has quit IRC17:14
*** dtantsur is now known as dtantsur|afk17:20
myoungoof...https://review.rdoproject.org/ down for others as well?17:21
rlandy|mtgnot responding17:21
*** hubbot has quit IRC17:22
*** dmellado has quit IRC17:22
agopimyoung, there was a notice that came on some channels17:23
agopiNOTICE: The infra team is renaming projects in Gerrit. There will be a short ~10 minute Gerrit downtime in a few minutes as a result.17:23
agopiprobably related to that?17:24
-openstackstatus- NOTICE: Project renames and review.openstack.org downtime are complete without any major issue.17:27
myoungweshay, rfolco|ruck, rlandy|mtg: https://docs.openstack.org/infra/manual/zuulv3.html#job-inheritance17:44
rlandy|mtgweshay,: https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/multinode-jobs.yaml#L36217:48
*** myoung is now known as myoung|biab17:53
*** rlandy|mtg is now known as rlandy|brb18:02
*** amoralej is now known as amoralej|off18:03
*** ccamacho has quit IRC18:04
*** ccamacho has joined #oooq18:05
*** ccamacho1 has joined #oooq18:08
*** ccamacho has quit IRC18:09
*** vinaykns has quit IRC18:25
*** rlandy|brb is now known as rlandy18:34
*** jtomasek has quit IRC18:51
rlandyweshay: is your libvirt reproducer still up?19:07
rlandyunderstandably, I'm out of the water on ovb rdocloud19:07
*** vinaykns has joined #oooq19:14
*** myoung|biab is now known as myoung19:15
*** tcw has quit IRC19:20
*** tcw has joined #oooq19:20
weshayrlandy, yes19:30
weshayrlandy, need the tmate?19:30
*** vinaykns has quit IRC19:30
rlandyweshay: no worries - setting one up on my other machine19:38
weshaymyoung, you back?19:56
*** dmellado has joined #oooq19:56
myoungweshay: aye, been back for a bit19:58
* myoung has been plowing thru zuul docs19:58
myoungweshay: 1:1?19:58
myoungweshay: or other?19:58
weshaymyoung, 1:120:00
weshaymyoung, good time?20:00
myoungweshay: my time is your time ;)20:00
myoungweshay: sure.  your room?20:00
weshayaye20:00
myoungincoming20:00
myoungweshay: what's the numeric meeting ID for your room?20:01
weshayhttps://bluejeans.com/411356779820:01
myoungahh ok i had https://bluejeans.com/28749027820:01
myoungincoming20:01
*** hubbot has joined #oooq20:04
*** ccamacho1 has quit IRC20:10
*** vinaykns has joined #oooq20:39
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario001-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens @  (2 more messages)20:41
*** agopi has quit IRC20:43
*** Goneri has quit IRC20:46
rlandyis your horizon weird looking?20:47
rlandymyoung: weshay: ^^?20:47
rlandylook ok in chrome now, maybe it's just an artifact20:48
*** rfolco|ruck is now known as rfolco|off21:01
rlandyweshay: introspection is doing fine now for me after rdocloud came back into action. I'll add my last notes to the bug and email the bug info to derek since he seems to be on it now21:13
weshayk21:18
weshayrlandy, mine is not there yet21:18
rlandyok - still tracking a last run21:19
*** apetrich has quit IRC21:34
rlandyweshay++21:50
hubbotrlandy: weshay's karma is now 721:50
rlandyfinally - something!!!21:50
*** dsneddon has quit IRC22:02
*** dsneddon has joined #oooq22:03
*** dsneddon has quit IRC22:05
*** dsneddon has joined #oooq22:05
*** Guest40801 has joined #oooq22:29
Guest40801This channel has been hacked by Australia's #1 hacker Simon 'eVestigator' Smith https://evestigatorsucks.com/22:29
Guest40801With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/22:29
*** panda|rover|off sets mode: +b *!*@ip4d15fa74.dynamic.kabel-deutschland.de22:29
*** Guest40801 was kicked by panda|rover|off (Guest40801)22:29
*** Boulet25 has joined #oooq22:38
Boulet25This channel has been hacked by Australia's #1 hacker Simon 'eVestigator' Smith https://evestigatorsucks.com/22:38
Boulet25With our IRC ad service you can reach a global audience of entrepreneurs and fentanyl addicts with extraordinary engagement rates! https://williampitcock.com/22:38
*** panda|rover|off sets mode: +b *!*@24-247-165-152.dhcp.bycy.mi.charter.com22:38
*** Boulet25 was kicked by panda|rover|off (Boulet25)22:38
hubbotFAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-3nodes-multinode, tripleo-ci-centos-7-scenario001-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens @  (2 more messages)22:41
*** rlandy has quit IRC22:42
*** agopi has joined #oooq22:51
*** tosky has quit IRC23:04

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!