Thursday, 2019-09-19

*** aakarsh has joined #oooq00:40
*** Goneri has quit IRC00:45
*** pierrepr1netti has joined #oooq00:51
*** pierreprinetti has quit IRC00:54
*** dsneddon has joined #oooq01:12
*** dsneddon has quit IRC01:17
*** rlandy|bbl is now known as rlandy02:01
*** rlandy has quit IRC02:01
*** apetrich has quit IRC02:10
*** dsneddon has joined #oooq02:18
*** dsneddon has quit IRC02:23
*** skramaja has joined #oooq02:49
*** ykarel|away has joined #oooq03:11
*** weshay has quit IRC03:27
*** udesale has joined #oooq04:04
*** surpatil has joined #oooq04:17
*** dsneddon has joined #oooq04:19
*** dsneddon has quit IRC04:24
*** ykarel|away has quit IRC04:30
*** ratailor has joined #oooq04:49
*** ykarel|away has joined #oooq04:54
*** ykarel|away is now known as ykarel04:57
*** jtomasek has joined #oooq05:02
*** soniya29 has joined #oooq05:47
*** marios has joined #oooq05:56
*** kopecmartin|off is now known as kopecmartin06:18
*** dsneddon has joined #oooq06:20
*** holser has joined #oooq06:21
*** dsneddon has quit IRC06:24
*** jaosorior has joined #oooq06:43
*** dsneddon has joined #oooq06:45
*** udesale has quit IRC06:57
*** udesale has joined #oooq06:58
*** matbu has joined #oooq07:14
*** apetrich has joined #oooq07:15
*** udesale has quit IRC07:18
*** udesale has joined #oooq07:19
*** tosky has joined #oooq07:23
*** tesseract has joined #oooq07:26
*** bogdando has joined #oooq07:27
*** jpena|off is now known as jpena07:33
*** akahat has joined #oooq07:35
*** amoralej|off is now known as amoralej07:37
*** ykarel is now known as ykarel|lunch07:38
*** akahat has quit IRC07:47
*** dsneddon has quit IRC07:49
*** chem has joined #oooq07:49
*** jaosorior has quit IRC07:51
*** akahat has joined #oooq07:52
*** zbr is now known as zbr|ruck07:57
zbr|ruckmorning!07:58
*** dsneddon has joined #oooq08:00
*** soniya29 has quit IRC08:04
*** chem has quit IRC08:13
*** chem has joined #oooq08:13
*** chem has quit IRC08:14
*** chem has joined #oooq08:15
arxcruz|ruckzbr|ruck: morning08:20
*** arxcruz|ruck is now known as arxcruz|rover08:20
arxcruz|roverman, my daughter cried the whole night08:21
arxcruz|roveri'm just the dust08:21
*** ykarel|lunch is now known as ykarel08:21
*** chem has quit IRC08:22
*** chem has joined #oooq08:23
*** soniya29 has joined #oooq08:26
*** chem has quit IRC08:26
*** chem has joined #oooq08:27
zbr|ruckarxcruz|rover: keep in mind tomorrow, I will leave ~1pm and I will be back only on Tuesday, fully offline.08:27
arxcruz|roverzbr|ruck: HOW DARE YOU?!?!08:28
*** derekh has joined #oooq08:30
*** dtantsur|afk is now known as dtantsur08:32
*** chem has quit IRC08:35
*** chem has joined #oooq08:35
*** dsneddon has quit IRC09:04
*** akahat has quit IRC09:11
arxcruz|rovermarios: so, it run tempest but it fails, it's a progress, i check validate-tempest and see some differences, i'm trying now09:16
* arxcruz|rover is very hopefull09:16
*** akahat has joined #oooq09:17
mariosarxcruz|rover: ok sounds good shout if you need reviews etc09:19
mariosarxcruz|rover: did you end up speaking with beagles or jakub?09:19
arxcruz|rovermarios: no, but after i set the physical_network to empty string it pass and run tempest, but it fails, i'm investigating why09:20
arxcruz|roverbut it pass the step that wasn't passing before09:20
*** akahat has quit IRC09:49
*** akahat has joined #oooq10:01
*** udesale has quit IRC10:01
zbr|ruckif you use pytest, you may want to look install pytest-sugar locally10:03
*** ykarel is now known as ykarel|afk10:27
*** udesale has joined #oooq10:32
*** udesale has quit IRC10:37
*** udesale has joined #oooq10:37
zbr|rucki wonder if anyone produce a gmail filter that removes a full thread when "Build succeeded (gate pipeline)." is received.10:45
zbr|ruckafter 1h, i still have 1.7k msg in my "code" folder.10:46
*** akahat has quit IRC10:52
*** akahat has joined #oooq10:52
*** dsneddon has joined #oooq11:00
*** udesale has quit IRC11:01
*** jpena is now known as jpena|lunch11:04
*** dsneddon has quit IRC11:05
*** amoralej is now known as amoralej|lunch11:07
*** weshay_ has joined #oooq11:59
sshnaidmzbr|ruck, arxcruz|rover weshay_ panda mtg?12:00
arxcruz|roversshnaidm: joining12:00
weshay_ya.. joining.. sorry.. my laptop picked up a ipv6 ip and wouldn't connect to irc12:00
sshnaidmweshay_, lucky12:01
*** ykarel|afk is now known as ykarel12:02
weshay_zbr|ruck, hello12:03
arxcruz|roverzbr|ruck: meeting12:03
*** rfolco has joined #oooq12:04
weshay_http://dashboard-ci.tripleo.org/d/si1tipHZk/jobs-exploration?orgId=1&from=now-90d&to=now&var-influxdb_filter=job_name%7C%3D%7Ctripleo-ci-centos-7-containers-multinode&var-influxdb_filter=branch%7C%3D%7Cmaster&var-influxdb_filter=result%7C%3D%7CSUCCESS12:05
weshay_sshnaidm, arxcruz|rover panda ^12:06
*** Goneri has joined #oooq12:07
*** amoralej|lunch is now known as amoralej12:08
weshay_sshnaidm, arxcruz|rover panda zbr|ruck sorry https://redhat.bluejeans.com/4113567798/12:09
weshay_not sure what's up12:09
weshay_hello?12:10
*** akahat has quit IRC12:18
weshay_zbr|ruck, ping12:19
*** jpena|lunch is now known as jpena12:23
zbr|rucki am back12:24
weshay_https://redhat.bluejeans.com/4113567798/12:24
weshay_zbr|ruck, ^12:24
*** jfrancoa has joined #oooq12:28
chandankumarsshnaidm: panda can you check https://review.rdoproject.org/r/#/c/22304/ if we missed any job to enable registry login?12:28
weshay_zbr|ruck, http://dashboard-ci.tripleo.org/d/si1tipHZk/jobs-exploration?orgId=1&from=now-90d&to=now&var-influxdb_filter=job_name%7C%3D%7Ctripleo-ci-centos-7-containers-multinode&var-influxdb_filter=branch%7C%3D%7Cmaster12:29
pandachandankumar: does it work ?12:29
chandankumarpanda: currently login is working it does not got failed12:30
rfolcoweshay_, we don't have any user story/epic for branching12:31
sshnaidmchandankumar, tripleo-ci-base-ovb-periodic ?12:31
chandankumarpanda: it depends on this task https://tree.taiga.io/project/morucci-software-factory/us/270312:31
chandankumarrfolco: we have epic for branching12:31
chandankumarrfolco: https://docs.google.com/document/d/1LP23IwpCJLKCMe3wzDOvtqZZ79-eCxhxHVf7NOPeQbI/edit12:31
*** rlandy has joined #oooq12:31
rfolcochandankumar, can you pls create an user story and epic in taiga ?12:32
chandankumarrfolco: added there https://tree.taiga.io/project/tripleo-ci-board/epic/127912:32
chandankumarweshay_: 1:1 timing12:32
chandankumarsshnaidm: sending a patch12:34
sshnaidmchandankumar, need to look which jobs use it, on mtg now12:34
chandankumarsshnaidm: ok12:34
weshay_chandankumar, ok.. you can join12:36
weshay_or we can do it later12:36
zbr|ruckarxcruz|rover: https://bluejeans.com/2655417928 --- debugging master timeouts session12:37
weshay_https://redhat.bluejeans.com/4113567798/12:37
*** brault has joined #oooq12:37
sshnaidmzbr|ruck, arxcruz|rover just fyi, you can rerun any periodic jobs like that: https://review.rdoproject.org/r/#/c/22163/2/zuul.yaml12:37
weshay_chandankumar, https://github.com/sshnaidm/jcomparison12:38
rfolcorlandy, panda: can you pls review ? this includes the fix for promoter server (docker-ce) https://review.rdoproject.org/r/#/c/21872/12:43
*** aakarsh has quit IRC12:46
pandarfolco: I don't know what happens after this merges12:48
rlandyrfolco: this fixes the actual promoter or the staging one?12:48
rfolcorlandy, production promoter12:48
rlandyrfolco: wher eis clena up called?12:49
rlandywhere is clean up called12:49
rfolcoafter calling promotion_run12:49
rfolcoin playbooks12:49
rlandyhttps://review.rdoproject.org/r/#/c/21872/24/ci-scripts/infra-setup/roles/promoter/molecule/default/playbook.yml12:50
rlandyok12:50
*** brault has quit IRC12:50
*** brault has joined #oooq12:51
rfolcorlandy, panda risks of breakage are in dlrn-promoter.sh, and in container-push.yml12:51
rfolcoand the fix is in tasks/main.yml12:52
rlandyand where is setup_staging set to true?12:55
zbr|ruckweshay_: i think on master can and containers push error we are hitting the same https://bugs.launchpad.net/tripleo/+bug/177163412:55
openstackLaunchpad bug 1771634 in tripleo "periodic: container build jobs are failing when pushing to rdo registry (500, 504, read timeout)" [Critical,Fix released] - Assigned to Quique Llorente (quiquell)12:55
*** brault has quit IRC12:55
zbr|ruckUnknown error when pushing -ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out.12:55
sshnaidmweshay_, https://review.opendev.org/#/c/683039/12:55
sshnaidmweshay_, works, takes about 55M :(\12:56
rlandyhttps://github.com/rdo-infra/ci-config/blob/master/playbooks/staging.yml12:56
rlandyok12:56
rlandyhttps://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/roles/promoter/molecule/default/playbook.yml12:57
rfolcoplanning mtg12:59
rfolcohttps://etherpad.openstack.org/p/tripleo-ci-squad-meeting13:00
rfolcopinglist: marios, sshnaidm, weshay, panda, rlandy, arxcruz, rfolco, chandankumar, zbr, kopecmartin13:00
rfolcohttps://tree.taiga.io/project/tripleo-ci-board/taskboard/unified-sprint-1613:00
*** dsneddon has joined #oooq13:01
weshay_zbr|ruck, arxcruz|rover do not join the scrum.. please focus on ruck/rover13:01
arxcruz|roverweshay_: ack13:01
rfolcorlandy, chandankumar kopecmartin ping planning mtg13:02
*** dsneddon has quit IRC13:05
*** dtantsur is now known as dtantsur|bbl13:14
*** jaosorior has joined #oooq13:18
*** brault has joined #oooq13:18
*** brault has quit IRC13:20
*** brault has joined #oooq13:20
*** aakarsh has joined #oooq13:20
*** udesale has joined #oooq13:23
*** udesale has quit IRC13:24
*** udesale has joined #oooq13:25
*** recheck has quit IRC13:28
*** recheck has joined #oooq13:29
*** derekh has quit IRC13:29
*** ratailor has quit IRC13:51
recheck[gh-molecule] Molecule Modularized → https://github.com/ansible/molecule/releases/tag/3.0a313:58
*** derekh has joined #oooq13:58
*** tesseract has quit IRC14:00
*** saneax has joined #oooq14:01
*** Vorrtex has joined #oooq14:04
*** soniya29 has quit IRC14:04
arxcruz|roverzbr|ruck: http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build-push/62071b1/ passes14:04
arxcruz|roverzbr|ruck: the other jobs are running14:04
*** saneax has quit IRC14:04
arxcruz|roverweshay_: ^14:04
*** saneax has joined #oooq14:05
*** surpatil has quit IRC14:05
zbr|ruckarxcruz|rover: yeah. it passed but i am still waiting for an answer from rdo, i cannot rely on luck to get a promotion.14:06
zbr|rucki need to check if we have any retry guard put on push, we should14:06
zbr|ruckif the registry is down for 5-10mins, the job should not fail.14:06
arxcruz|roverzbr|ruck: yeah, now we need to wait the other jobs, the failure this morning doesn't seems to be an consistent one14:07
*** saneax has quit IRC14:12
*** ykarel is now known as ykarel|afk14:13
*** ykarel|afk is now known as ykarel|away14:22
*** jfrancoa has quit IRC14:40
*** ykarel|away has quit IRC14:41
*** brault has quit IRC14:46
*** dtantsur|bbl is now known as dtantsur14:54
*** tosky has quit IRC14:57
*** tosky has joined #oooq14:58
*** bogdando has quit IRC14:59
*** dsneddon has joined #oooq15:00
*** skramaja has quit IRC15:03
marioszbr|ruck: rfolco panda rlandy (if/when you next have a minute - manifests molecule job - any idea what i can do about the 'sudo requires password' ? do i need to add that to the molecule user, or something else? task logs there http://logs.rdoproject.org/02/22002/17/check/rdo-tox-molecule/5f37347/tox/reports.html the failing task is that one15:07
marioshttps://review.rdoproject.org/r/#/c/22002/17/ci-scripts/infra-setup/roles/promoter/tasks/main.yml@15815:07
marioszbr|ruck: but am confused as that is NOT the first 'become' in the file maybe i'm missing something else15:08
rlandymarios: https://review.rdoproject.org/r/#/c/22255/15/ci-scripts/infra-setup/roles/promoter/tasks/setup_staging.yaml uses:15:08
marioshmmm 'delegate_to' maybe is the difference/issue15:08
rlandyhttps://review.rdoproject.org/r/#/c/22255/15/ci-scripts/infra-setup/roles/promoter/tasks/setup_staging.yaml15:08
rlandy  become: true15:09
rlandy  become_user: "{{ promoter_user }}"15:09
mariosrlandy: ack lemme try15:09
rlandymarios: honestly I am not sure how that works :)15:09
rlandylike two becomes15:09
rlandybut so far so good15:09
mariosrlandy: the problm is i'm running on localhost with delegate_to15:09
mariosrlandy: which is why the other become in that file seems to work OK ... depends if promoeter_user has becomes i think it should lets see15:10
rlandymarios: k - let me know if you need another solution15:10
mariosrlandy: thx15:10
*** dsneddon has quit IRC15:11
*** dsneddon has joined #oooq15:11
* rlandy just setting up weekend pipeline - will look more afterwards15:11
*** jfrancoa has joined #oooq15:13
*** derekh has quit IRC15:16
*** derekh has joined #oooq15:18
*** derekh has quit IRC15:18
*** kopecmartin is now known as kopecmartin|off15:35
rlandyweshay_, https://review.rdoproject.org/r/22413 Add a weekend pipeline to run older releases  - there is some confusion about what the first day of the week is :)15:39
rlandycron seems to think it's sunday and zuul seems to think it's monday15:39
rlandyI left a doc note15:39
rlandybut it could land up being an 'off-by-one-day' error15:40
weshay_rlandy,  saturday would be the best day imho15:40
weshay_sunday is a work day in israel15:40
rlandyweshay: so the weekend job kicks only once a week?15:42
rlandyI have it kicking twice15:42
rlandyonce on saturday and once on sunday15:42
weshay_twice is great15:42
weshay_perfect15:42
weshay_rlandy++15:42
*** ykarel|away has joined #oooq15:46
*** Goneri has quit IRC15:47
*** ykarel|away has quit IRC15:54
*** marios has quit IRC16:02
*** holser has quit IRC16:13
*** ykarel|away has joined #oooq16:18
*** chem has quit IRC16:18
rfolcorlandy, will w+ https://review.rdoproject.org/r/#/c/21872/16:20
rfolcorlandy, and watch promoter logs16:20
rlandyrfolco: ack16:21
rlandywe can revert of need be16:21
rfolcok16:21
*** dtantsur is now known as dtantsur|afk16:22
*** weshay_ is now known as weshay16:23
*** udesale has quit IRC16:27
*** udesale has joined #oooq16:28
*** Goneri has joined #oooq16:29
weshaysshnaidm, you still here? /me trying jcompare16:39
weshayw/ two ovb jobs16:39
sshnaidmweshay, ya16:39
weshayhttp://logs.rdoproject.org/56/682956/6/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-stein-branch/37d7871/ and16:39
weshayhttp://logs.rdoproject.org/56/682956/6/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/43e2b84/16:39
weshaynothing can be fetched16:39
chandankumarsshnaidm: I am thinking of reusing your project and add remaining stuff there if it is ok?16:40
chandankumarfor ci job compare16:40
sshnaidmweshay, because we don't generate html in ovb jobs, there is sqlite file that I'll add its processing soon16:41
sshnaidmweshay, today you can compare all multinode job, though16:41
*** Goneri has quit IRC16:41
weshaysshnaidm, really! /me tries.. /me looked at commits.. didn't see that16:42
sshnaidmchandankumar, yeah, need to sync about that16:43
weshaywha hoooo!16:44
weshayworking16:44
chandankumarsshnaidm: let's sync on monday i will drop a meeting invite tomorrow16:46
*** holser has joined #oooq16:47
sshnaidmchandankumar, ack16:48
*** udesale has quit IRC16:52
weshaysshnaidm, /me assumes that's taking the 10 slowest tasks from ara?16:52
weshaysshnaidm, since most of the containers are pulled in the undercloud deployment now.. I think we need ara there16:58
weshaysshnaidm, this is interesting.. collect-logs : Gather the logs to /tmp290806+516 sec177%17:00
sshnaidmweshay, it prints most different tasks - which their time difference is more than 10 sec17:00
sshnaidmweshay, yeah, but for that we need ara server at least, working on that17:01
*** Goneri has joined #oooq17:01
sshnaidmweshay, we can't use sqlite as it doesn't support parallel writing17:01
zbr|ruckme out17:07
*** zbr|ruck is now known as zbr17:07
*** Goneri has quit IRC17:14
*** ChanServ sets mode: +o weshay17:18
weshaysshnaidm, perhaps we can have ara do different things on different jobs17:20
sshnaidmweshay, ?17:20
rlandyzbr: just fyi - merging the removal of f28 depredcated job definitions17:20
weshaysshnaidm, want to chat for a min?17:22
*** jpena is now known as jpena|off17:22
weshayrlandy, thanks17:22
sshnaidmweshay, sure17:23
weshaysshnaidm, https://meet.google.com/ygj-btmp-brh17:23
*** amoralej is now known as amoralej|off17:26
*** ykarel|away has quit IRC17:34
*** holser has quit IRC17:36
rfolcopanda, rlandy I am no longer seeing any failures on the promoter, I assume the bug is fixed and promoter is healthy.17:43
rfolcoarxcruz|rover, zbr: let me know if anything suspicious about the promoter ^17:44
weshayreviews on https://review.rdoproject.org/r/#/c/22413/17:46
rlandygood18:01
rlandythank you18:01
rlandymerging https://review.rdoproject.org/r/#/c/22415/18:02
*** brault has joined #oooq18:09
*** brault has quit IRC18:10
*** Goneri has joined #oooq18:12
*** brault has joined #oooq18:55
*** brault has quit IRC18:56
*** brault has joined #oooq18:56
*** holser has joined #oooq19:14
*** brault has quit IRC19:16
rfolcorlandy, is there anything in rdo equivalent to https://github.com/openstack/tripleo-ci/blob/master/playbooks/tripleo-ci/post.yaml19:34
rfolcorlandy, I wanted to use it to collect our *.log files but I don't want to reparent to tripleo-ci-base for that and run unnecessary stuff19:35
*** jfrancoa has quit IRC19:37
rlandysec - looking19:43
rlandypanda: you around?19:54
rlandydesign question about testing promoter19:54
rlandyis it reasonable to consider the staging promoter fixtures also as web hosted?19:55
rlandyrfolco: idk what piece of the logging you are interested in19:56
rfolcorlandy, for example dlrnapi.log, promoter_logs/* etc19:59
rlandyline number?19:59
rlandyrfolco: ^^?20:00
rfolcowill copy those to workspace first...20:00
rfolcoI still don't have them20:00
rlandy              timeout --preserve-status -s 15 \20:00
rlandy                -k {{ [collect_timeout_sec|int, 60]|sum|string }} {{ collect_timeout_sec|string }} \20:00
rlandy                bash {{ ansible_user_dir }}/workspace/logs/collect_logs.sh20:00
rlandy?20:00
rlandysorry, I ma not answering this too well :(20:00
rfolcorlandy, right I think my answer is yes you should create a post playbook in ci-config and copy logs20:01
rlandyrfolco: easiest is to do it put up a review and then we can compare20:02
rfolcocoz I don't want to parent off triple-ci-base job in tripleo-ci upstream20:02
rfolcook20:02
rlandyhard to know how much you need atm20:02
rfolcothx rlandy20:02
*** holser has quit IRC20:20
*** holser has joined #oooq20:34
sshnaidmweshay, added sqlite20:41
sshnaidmweshay, container will update soon20:41
*** sshnaidm is now known as sshnaidm|off20:42
weshaysshnaidm|off, k20:42
weshaysshnaidm|off, thanks20:42
*** brault has joined #oooq20:51
*** brault has quit IRC20:55
*** Vorrtex has quit IRC20:58
*** brault has joined #oooq21:02
*** brault has quit IRC21:08
*** pierreprinetti has joined #oooq21:11
*** pierrepr1netti has quit IRC21:12
*** aakarsh has quit IRC21:13
*** brault has joined #oooq21:17
weshayrlandy, ping21:21
weshayrlandy,  look out for a ping re: Emilien and zuul secrets :)21:22
*** holser has quit IRC21:48
rlandyweshay: k21:51
rlandydidn;t see anything on pm21:51
*** brault has quit IRC22:22
*** rlandy is now known as rlandy|biab22:30
*** ksambor has quit IRC22:37
*** holser has joined #oooq22:38
*** Tengu has quit IRC22:51
*** holser has quit IRC22:59
*** Tengu has joined #oooq23:03
*** tosky has quit IRC23:12
*** rlandy|biab is now known as rlandy|bbl23:18

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!