Wednesday, 2020-11-25

*** frenzy_friday has joined #oooq01:16
*** holser has quit IRC01:40
*** Goneri has quit IRC02:03
*** skramaja has joined #oooq02:52
*** udesale has joined #oooq03:45
*** ykarel|away has joined #oooq04:13
*** ykarel|away has quit IRC04:34
*** raukadah is now known as chandankumar04:52
*** ykarel|away has joined #oooq05:00
*** ksambor has quit IRC05:05
*** ykarel|away is now known as ykarel05:07
*** saneax has joined #oooq07:01
*** ksambor has joined #oooq07:02
*** ysandeep|away is now known as ysandeep07:15
*** marios has joined #oooq07:20
*** ykarel_ has joined #oooq07:37
*** ykarel has quit IRC07:40
*** amoralej|off is now known as amoralej08:04
chandankumararxcruz|rover: hello, please update this patch https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/746509 when free :-)08:08
chandankumarmarios: hello, needs your PTL power here https://review.opendev.org/c/openstack/releases/+/76399008:09
arxcruz|roverchandankumar: it needs more work on that08:09
marioschandankumar: ack yeah its in my list will check in few08:09
chandankumararxcruz|rover: want me to take over since you are on rr?08:09
arxcruz|roverchandankumar: nah, i'm good :)08:09
chandankumararxcruz|rover: ok :-)08:10
*** ykarel_ is now known as ykarel08:34
chandankumararxcruz|rover: soniya29 akahat kopecmartin Agenda for today's tempest meeting https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ?view#2020-11-25 please add or remove to that, thanks :-)08:46
arxcruz|roverchandankumar: I have prod call in the same time08:47
arxcruz|roverwon't participate08:47
chandankumararxcruz|rover: no problem, kopecmartin weshay|ruck and others can attend, I will update the summary on hackmd for you.08:48
*** tosky has joined #oooq08:50
*** jpena|off is now known as jpena08:55
*** jbadiapa has joined #oooq09:04
*** derekh has joined #oooq09:13
*** gchamoul has quit IRC09:51
*** gchamoul has joined #oooq09:52
*** dtantsur|afk is now known as dtantsur10:33
*** ysandeep is now known as ysandeep|afk11:22
*** rfolco|ruck has joined #oooq11:36
*** udesale_ has joined #oooq11:45
*** udesale has quit IRC11:48
*** jlopezgu has joined #oooq11:49
*** ysandeep|afk is now known as ysandeep12:26
*** jpena is now known as jpena|lunch12:35
*** jlopezgu has quit IRC12:51
*** rlandy has joined #oooq12:52
chandankumarweshay|ruck: soniya29 akahat kopecmartin tempest meeting time13:00
akahatchandankumar, joining.13:01
*** TrevorV has joined #oooq13:02
*** ysandeep is now known as ysandeep|brb13:08
*** amoralej is now known as amoralej|lunch13:09
weshay|ruckrfolco|ruck, for the sprint summary.. please add https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/762405/13:10
rfolco|ruckweshay|ruck, ack13:10
chandankumararxcruz|rover: notes here https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ?view#2020-11-2513:15
rlandyok people - new gerrit - how do I get rid of the depends-on???13:15
rlandyhelp a sister out here13:15
weshay|ruckarxcruz|rover, fyi.. can you check if we need to bug and skip https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master/bae0f83/logs/undercloud/var/log/tempest/stestr_results.html.gz  maybe we have.. this run is older.. /me waiving for a promotion13:16
chandankumarrlandy: click on commit message file and you will see EDIT option  on right side13:16
chandankumarand then save and close13:17
chandankumaryou are done13:17
rlandychandankumar++ bless you sir13:17
chandankumarrlandy: :-)13:17
arxcruz|roverweshay|ruck: we already have a bug and skip for those13:18
weshay|ruckarxcruz|rover++13:18
weshay|ruckthanks13:18
*** ksambor has quit IRC13:28
*** jpena|lunch is now known as jpena13:35
ykarelrfolco|ruck, reopened https://bugs.launchpad.net/tripleo/+bug/1905386 with a suggestion there13:38
openstackLaunchpad bug 1905386 in tripleo "Centos 7 train jobs failing with pymod2pkg requiring python 3.6" [Critical,Triaged]13:38
ykareli seen 1 more python2 related error https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_23c/763961/1/check/tripleo-ci-centos-7-containers-multinode/23c75fe/job-output.txt13:40
ykarelso it's good to move away from python2 where ever possible, for ^ error i proposed https://review.opendev.org/c/openstack/tripleo-ci/+/764120/13:40
*** ysandeep|brb is now known as ysandeep13:55
*** sshnaidm is now known as sshnaidm|afk13:56
*** derekh has quit IRC14:02
*** derekh has joined #oooq14:02
*** amoralej|lunch is now known as amoralej14:05
weshay|ruckmatbu, https://docs.google.com/document/d/1bnhfx7F8Ue4-sLdIjF6fju3tWBg7hFH3r2Kad-IaXD0/edit#heading=h.8j93gcg14nx714:08
rfolco|ruckykarel, ok thanks14:11
rfolco|ruckarxcruz|rover, please help merge these https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/761019 and https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/76102114:11
rfolco|ruckchandankumar, ^14:11
chandankumarrfolco|ruck: pinged osa people14:15
matbuweshay|ruck: ack ok done, i have much more mtgs now :D14:17
rfolco|ruckarxcruz|rover, chandankumar soniya29 please review this one https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/76405514:19
rfolco|ruckenables stackviz when tempest fails14:19
chandankumarrfolco|ruck: can we remove wip from there?14:26
rfolco|ruckchandankumar, new gerri thing14:27
rfolco|ruckdoes it go away when you review ?14:28
chandankumarneed to check that14:28
rfolco|ruckoh mark as active14:29
rfolco|ruckfound it14:29
rfolco|ruckat the top14:29
rfolco|ruckchandankumar, ^14:29
rfolco|rucknow it's active14:29
chandankumarrfolco|ruck: cool14:29
*** sshnaidm|afk is now known as sshnaidm14:34
*** Goneri has joined #oooq14:56
*** saneax has quit IRC15:03
* zbr realises it is already the compliance time of the year...15:10
zbrlucky panda, he managed to get away without these15:11
chandankumarweshay|ruck: please have a look at this one https://bugzilla.redhat.com/show_bug.cgi?id=189947615:42
openstackbugzilla.redhat.com bug 1899476 in distribution "[RFE] Integrate ansible-role-collect-logs role with sosreport" [High,Assigned] - Assigned to shrjoshi15:42
*** skramaja has quit IRC15:49
*** ykarel is now known as ykarel|away15:52
chandankumararxcruz|rover: I am taking over this task https://projects.engineering.redhat.com/browse/TRIPLEOCI-248 if ok? rdo job to push containers to rdo registry if ok?15:53
arxcruz|roverchandankumar: sure, you can add on the script15:54
arxcruz|roveractually, i should had done this :/ sorry15:54
chandankumararxcruz|rover: thanks!15:55
arxcruz|roverchandankumar: there's a script running on the toolbox pulling containers from rdo registry to quay15:55
arxcruz|roverchandankumar: if you require access to something please let me know :)15:55
chandankumararxcruz|rover: yes may be tomorrow I will ping you15:55
arxcruz|roverchandankumar: sure :)15:55
weshay|ruckthanks chandankumar I'll read through the bug16:05
weshay|rucksshnaidm, you avail on nov 30, to talk to fi folks?16:08
sshnaidmweshay|ruck, yep16:08
weshay|rucksshnaidm, k.. thank you!16:09
*** ysandeep is now known as ysandeep|away16:12
rlandyweshay|ruck: marios: chandankumar: taking a list stab at getting ipa voting again ... https://review.opendev.org/c/openstack/tripleo-ci/+/762766 Make standalone-ipa jobs voting again - minus train16:14
*** ykarel|away has quit IRC16:14
rlandyI know that stats are not great atm16:14
rlandybut we are still waiting on martin's backports16:14
weshay|ruckrlandy, fyi.. I think that job just failed in the periodic master line16:14
rlandyweshay|ruck: looking16:14
rlandyit shoudl be passing unless the fix is blocked somewhere16:15
rlandyweshay|ruck: on the upside, the failures don't show dns issues16:15
rlandyso I think we are by that one16:15
rlandychecking periodics16:16
rlandyoy vey ... https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-master16:17
rlandydebugging16:17
rlandy The task includes an option with an undefined variable. The error was: "hostvars['subnode-1']" is undefined16:17
rlandyweird16:18
chandankumararxcruz|rover: update the url at these two places https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers/defaults/main.yaml#L42 and https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/container-build/tasks/main.yaml#L2116:18
chandankumarrlandy: due to change in nodeset16:18
rlandychandankumar: https://github.com/rdo-infra/rdo-jobs/commit/086a620b8c98c1ba558255d0a62959e7e72dbdab?16:19
mariosrlandy: ack16:19
rlandymarios: how does that change subnode-1/16:21
mariosrlandy: i guess you meant chandankumar ^16:21
rlandyanyone who knows?16:22
*** udesale_ has quit IRC16:23
rlandy Label: upstream-centos-8-vexxhost16:23
rlandyah wrong nodeset16:23
chandankumarrlandy: sorry for the mess16:24
rlandychandankumar: np - will fix16:24
rlandyeasy fix16:24
rlandyone sec - putting up patch16:24
arxcruz|roverhttps://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/76421716:27
rlandychandankumar: https://review.rdoproject.org/r/31204 Enure multinode ipa job uses two nodes on vexxhost16:29
arxcruz|roverhttps://review.opendev.org/c/openstack/tripleo-ci/+/76421816:29
chandankumarrlandy: weshay|ruck ^^ please +w it16:29
chandankumarsee ya soon16:30
*** chandankumar is now known as raukadah16:30
zbrchandankumar: rlandy: https://review.rdoproject.org/r/#/c/30803/ -- simplifies how we run molecule, see the new molecule report (no longer using pytest-html)16:30
zbrexample: https://logserver.rdoproject.org/03/30803/6/check/mol-get_hash-centos-7/f8fd6ad/tox/reports.html16:30
rlandyzbr: nice16:31
zbris anew feature, if you define MOLECULE_REPORT=foo.html and run it, it will save its own output in that file (with colors)16:31
zbrno longer needing other wrappers for producing the report.16:31
zbrnext step would be to run "tox -e schenarioname" -- in fact is already possible with minimal changes.16:33
arxcruz|roverraukadah: weshay|ruck i update the patch with the bug16:35
zbrnot sure how many of you are aware that if they do "pip install tox-ansible", tox will start to recognize each molecule scenario as a "virtual" tox environment, run "tox -va" after to see the difference.16:38
*** irclogbot_3 has quit IRC16:41
*** irclogbot_2 has joined #oooq16:46
*** marios is now known as marios|out17:03
*** marios|out has quit IRC17:15
*** amoralej is now known as amoralej|off17:22
raukadahrfolco|ruck: we pushed all containers to ceph namespace on rdo registry17:23
raukadahit will work17:23
raukadahrfolco|ruck: check the fallback playbook https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/container-build/tasks/non_tripleo_containers.yml#L4617:23
rfolco|rucksorry17:24
rfolco|ruckthats right17:24
weshay|ruckraukadah++17:24
rfolco|ruckI beg your forgiveness arxcruz|rover17:24
raukadahrfolco|ruck: no need to sorry sir17:25
raukadahrfolco|ruck: I will think of more robust solution may be in upcoming days17:25
rfolco|ruckraukadah, ok cool17:25
* raukadah goes back to watch crown17:26
arxcruz|roverrfolco|ruck: i'll think if i can forgive you17:28
rlandyarxcruz|rover: rfolco|ruck: weshay|ruck: multinide-ipa job should be passing again in periodic with https://review.rdoproject.org/r/#/c/31204/ merge17:29
rlandyping if you still see issues17:29
rlandyI am trying to get that job consistently passing17:29
rfolco|ruckthats good rlandy thx17:29
rlandygoal for end of year17:29
weshay|ruckrlandy++ very nice17:38
weshay|ruckthanks for taking care of that17:38
*** derekh has quit IRC18:01
*** jpena is now known as jpena|off18:03
*** jbadiapa has quit IRC18:14
*** sshnaidm has quit IRC18:23
*** sshnaidm has joined #oooq18:24
*** sshnaidm has quit IRC18:28
*** sshnaidm has joined #oooq18:28
weshay|ruckrfolco|ruck, /me updated the google spreadsheet.. great work :)18:43
rfolco|ruckweshay|ruck, cool let me know if anything I am missing. I did a quick pass on gate failures, 90% is quay.io related18:44
weshay|ruckrfolco|ruck, ya.. not worth looking at today18:45
weshay|rucklolz18:45
*** holser has joined #oooq18:57
*** holser has quit IRC19:05
*** holser has joined #oooq19:05
*** holser has quit IRC19:06
*** holser has joined #oooq19:07
*** holser has quit IRC19:08
*** holser has joined #oooq19:09
*** holser has quit IRC19:10
*** holser has joined #oooq19:11
*** holser has quit IRC19:11
*** holser has joined #oooq19:13
sshnaidmraukadah, fyi https://bugs.launchpad.net/tripleo/+bug/190561519:13
openstackLaunchpad bug 1905615 in tripleo "Failover of container registries fails" [Critical,Triaged] - Assigned to chandan kumar (chkumar246)19:13
weshay|rucksshnaidm, I think we had one written up on that.. /me looks19:24
*** dtantsur is now known as dtantsur|afk19:33
*** rfolco|ruck has quit IRC21:08
rlandywe did21:10
rlandyhttps://bugs.launchpad.net/tripleo/+bug/190388221:11
openstackLaunchpad bug 1903882 in tripleo "tripleo-ci-centos-8-content-provider failed on TASK [container-build : Pull failed containers from RDO registry] with error was: 'dict object' has no attribute 'split'" [Critical,Triaged]21:11
rlandyweshay|ruck: ^^21:12
weshay|ruckthanks!21:12
rlandysshnaidm: weshay|ruck: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/76240021:12
weshay|ruckI need to get my tool to triage alert bugs21:12
rlandyweshay|ruck: there are some comments on that patch21:13
rlandyit should address the problem21:13
weshay|ruck++21:13
rlandywe can retest it to be sure21:14
rlandyalso yatin's comment21:14
rlandyre: non-tripleo containers21:14
rlandyprobably also needs to be checked21:14
*** TrevorV has quit IRC21:48
*** rlandy has quit IRC23:07
*** tosky has quit IRC23:51

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!