Thursday, 2022-03-31

*** dviroel|out is now known as dviroel00:05
opendevreviewDouglas Viroel proposed openstack/tripleo-quickstart-extras master: Include dependency vars on subnodes  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/83497200:06
opendevreviewDouglas Viroel proposed openstack/tripleo-ci master: Adds compose-repos pre playbook on base jobs  https://review.opendev.org/c/openstack/tripleo-ci/+/83451500:15
opendevreviewMerged openstack/tripleo-ci master: Call container-registry role at required job level  https://review.opendev.org/c/openstack/tripleo-ci/+/83510100:17
*** dviroel is now known as dviroel|out00:20
*** rlandy|bbl is now known as rlandy|out01:17
opendevreviewMerged openstack/puppet-tripleo stable/wallaby: Format [keystone_authtoken] memcache_servers when IPv6 is used  https://review.opendev.org/c/openstack/puppet-tripleo/+/83414201:39
*** soniya29 is now known as soniya29|rover04:19
*** marios is now known as marios|ruck04:58
*** ysandeep|out is now known as ysandeep05:05
*** pojadhav|afk is now known as pojadhav05:11
opendevreviewHarald Jensås proposed openstack/puppet-tripleo stable/train: Format [oslo_cache] memcache_server when IPv6 is used  https://review.opendev.org/c/openstack/puppet-tripleo/+/83488406:02
opendevreviewRajesh Tailor proposed openstack/tripleo-heat-templates master: Deprecate `NovaVGPUTypesDeviceAddressesMapping parameter  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83544006:04
opendevreviewHarald Jensås proposed openstack/puppet-tripleo stable/train: Format [oslo_cache] memcache_server when IPv6 is used  https://review.opendev.org/c/openstack/puppet-tripleo/+/83488406:04
opendevreviewHarald Jensås proposed openstack/puppet-tripleo stable/train: Format [oslo_cache] memcache_server when IPv6 is used  https://review.opendev.org/c/openstack/puppet-tripleo/+/83488406:06
opendevreviewyatin proposed openstack/tripleo-quickstart-extras master: [DNM] enable ovn debug  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/83559706:07
opendevreviewHarald Jensås proposed openstack/puppet-tripleo stable/train: Format [keystone_authtoken] memcache_servers when IPv6 is used  https://review.opendev.org/c/openstack/puppet-tripleo/+/83488306:17
opendevreviewHarald Jensås proposed openstack/puppet-tripleo stable/train: Format [oslo_cache] memcache_server when IPv6 is used  https://review.opendev.org/c/openstack/puppet-tripleo/+/83488406:18
opendevreviewHarald Jensås proposed openstack/puppet-tripleo stable/train: Format [oslo_cache] memcache_server when IPv6 is used  https://review.opendev.org/c/openstack/puppet-tripleo/+/83488406:21
opendevreviewMerged openstack/tripleo-docs master: Add initial doc for compose pinning dependency jobs  https://review.opendev.org/c/openstack/tripleo-docs/+/83495706:37
ysandeepmarios|ruck soniya29|rover: Seeing some NODE_FAILURE issues today: https://zuul.opendev.org/t/openstack/builds?result=NODE_FAILURE 06:38
* ysandeep not sure what percentage of jobs are hitting issue, but worth to keep an eye incase we hit more NODE_FAILURE.06:39
ysandeepseeing some more node_failure in running jobs, for ex: https://zuul.opendev.org/t/openstack/status#83550006:43
marios|ruckysandeep: looking thanks06:46
marios|ruckunfortunately no logs ysandeep cant even see which cloud06:49
ysandeepmarios|ruck: thanks! I see you have reached out to infra already, lets see if they can find anything.06:53
*** amoralej|off is now known as amoralej07:13
*** ysandeep is now known as ysandeep|lunch07:23
*** jpena|off is now known as jpena07:34
Tenguuho.... RuntimeError: This module requires ansible-collections-openstack  when trying to provision OC node today. Is this known already?08:09
*** gthiemon1e is now known as gthiemonge08:10
Tengumarios|ruck: -^^does it ring any bell?08:12
Tenguuho. maybe a missing (new) dependency on tripleo-ansible?08:16
Tenguah. better.... it's installed already.08:18
Tenguhmmm.08:18
marios|ruckTengu: not seen that one around 08:19
Tenguweird. collection IS installed, but something is masking it. 08:19
hjensastkajinam: do we need to backport https://review.opendev.org/c/openstack/puppet-tripleo/+/835589 for the memcache backports to land?08:22
Tenguand the affected file (/usr/share/ansible/plugins/modules/metalsmith_instances.py) wasn't change for ages.....08:22
tkajinamhjensas, I believe we don't need that. setup.py stuff only affects ubuntu job but the job has setup.py pinned by global constraints08:25
tkajinamrelease note job in ubuntu08:25
hjensastkajinam: ok, then I won't propose a backport. Thanks!08:26
tkajinamhjensas, ack :-)08:28
opendevreviewSlawek Kaplonski proposed openstack/tripleo-heat-templates stable/wallaby: Don't add conntrack entries for vxlan  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83158908:30
opendevreviewSlawek Kaplonski proposed openstack/tripleo-heat-templates stable/victoria: Don't add conntrack entries for vxlan  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83175008:31
opendevreviewSlawek Kaplonski proposed openstack/tripleo-heat-templates stable/train: Don't add conntrack entries for vxlan  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83173408:33
opendevreviewSergii Golovatiuk proposed openstack/tripleo-upgrade stable/train: Add new scheme of driver removal  https://review.opendev.org/c/openstack/tripleo-upgrade/+/83399909:20
*** ysandeep|lunch is now known as ysandeep10:07
*** soniya29|rover is now known as soniya29|rover|lunch10:08
opendevreviewLukas Bezdicka proposed openstack/tripleo-heat-templates stable/train: [train-only] Use state file for workaround 1925078  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83428510:17
*** rlandy|out is now known as rlandy10:25
*** soniya29|rover|lunch is now known as soniya29|rover10:27
opendevreviewMerged openstack/tripleo-heat-templates stable/train: Memcached: Allow puppet to be aware IPv6 is used  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83459710:37
opendevreviewMerged openstack/tripleo-heat-templates stable/ussuri: Memcached: Allow puppet to be aware IPv6 is used  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83459610:37
opendevreviewMerged openstack/tripleo-quickstart-extras master: Include dependency vars on subnodes  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/83497210:37
marios|rucktkajinam: o/ hey can you please check when you have time https://bugs.launchpad.net/tripleo/+bug/1966625 this blocks tripleo component lines11:03
marios|rucktkajinam: mainly what is the latest/something we can help/push please? 11:03
opendevreviewSaravanan KR proposed openstack/tripleo-heat-templates stable/wallaby: Add nvidia firstboot yaml to tripleo-heat-template  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83494611:06
*** marios|ruck is now known as marios|ruck|lunch11:09
*** bhagyashris_ is now known as bhagyashris11:09
paramiteGreetings guys, can somebody please push remaining backports of https://review.opendev.org/q/Ie7ee7c90107cfe5bff08f5c778a6273ae9ffcc76 through?11:12
*** marios|ruck|lunch is now known as marios|ruck11:15
*** pojadhav is now known as pojadhav|afk11:17
*** soniya29|rover is now known as soniya29|rover|afk11:18
rlandyhjensas: hi - I think you (maybe steve?)are the only one left with +2 rights on https://review.opendev.org/c/openstack/openstack-virtual-baremetal/+/83481311:20
rlandycan we add ysandeep to that list?11:21
rlandyso he can w+11:21
rlandyalso can you w+^^11:21
*** dviroel_ is now known as dviroel11:22
opendevreviewJuan Larriba proposed openstack/tripleo-ansible master: [RFE] Add a new Snapshot And Revert feature to Backup and Restore  https://review.opendev.org/c/openstack/tripleo-ansible/+/83596111:23
opendevreviewJiri Podivin proposed openstack/python-tripleoclient stable/train: Capping validations-libs at 1.6.*  https://review.opendev.org/c/openstack/python-tripleoclient/+/83596211:35
opendevreviewJiri Podivin proposed openstack/python-tripleoclient stable/train: [Train-only] Capping validations-libs at 1.6.*  https://review.opendev.org/c/openstack/python-tripleoclient/+/83596211:36
opendevreviewSandeep Yadav proposed openstack/tripleo-upgrade stable/victoria: Remove additional dependency from pre-commit  https://review.opendev.org/c/openstack/tripleo-upgrade/+/83597211:44
*** ysandeep is now known as ysandeep|brb11:49
*** amoralej is now known as amoralej|lunch11:54
opendevreviewJuan Larriba proposed openstack/tripleo-ansible master: [RFE] Add a new Snapshot And Revert feature to Backup and Restore  https://review.opendev.org/c/openstack/tripleo-ansible/+/83596111:57
opendevreviewDouglas Viroel proposed openstack/tripleo-ci master: Adds compose-repos pre playbook on base jobs  https://review.opendev.org/c/openstack/tripleo-ci/+/83451512:01
opendevreviewDouglas Viroel proposed openstack/tripleo-ci master: Adds compose-repos pre playbook on base jobs  https://review.opendev.org/c/openstack/tripleo-ci/+/83451512:02
*** ysandeep|brb is now known as ysandeep|afk12:05
opendevreviewJuan Larriba proposed openstack/tripleo-ansible master: [RFE] Add a new Snapshot And Revert feature to Backup and Restore  https://review.opendev.org/c/openstack/tripleo-ansible/+/83596112:07
*** soniya29 is now known as soniya29|rover12:07
opendevreviewJuan Larriba proposed openstack/tripleo-ansible master: [RFE] Add a new Snapshot And Revert feature to Backup and Restore  https://review.opendev.org/c/openstack/tripleo-ansible/+/83596112:12
*** chem is now known as Guest85212:32
*** pojadhav|afk is now known as pojadhav12:33
opendevreviewMerged openstack/tripleo-heat-templates stable/victoria: Disable auto discovery  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83567812:38
opendevreviewMerged openstack/tripleo-heat-templates stable/ussuri: Disable auto discovery  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83579612:38
opendevreviewMerged openstack/tripleo-ansible stable/wallaby: Disable auto discovery  https://review.opendev.org/c/openstack/tripleo-ansible/+/83568012:38
opendevreviewMerged openstack/validations-common master: Removing leftover coverchange reference  https://review.opendev.org/c/openstack/validations-common/+/83483712:38
beaglesslagle: I proposed a "designate update" to the ptg etherpad if there's room on the schedule12:40
*** amoralej|lunch is now known as amoralej13:06
*** ysandeep|afk is now known as ysandeep13:14
opendevreviewRabi Mishra proposed openstack/tripleo-heat-templates master: Disabling container validation when running redhat-subscription  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83181313:15
slaglebeagles: wednesday ok?13:24
beaglesslagle: yes, thanks13:30
opendevreviewSandeep Yadav proposed openstack/tripleo-upgrade master: [DNM] to see if we hit job freeze  https://review.opendev.org/c/openstack/tripleo-upgrade/+/83602113:47
opendevreviewSandeep Yadav proposed openstack/tripleo-upgrade stable/victoria: Remove additional dependency from pre-commit  https://review.opendev.org/c/openstack/tripleo-upgrade/+/83597213:54
*** dasm|off is now known as dasm14:00
*** marios|ruck is now known as marios15:00
*** marios is now known as marios|ruck15:01
*** ysandeep is now known as ysandeep|out15:40
*** dviroel_ is now known as dviroel15:54
marios|ruckbogdando: o/ hey can you please check https://bugs.launchpad.net/tripleo/+bug/1966625/comments/17  why is wf -1 @ https://review.opendev.org/c/openstack/puppet-tripleo/+/835448 15:56
marios|ruckrlandy: dasm: ^^ 15:56
dasmack15:56
bogdandomarios|ruck: because there is no 8.0.1 affected version in that job16:02
bogdandoand it fails with puppet-ssh 8.0.016:02
bogdando8.0.1 was not yet promoted (at least by the writing time)16:02
marios|ruckbogdando: ok thanks can you write something in the bug about what we're waiting for (we were discussing during a ruck|rover handoff call we switch shifts tomorrow) thanks bogdando 16:03
bogdandoI would update it so it should pass16:04
opendevreviewBogdan Dobrelya proposed openstack/puppet-tripleo master: Fix double declaration of ssh::server class  https://review.opendev.org/c/openstack/puppet-tripleo/+/83544816:07
opendevreviewBogdan Dobrelya proposed openstack/puppet-tripleo master: Fix double declaration of ssh::server class  https://review.opendev.org/c/openstack/puppet-tripleo/+/83544816:07
*** rlandy is now known as rlandy|rover16:09
*** marios|ruck is now known as marios|out16:12
*** dviroel is now known as dviroel|lunch16:14
opendevreviewDariusz Smigiel proposed openstack/openstack-tempest-skiplist master: Revert "Skip failing test 'test_image_web_download_import'"  https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/83597416:23
opendevreviewDariusz Smigiel proposed openstack/openstack-tempest-skiplist master: Revert "Skip failing test 'test_image_web_download_import'"  https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/83597416:26
opendevreviewDariusz Smigiel proposed openstack/openstack-tempest-skiplist master: Revert "Skip failing test 'test_image_web_download_import'"  https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/83597416:28
*** dasm is now known as dasm|ruck16:38
*** jpena is now known as jpena|off16:39
opendevreviewMerged openstack/tripleo-heat-templates stable/ussuri: Add CephNfs service on roles providing "external" network connectivity  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83333216:45
*** amoralej is now known as amoralej|off17:15
*** dviroel|lunch is now known as dviroel17:18
opendevreviewMikolaj Ciecierski proposed openstack/tripleo-upgrade stable/wallaby: Run control plane update first, next compute and cephstorage last  https://review.opendev.org/c/openstack/tripleo-upgrade/+/83374317:23
opendevreviewMikolaj Ciecierski proposed openstack/tripleo-upgrade stable/ussuri: Run control plane update first, next compute and cephstorage last  https://review.opendev.org/c/openstack/tripleo-upgrade/+/83414517:25
opendevreviewSergii Golovatiuk proposed openstack/tripleo-upgrade stable/train: Add new scheme of driver removal  https://review.opendev.org/c/openstack/tripleo-upgrade/+/83399917:26
opendevreviewJohn Fulton proposed openstack/tripleo-quickstart-extras master: Add ceph-install playbook to install ceph before overcloud  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/83435217:30
opendevreviewCedric Jeanneret proposed openstack/tripleo-ansible master: Async has no value here  https://review.opendev.org/c/openstack/tripleo-ansible/+/83604917:31
opendevreviewCedric Jeanneret proposed openstack/tripleo-ansible master: Async has no value here  https://review.opendev.org/c/openstack/tripleo-ansible/+/83604917:35
opendevreviewCedric Jeanneret proposed openstack/tripleo-ansible master: Async has no value here  https://review.opendev.org/c/openstack/tripleo-ansible/+/83604917:37
opendevreviewCedric Jeanneret proposed openstack/tripleo-heat-templates master: Async has no value here  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83605117:40
opendevreviewCedric Jeanneret proposed openstack/tripleo-ansible master: Async has no value here  https://review.opendev.org/c/openstack/tripleo-ansible/+/83604917:49
opendevreviewMerged openstack/puppet-tripleo stable/ussuri: Add powermax support  https://review.opendev.org/c/openstack/puppet-tripleo/+/83493617:50
opendevreviewCedric Jeanneret proposed openstack/tripleo-heat-templates master: Async has no value here  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83605117:51
opendevreviewSergii Golovatiuk proposed openstack/tripleo-upgrade stable/train: Add new scheme of driver removal  https://review.opendev.org/c/openstack/tripleo-upgrade/+/83399918:11
opendevreviewCedric Jeanneret proposed openstack/tripleo-common master: Async has no value here  https://review.opendev.org/c/openstack/tripleo-common/+/83607619:09
opendevreviewRonelle Landy proposed openstack/tripleo-ci master: Adds compose-repos pre playbook on base jobs  https://review.opendev.org/c/openstack/tripleo-ci/+/83451519:17
opendevreviewMerged openstack/tripleo-heat-templates master: libvirt: Fix invalid dependencies when virtlogd wrapper is enabled  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83576819:25
opendevreviewJohn Fulton proposed openstack/tripleo-quickstart-extras master: Add ceph-install playbook to install ceph before overcloud  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/83435219:48
opendevreviewCedric Jeanneret proposed openstack/tripleo-heat-templates master: Async has no value here  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83605120:14
opendevreviewCedric Jeanneret proposed openstack/tripleo-heat-templates master: Async has no value here  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83605120:20
*** dviroel is now known as dviroel|afk20:24
rlandy|rovercloudnull: hello ... looks like you have +2 rights on openstack-ansible-os_tempest/20:41
rlandy|roverdasm|ruck: ^^20:41
rlandy|roverhttps://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/83597720:42
cloudnulldone 20:42
rlandy|rovercloudnull: https://bugs.launchpad.net/tripleo/+bug/196743020:42
rlandy|rovergoing to take all our jobs out20:42
rlandy|rovercloudnull: thank you20:43
rlandy|roverwe will need a w+ on that when zuul gets done20:45
rlandy|roverlooking for other cores20:45
cloudnullrlandy|rover let me know20:47
cloudnull i can merge it if needed20:47
rlandy|roverif I don;t find another core will ping20:47
rlandy|rovernot sure it helped20:50
rlandy|roverthis may have to be rebuilt20:52
rlandy|rovercloudnull: ^^ would depends-on testing work in this case?20:53
rlandy|roverhttps://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/build/1b28ac882f024d29b6fdd949bd8ef889 failed same way20:53
opendevreviewJohn Fulton proposed openstack/tripleo-ansible master: Support ceph_conf_overrides for multiple_external_ceph_clusters  https://review.opendev.org/c/openstack/tripleo-ansible/+/83608720:55
opendevreviewRonelle Landy proposed openstack/tripleo-heat-templates master: DNM Testing os-tempest  https://review.opendev.org/c/openstack/tripleo-heat-templates/+/83608820:57
rlandy|roverguess that is a no show20:59
rlandy|roverwaiting on other fix20:59
opendevreviewJohn Fulton proposed openstack/python-tripleoclient master: cephadm support for 'openstack overcloud export ceph'  https://review.opendev.org/c/openstack/python-tripleoclient/+/83551120:59
rlandy|roverdasm|ruck: emailing openstack-discuss20:59
slaglecloudnull: hey, for the async thing, if we had 1,000 nodes, we'd start all 1,000 in parallel21:02
slagleand then the poll is based on forks, so if forks=25, we poll 25 at a time, but 1,000 are still running in parallel. so the polls all end "about" the same time21:03
slagleit means these tasks run in ~ constant time, regardless of # of nodes21:04
dasm|ruckrlandy|rover: ack, thx21:21
cloudnullslagle it still starts using forks 21:24
cloudnullit just doesn't check the return until the async_status call 21:24
cloudnullfrom the docs "Using a higher value for --forks will result in kicking off asynchronous tasks even faster. This also increases the efficiency of polling." 21:24
slaglecloudnull: yes, it uses forks, but it's going to fire and forget 1,000 just as fast as it can21:25
slagleso then 1,000 are running in parallel, for all practical purposes21:26
cloudnulltrue, it will run then one by one in the background as fast as possible. then block and poll for the results. 21:27
slagleright in groups of "forks", so say 25 if forks=25. as soon as the first is done, it will move on to polling the next, which is also likely now done21:28
slaglefor a task that takes 5 minutes, instead of taking (nodes/forks * 5) minutes, it takes ~ 5 minutes21:31
cloudnullI believe its still governed by forks even when executing in async, so we're not actually running all tasks simultaneously. but I guess we should time it.21:40
cloudnullasync just backgrounds the task so you can run other tasks then poll before moving on to the next roll or playbook 21:40
cloudnullasync isn't a task level replacement for the free strategy 21:41
slagleyea it would be easy to test with 2 nodes, a sleep task and forks=121:41
slagleit does run them in parallel, https://paste.opendev.org/show/bKip9R9oer2qvGrNUgeW/21:48
slaglei didn't poll them, but both were running despite forks=121:48
dasm|ruckcloudnull: you +2-ed earlier. Could you do again? https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/835977 21:53
dasm|ruckAnd if you have super powers for: https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/836091 that would be nice to +2 too21:53
cloudnullslagle++ fair enough. local test https://paste.opendev.org/show/b15er52VEAgvuiMWgTwy/ it was faster, albeit not by much. 21:56
* cloudnull will try and figure out how to get more info out of those tasks 21:57
slaglewhat was in your tests-async.yaml?21:58
dasm|ruckthx!21:58
cloudnulltest playbook https://paste.opendev.org/show/berMaNtPmpKkHCIjuc0q/21:58
cloudnullthe test without that async was just the one command 21:59
slagleright, b/c the poll is subject to forks=121:59
cloudnullpolling added using params similar to the pr 21:59
slagleand delay21:59
slaglebut yea, at scale, it would approach constant time22:00
cloudnullthat said, with no async and forks 2 (my 2 hosts) the no-async example is faster. 22:01
cloudnullhttps://paste.opendev.org/show/b9Q8Z9bvs4YYoDTgACXe/22:02
slaglewell, right :). if forks is always >= nodes, then no reason to use async here22:03
cloudnulland with our default forks (4xCPU or 100 whatever is less)  I wonder if it matters ?22:03
cloudnullissue: async is hiding task output 22:03
cloudnullassuming we're running an undercloud on 16 core machines == 64 default forks. 22:04
* cloudnull something to think about 22:04
slagledon't we get the output from the next debug task?22:05
slaglepuppet_host_outputs?22:05
cloudnullnope its skipped 22:07
cloudnulland its worse in async tasks with loops 22:08
slaglewhich condition causes the skip? the check for rc? maybe we could default that22:10
cloudnullone of the three conditions is faulting, it doesn't say which one. I'll rerun it and see if I can produce something more.22:13
opendevreviewJohn Fulton proposed openstack/tripleo-quickstart-extras master: Add ceph-install playbook to install ceph before overcloud  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/83435222:25
opendevreviewMerged openstack/tripleo-quickstart-extras master: Override image.http_image on tempestconf  https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/83547522:47
*** dasm|ruck is now known as dasm|off22:59
opendevreviewRafael Castillo proposed openstack/tripleo-quickstart master: Install roles with ansible-galaxy  https://review.opendev.org/c/openstack/tripleo-quickstart/+/83610423:01
*** rlandy|rover is now known as rlandy|out23:08

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!