Tuesday, 2020-08-25

« Monday, 2020-08-24 Index Wednesday, 2020-08-26 »
*** holser has joined #tripleo00:06
openstackgerritBrent Eagles proposed openstack/tripleo-ansible stable/ussuri: Add OpenFlow10 to protocols for backwards compatability  https://review.opendev.org/74780700:19
openstackgerritBrent Eagles proposed openstack/tripleo-ansible stable/train: Add OpenFlow10 to protocols for backwards compatability  https://review.opendev.org/74780800:20
*** artom has quit IRC00:26
*** Goneri has quit IRC00:27
openstackgerritAmy Marrich (spotz) proposed openstack/tripleo-docs master: Clean up of wording and layout  https://review.opendev.org/74780900:52
openstackgerritMatthew Thode proposed openstack/diskimage-builder master: DNM: copy keys into dirs  https://review.opendev.org/74781000:55
*** tkajinam_ is now known as tkajinam01:04
*** rcernin has quit IRC01:14
*** rcernin has joined #tripleo01:16
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient stable/train: (UX) Improve logging if the ephemeral heat stack wasn't created  https://review.opendev.org/74771401:17
openstackgerritEmilien Macchi proposed openstack/tripleo-ansible stable/train: Create a role for network config  https://review.opendev.org/74660001:18
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient stable/train: TCIB: add --rhel-modules argument  https://review.opendev.org/74741401:19
openstackgerritJohn Fulton proposed openstack/tripleo-ansible master: WIP/DNM: Introduce module tripleo_derive_hci_parameters  https://review.opendev.org/74659501:42
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Switch default container prefix to openstack  https://review.opendev.org/74722201:42
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Remove overcloud_containers.yaml  https://review.opendev.org/74701101:43
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Switch default container prefix to openstack  https://review.opendev.org/74722201:43
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Remove Kolla overrides  https://review.opendev.org/74622101:45
*** cfontain has joined #tripleo01:55
openstackgerritMatthew Thode proposed openstack/diskimage-builder master: copy apt gpg keys directly into trusted.gpg.d  https://review.opendev.org/74781002:11
*** hjensas has quit IRC02:25
*** cdearborn has quit IRC02:55
openstackgerritSagi Shnaidman proposed openstack/tripleo-ansible master: WIP - tripleo_container_manage: switch to use podman_containers action plugin  https://review.opendev.org/71638102:56
*** rfolco has quit IRC03:13
*** rh-jelabarre has quit IRC03:16
openstackgerritSagi Shnaidman proposed openstack/tripleo-ansible master: Update containers info module from collection  https://review.opendev.org/74768503:19
openstackgerritSagi Shnaidman proposed openstack/tripleo-ansible master: WIP multiple containers support for podman module  https://review.opendev.org/74715603:27
openstackgerritSagi Shnaidman proposed openstack/tripleo-ansible master: WIP multiple containers support for podman module  https://review.opendev.org/74715603:31
*** jmasud has quit IRC03:31
*** udesale has joined #tripleo03:32
openstackgerritSagi Shnaidman proposed openstack/tripleo-ansible master: WIP - tripleo_container_manage: switch to use podman_containers action plugin  https://review.opendev.org/71638103:37
*** ratailor has joined #tripleo03:39
openstackgerritMerged openstack/tripleo-heat-templates stable/ussuri: Fix up ovn-dbs update tasks  https://review.opendev.org/74745603:42
openstackgerritMerged openstack/tripleo-heat-templates master: Revert "Disable Designate service for scenario 03"  https://review.opendev.org/73786003:42
*** dsneddon has quit IRC03:45
openstackgerritMerged openstack/tripleo-heat-templates master: Fix wrong description about cinder db purge  https://review.opendev.org/74735403:51
openstackgerritMerged openstack/puppet-tripleo master: Add support for glance db purge job  https://review.opendev.org/74734603:51
openstackgerritRabi Mishra proposed openstack/tripleo-ansible master: Add new playbook for container image prepare cli  https://review.opendev.org/74647603:55
openstackgerritMatthew Thode proposed openstack/diskimage-builder master: copy apt gpg keys directly into trusted.gpg.d  https://review.opendev.org/74781004:15
*** psahoo has joined #tripleo04:16
*** ykarel has joined #tripleo04:19
*** psahoo has quit IRC04:29
openstackgerritMerged openstack/tripleo-common stable/train: Wait for lock release during power state change  https://review.opendev.org/74625804:30
*** rcernin has quit IRC04:32
*** evrardjp has quit IRC04:33
*** evrardjp has joined #tripleo04:35
openstackgerritMerged openstack/tripleo-ansible master: Fix role name typo in octavia playbook  https://review.opendev.org/74772504:42
*** udesale has quit IRC04:44
*** psahoo has joined #tripleo04:47
*** udesale has joined #tripleo04:55
*** raukadah is now known as chandankumar04:56
*** jmasud has joined #tripleo05:03
*** rcernin has joined #tripleo05:04
*** ratailor has quit IRC05:09
*** psahoo has quit IRC05:11
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/ussuri: Fix linters  https://review.opendev.org/74782405:12
openstackgerritRabi Mishra proposed openstack/tripleo-ansible master: Adds pids_limit config to podman_container module  https://review.opendev.org/74782505:14
openstackgerritRabi Mishra proposed openstack/tripleo-heat-templates master: Add ContainerNovaLibvirtPidsLimit parameter  https://review.opendev.org/74782605:18
*** ysandeep|away is now known as ysandeep05:20
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Fix linters  https://review.opendev.org/74782705:22
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Move first set of working mol jobs to CentOS-8  https://review.opendev.org/74772405:22
*** psahoo has joined #tripleo05:24
*** lmiccini has joined #tripleo05:27
*** psahoo has quit IRC05:33
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Fix linters  https://review.opendev.org/74782705:36
*** bandini has joined #tripleo05:36
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Move first set of working mol jobs to CentOS-8  https://review.opendev.org/74772405:39
*** rcernin has quit IRC05:41
*** psahoo has joined #tripleo05:50
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Move first set of working mol jobs to CentOS-8  https://review.opendev.org/74772405:55
openstackgerritCédric Jeanneret (Tengu) proposed openstack/paunch master: [USSURI-ONLY] Add new parameter: pids_limit  https://review.opendev.org/74783005:58
Tenguo___O what happened in paunch in its stable/ussuri branch ?!05:59
Tenguerrr.. wait06:00
Tengu'k, my gerrit thing is borked.06:00
openstackgerritCédric Jeanneret (Tengu) proposed openstack/paunch stable/ussuri: [USSURI-ONLY] Add new parameter: pids_limit  https://review.opendev.org/74783106:02
Tengubetter.06:02
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Install latest version of pre-commit and virtualenv  https://review.opendev.org/74783206:05
*** jpalanis_ has joined #tripleo06:06
*** jfrancoa has joined #tripleo06:08
openstackgerritCédric Jeanneret (Tengu) proposed openstack/tripleo-ansible master: Enable pids_limit support  https://review.opendev.org/74783406:11
openstackgerritCédric Jeanneret (Tengu) proposed openstack/tripleo-heat-templates master: Remove PID limit for nova_libvirt container  https://review.opendev.org/74783506:16
*** noama has joined #tripleo06:29
openstackgerritMatthias Runge proposed openstack/tripleo-heat-templates master: dpdk_telemetry is not used  https://review.opendev.org/74783806:30
noamahi dtantsur could you backport https://review.opendev.org/#/c/743498/ to ussuri?06:30
*** ratailor has joined #tripleo06:31
*** slaweq has joined #tripleo06:36
*** xek has joined #tripleo06:44
*** apetrich has joined #tripleo06:56
*** rcernin has joined #tripleo06:57
*** xek has quit IRC07:02
*** rcernin has quit IRC07:03
*** rcernin has joined #tripleo07:06
*** amoralej has joined #tripleo07:06
*** bogdando has joined #tripleo07:08
zbrsshnaidm: tristanC: i need some help to convince zuul-main about https://review.opendev.org/#/c/734833/ -- which directly affects UX for yum/dnf module use.07:08
*** rcernin has quit IRC07:14
openstackgerritMerged openstack/tripleo-heat-templates stable/ussuri: Fix endpoint map tls - zaqar wss port  https://review.opendev.org/74747907:18
openstackgerritMerged openstack/tripleo-upgrade stable/ussuri: Comment out also the wrapped l3 check for Ceph  https://review.opendev.org/74737507:18
*** iurygregory has joined #tripleo07:21
*** dtantsur|afk is now known as dtantsur07:26
*** tosky has joined #tripleo07:36
openstackgerritMerged openstack/tripleo-common stable/ussuri: Fix config parser warning  https://review.opendev.org/74771107:36
openstackgerritMerged openstack/tripleo-validations stable/train: Get UUID from ansible environment variables  https://review.opendev.org/74691807:36
openstackgerritMerged openstack/tripleo-common stable/ussuri: Add modify_only_with_source  https://review.opendev.org/74772007:36
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo stable/ussuri: HA: ensure scaling up galera does not cause promotion errors  https://review.opendev.org/74785107:36
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo stable/train: HA: ensure scaling up galera does not cause promotion errors  https://review.opendev.org/74785207:36
openstackgerritMerged openstack/tripleo-common stable/train: Add modify_only_with_source  https://review.opendev.org/74772107:37
*** xek has joined #tripleo07:43
*** ysandeep is now known as ysandeep|lunch07:48
*** jpich has joined #tripleo07:57
*** suuuper has joined #tripleo07:58
*** gfidente has joined #tripleo07:58
openstackgerritDaniel Bengtsson proposed openstack/tripleo-ansible stable/ussuri: Add OpenFlow10 to protocols for backwards compatability  https://review.opendev.org/74780708:02
*** xek has quit IRC08:04
*** udesale has quit IRC08:07
*** udesale has joined #tripleo08:07
*** pcaruana has quit IRC08:07
*** lucasagomes has joined #tripleo08:13
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: Default to storage_dashboard, when set, for the CephDashboard service  https://review.opendev.org/74786008:16
*** pcaruana has joined #tripleo08:20
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Install latest version of virtualenv  https://review.opendev.org/74783208:29
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Move first set of working mol jobs to CentOS-8  https://review.opendev.org/74772408:31
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Add c8 multinode container job to check & gate  https://review.opendev.org/74762508:32
*** pkopec has joined #tripleo08:32
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: [tripleo-podman] Install specific version of buildah  https://review.opendev.org/74741508:32
*** saneax has quit IRC08:33
*** saneax has joined #tripleo08:34
*** derekh has joined #tripleo08:41
openstackgerritArx Cruz proposed openstack/openstack-tempest-skiplist master: Add tempest.api.compute.servers.test_servers_negative to skiplist  https://review.opendev.org/74786808:50
openstackgerritmathieu bultel proposed openstack/tripleo-validations master: Change path for validation Ansible files  https://review.opendev.org/74776008:51
*** saneax has quit IRC08:55
*** ysandeep|lunch is now known as ysandeep09:02
openstackgerritCédric Jeanneret (Tengu) proposed openstack/tripleo-heat-templates master: Set a higher PIDs limit for nova_libvirt container  https://review.opendev.org/74783509:05
Tenguramishra -^ updated as discussed :).09:06
openstackgerritCédric Jeanneret (Tengu) proposed openstack/tripleo-heat-templates master: Set a higher PIDs limit for nova_libvirt container  https://review.opendev.org/74783509:09
openstackgerritXinliang Liu proposed openstack/diskimage-builder master: Make iscsi-boot element support centos 8  https://review.opendev.org/74787809:29
*** hjensas has joined #tripleo09:29
openstackgerritCédric Jeanneret (Tengu) proposed openstack/tripleo-validations stable/ussuri: Add os-cloud support for tripleo-ansible-inventory script  https://review.opendev.org/74788009:39
openstackgerritCédric Jeanneret (Tengu) proposed openstack/tripleo-validations stable/train: Add os-cloud support for tripleo-ansible-inventory script  https://review.opendev.org/74788109:40
openstackgerritMerged openstack/tripleo-heat-templates stable/ussuri: Create container config scripts with a new module  https://review.opendev.org/74770909:42
openstackgerritSagi Shnaidman proposed openstack/tripleo-ansible master: WIP - tripleo_container_manage: switch to use podman_containers plugin  https://review.opendev.org/71638109:44
openstackgerritSagi Shnaidman proposed openstack/tripleo-ci master: Increase timeout for build images job  https://review.opendev.org/74788209:46
sshnaidmakahat|rover, arxcruz|ruck chandankumar ^^09:46
*** saneax has joined #tripleo09:47
chandankumarsshnaidm: any reason to increasing the timing?09:47
akahat|roverchandankumar, there are some jobs which are failing with timeout error.09:48
chandankumarakahat|rover: job log link?09:48
akahat|roverchandankumar, https://8442fc48a58b3d85474c-27618e467ab9b983fdfb7a89cc8b48ca.ssl.cf1.rackcdn.com/745327/1/gate/tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates/e702a56/job-output.txt09:49
akahat|roverhttps://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_1comp-featureset002-ussuri/59424a6/job-output.txt09:49
sshnaidmchandankumar, hmm, I see some of them fail on patch because of timeout: https://review.opendev.org/#/c/74712909:50
sshnaidmchandankumar, I don't see it happens a lot, though..09:50
sshnaidmchandankumar, maybe this patch affects, although no idea how09:50
arxcruz|ruckmwhahaha: EmilienM sshnaidm we are getting rate limit errors again09:51
arxcruz|rucktripleo_common.image.exception.ImageRateLimitedException: Rate Limited while requesting http://mirror.mtl01.inap.opendev.org:8082/v2/tripleomaster/centos-binary-cron/blobs/sha256:19d6b08ac6693ff5f42d216606e8d11b41e03a95a0dc679ee4f54142ce04fdce09:51
sshnaidmarxcruz|ruck, and we will, until it's resolved ;)09:52
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Ironic role_specific parameters  https://review.opendev.org/73111009:52
arxcruz|rucksshnaidm: I thought it was, with mwhahaha patch09:52
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Ironic role_specific parameters  https://review.opendev.org/73111009:52
sshnaidmarxcruz|ruck, nope09:52
arxcruz|ruckok09:52
chandankumarsshnaidm: it is happening while doing rpm -va command time09:53
chandankumarsshnaidm: let's merge it and see09:53
sshnaidmarxcruz|ruck, what is the bug about rate limits?09:53
arxcruz|rucksshnaidm: https://bugs.launchpad.net/tripleo/+bug/188912209:54
openstackLaunchpad bug 1889122 in tripleo "mirror timeouts in upstream causing undercloud and standalone failures" [Critical,In progress] - Assigned to Ronelle Landy (rlandy)09:54
sshnaidmarxcruz|ruck, thanks09:55
*** paramite|clone has joined #tripleo09:56
sshnaidmFree plan – authenticated users: 200 pulls per 6 hours https://www.docker.com/blog/scaling-docker-to-serve-millions-more-developers-network-egress/10:00
sshnaidmnew docker policy ^10:01
arxcruz|ruckLOL10:01
arxcruz|rucksshnaidm: so the mirrors ends reaching docker.io10:02
sshnaidm"For anonymous (unauthenticated) users, pull rates are limited based on the individual IP address. "10:02
arxcruz|ruckwell, the pro/team plan isn't so high imho10:03
arxcruz|ruckmaybe we can reach docker for an open source plan ?10:03
sshnaidm" we are limiting on manifest requests, the number of layers (blob requests) related to a pull is unlimited at this time"10:04
sshnaidmarxcruz|ruck, I didn't see open source plans..10:05
arxcruz|rucksshnaidm: I mean, maybe we can ask for an open source projects plan, I know other companies have it, sometimes is not listed10:05
arxcruz|rucki don't know, just trowing ideas :)10:05
sshnaidmarxcruz|ruck, idk, I don't think they are so responsive to RH10:07
sshnaidmseems like we'll move to quay or whatever..10:08
*** flaviof has quit IRC10:14
*** jonmasters has quit IRC10:14
*** NobodyCam has quit IRC10:14
openstackgerritMichele Baldessari proposed openstack/tripleo-ansible master: WIP Add tripleo_ha_wrapper role  https://review.opendev.org/72481710:14
*** dpaterson has quit IRC10:15
*** jonmasters has joined #tripleo10:15
*** flaviof has joined #tripleo10:15
*** johnsom has quit IRC10:16
*** NobodyCam has joined #tripleo10:16
*** dpaterson has joined #tripleo10:16
*** chem has quit IRC10:17
*** johnsom has joined #tripleo10:17
*** chem has joined #tripleo10:19
openstackgerritMichele Baldessari proposed openstack/tripleo-ansible master: WIP Add tripleo_ha_wrapper role  https://review.opendev.org/72481710:19
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: WIP pcs commands on host: haproxy bundle  https://review.opendev.org/70795310:26
*** ratailor_ has joined #tripleo10:26
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: WIP pcs commands on host: redis bundle  https://review.opendev.org/70795410:28
*** ratailor has quit IRC10:28
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: WIP pcs commands on host: mysql  https://review.opendev.org/70795510:29
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: WIP pcs commands on host: cinder backup/volume  https://review.opendev.org/70795610:31
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: WIP pcs commands on host: ovn dbs  https://review.opendev.org/70795710:33
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: WIP pcs commands on host: rabbitmq  https://review.opendev.org/70795810:34
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: WIP pcs commands on host: manila-share  https://review.opendev.org/70835710:35
openstackgerritMerged openstack/tripleo-heat-templates stable/train: Create container config scripts with a new module  https://review.opendev.org/74771010:38
*** csatari has quit IRC10:49
*** sri_ has quit IRC10:50
*** rpioso has quit IRC10:50
*** cfontain has quit IRC10:55
*** cfontain has joined #tripleo10:57
*** rcernin has joined #tripleo10:58
*** rpioso has joined #tripleo10:59
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Install latest version of virtualenv and pre-commit  https://review.opendev.org/74783211:00
*** sri_ has joined #tripleo11:01
*** csatari has joined #tripleo11:01
*** cfontain has quit IRC11:01
*** udesale has quit IRC11:05
*** saneax has quit IRC11:07
*** saneax has joined #tripleo11:07
bogdandoweshay, sshnaidm: hi, do you have WIP details for that new zuul pause feature and one-time containers registries?11:11
*** shyamb has joined #tripleo11:11
bogdandowanted to see code! :)11:11
*** shyamb has quit IRC11:11
bogdandothe provider job registry thing11:11
sshnaidmbogdando, https://review.opendev.org/#/q/topic:new-ci-job+(status:open+OR+status:merged)11:12
*** udesale has joined #tripleo11:12
bogdandothanks!11:12
bogdandozuul_return yay11:15
bogdandocool feature11:15
EmilienMbogdando: https://zuul-ci.org/docs/zuul/reference/jobs.html?highlight=zuul_return#return-values11:23
EmilienMhttps://zuul-ci.org/docs/zuul/reference/jobs.html?highlight=zuul%20pause#pausing-the-job11:23
bogdandoyea11:26
bogdandoand zuul pause is great thing as well11:26
openstackgerritLukas Bezdicka proposed openstack/tripleo-heat-templates stable/train: FFWD: OVN ip change  https://review.opendev.org/74776911:26
bogdandoglad to see we have decent devops tooling at last )11:26
bogdandosshnaidm: so it builds also local repo and containers from that mirror?11:27
sshnaidmbogdando, yep11:27
bogdandonice11:27
weshaybogdando, I sent an email11:27
*** udesale has quit IRC11:29
*** cfontain has joined #tripleo11:31
*** udesale has joined #tripleo11:31
*** saneax has quit IRC11:33
bogdandosshnaidm: that local repo is delorean-master-testing?11:34
bogdandotrying to get through build logs https://ee2a47738c0860e88dc7-e8158ca75f6bfe1eab8b9d0a8cfa2de1.ssl.cf2.rackcdn.com/747129/14/check/tripleo-ci-centos-8-standalone-provider/ce4f7ec/logs/undercloud/home/zuul/container-builds/6bd33c98-b354-438d-856f-bc827a04f101/base/base-build.log11:34
openstackgerritMichele Baldessari proposed openstack/tripleo-ansible master: WIP Add tripleo_ha_wrapper role  https://review.opendev.org/72481711:35
sshnaidmbogdando, it's gating-repo11:35
bogdandohm, there is no mentions of it for the base image build11:35
bogdandoso was it all fetched remotely?11:35
bogdandosshnaidm: do I read that correct?11:36
bogdandoI can see no use of gating-repo11:36
sshnaidmbogdando, in this patch nothing is built, you may look at https://review.opendev.org/#/c/74758911:36
bogdandoah11:36
bogdandoseeing ~1h 1.5h 2h max job time there is fantastic :)11:37
bogdandothat's surely win of all times11:38
*** cfontain has quit IRC11:40
sshnaidmbogdando, in which container may be openstack-heat-templates installed?11:42
bogdandosshnaidm: mhm, but still no mentions of local gating repo in https://ea4b2d4f8d5c187742d9-e77ce0ffc4ffea95c693e8c73a486dc2.ssl.cf2.rackcdn.com/747589/3/check/tripleo-ci-centos-8-standalone-provider/b55c9d4/logs/undercloud/home/zuul/container-builds/310386d2-86ff-46b2-9b78-0b37f44549b2/base/base-build.log11:43
bogdandosshnaidm: the openstack base perhaps11:43
sshnaidmbogdando, I see "Gating repository"11:44
weshayEmilienM, train had just one failure.. an update job.. checking it out11:44
EmilienMok11:44
bogdandosshnaidm: it's mentioned in metadata fetched yes, but nothing installed from it in the base image11:44
*** ccamacho has joined #tripleo11:44
sshnaidmbogdando, it's from https://review.opendev.org/#/c/747129/14/playbooks/tripleo-ci/install-built-repo.yml11:44
sshnaidmbogdando, yeah, maybe worth to try with puppet change11:44
bogdandoyup11:45
bogdandocuz with local repo the time to deploy will be even more cosmic11:45
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Install latest version of virtualenv and pre-commit  https://review.opendev.org/74783211:45
ykarelbogdando,  sshnaidm it's in mistral executor gating-repo https://ea4b2d4f8d5c187742d9-e77ce0ffc4ffea95c693e8c73a486dc2.ssl.cf2.rackcdn.com/747589/3/check/tripleo-ci-centos-8-standalone-provider/b55c9d4/logs/undercloud/home/zuul/container-builds/310386d2-86ff-46b2-9b78-0b37f44549b2/base/os/mistral-base/mistral-executor/mistral-executor-build.log11:46
ykarelopenstack-tripleo-heat-templates    noarch  12.5.1-0.20200825030405.d78e95f.el8   gating-repo11:46
bogdandoykarel: oh nice, thanks11:46
sshnaidm openstack-tripleo-heat-templates    noarch  12.5.1-0.20200825030405.d78e95f.el8   gating-repo                   547 k11:46
sshnaidm\o/11:46
sshnaidmok, I'll change anyway to puppet patch11:47
bogdandolet's merge it ASAP )11:48
ykarelsshnaidm, which puppet patch? puppet-packages installed in containers?11:50
openstackgerritSagi Shnaidman proposed openstack/puppet-tripleo master: DNM trigger patches  https://review.opendev.org/74790911:50
*** morazi has quit IRC11:51
openstackgerritSagi Shnaidman proposed openstack/tripleo-ci master: DNM test new CI job with THT patch  https://review.opendev.org/74758911:51
sshnaidmykarel, puppet-tripleo?11:51
ykarelsshnaidm, that i think only installed in tripleoclient container, which is not used11:51
sshnaidmbogdando, let's review it very well before:)11:52
bogdandononono just mergeit11:52
bogdando:)11:52
sshnaidmykarel, mm.. what can be a good candidate? I need it on overcloud containers as well11:52
bogdandosure we will, just seeing it works is enough IMO, then reiterate if more fixes needed11:52
bogdandosince its essential for all CI we have upstream11:53
*** rfolco has joined #tripleo11:53
bogdandothen we can fix or add some missing bits11:53
bogdandosshnaidm: this worth a story in openstack success for zuul use cases topics11:54
ykarelsshnaidm, tripleo-common?11:54
bogdandoshows great flexibility of zuul11:54
ykareldue to healthcheck thing, i think it's installed in all11:54
bogdandoweshay: ^^11:54
sshnaidmykarel, yeah, let's take it11:55
*** morazi has joined #tripleo11:55
ykarelack11:55
sshnaidmbogdando, finally it starts to look as a real CI should be :)11:56
bogdandoindeed11:56
bogdandosshnaidm: what is consumer_job? can't find any refs11:57
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Move first set of working mol jobs to CentOS-8  https://review.opendev.org/74772411:57
*** rlandy has joined #tripleo11:58
bogdandoand provider_job11:58
*** rh-jelabarre has joined #tripleo11:58
sshnaidmbogdando, it's jobs vars, https://review.opendev.org/#/c/747129/14/zuul.d/standalone-jobs.yaml line 46011:59
bogdandothanks11:59
sshnaidmand provider job in the bottom: https://review.opendev.org/#/c/747129/14/zuul.d/standalone-jobs.yaml12:00
bogdandoack, I found now where those are used12:02
bogdandowas thinking that's some hidden zuul options )12:02
*** amoralej is now known as amoralej|lunch12:03
weshayEmilienM, sorry.. update was already bugged.. w/ https://trello.com/c/RuedJsk6/1647-cixlp1887633tripleociproa-periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-train-job-fails-with-pymysqlerroperationalerr  and it's not criteria yet.. train is promoting12:06
openstackgerritMichele Baldessari proposed openstack/tripleo-ansible master: WIP Add tripleo_ha_wrapper role  https://review.opendev.org/72481712:06
*** ysandeep is now known as ysandeep|session12:06
openstackgerritSagi Shnaidman proposed openstack/tripleo-common master: DNM trigger patches  https://review.opendev.org/74791212:07
openstackgerritSagi Shnaidman proposed openstack/tripleo-ci master: DNM test new CI job with THT patch  https://review.opendev.org/74758912:08
openstackgerritSagi Shnaidman proposed openstack/tripleo-ci master: WIP new job for multinode  https://review.opendev.org/74759112:08
*** hoonetorg has quit IRC12:08
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Install latest version of virtualenv and pre-commit  https://review.opendev.org/74783212:08
*** xek has joined #tripleo12:10
*** Goneri has joined #tripleo12:11
bogdandosshnaidm: hm is https://ea4b2d4f8d5c187742d9-e77ce0ffc4ffea95c693e8c73a486dc2.ssl.cf2.rackcdn.com/747589/3/check/tripleo-ci-centos-8-standalone-provider/b55c9d4/logs/undercloud/home/zuul/ misses standalone_container_prep_log ?12:11
bogdandoin collect logs12:11
*** raildo has joined #tripleo12:13
sshnaidmbogdando, is it log for modifying containers?12:13
bogdandosshnaidm: https://review.opendev.org/#/c/746719/18/roles/standalone/tasks/containers.yml@7912:13
bogdandocan't find this logged12:14
*** ykarel_ has joined #tripleo12:16
bogdandosshnaidm, ykarel: seems like "/home/*/*.log", does not work12:17
bogdandolooking here https://ea4b2d4f8d5c187742d9-e77ce0ffc4ffea95c693e8c73a486dc2.ssl.cf2.rackcdn.com/747589/3/check/tripleo-ci-centos-8-standalone-provider/b55c9d4/logs/quickstart_collect_logs.log12:17
bogdandothere is no standalone_container_prep.log12:17
ykarel_bogdando, that should work, is that task running?12:18
bogdandoyes12:18
bogdandolinked the log example12:18
ykarel_i see there is condition - release in ['queens', 'rocky', 'stein', 'train']12:18
*** ykarel has quit IRC12:18
*** ykarel_ is now known as ykarel12:19
ykarelso that task runs only for ^^ releases12:19
bogdandoykarel: right, sorry12:19
ykarelack12:20
openstackgerritLon Hohberger proposed openstack/diskimage-builder master: rhel-common: Provide method to select module streams  https://review.opendev.org/74773212:21
*** hoonetorg has joined #tripleo12:21
openstackgerritMichele Baldessari proposed openstack/tripleo-ansible master: WIP Add tripleo_ha_wrapper role  https://review.opendev.org/72481712:22
bogdandoykarel: then there is no /var/log/tripleo-container-image-prepare(-ansible).logs again... I hoped https://review.opendev.org/#/c/745492/ fixed that12:23
bogdandosigh12:23
ykarelbogdando, likely because it's turned off in these jobs12:24
bogdandohm, but where?12:24
ykarelwhen containers are build, containers updates are turned off, i think that's happening in these jobs12:24
bogdandohttps://review.opendev.org/#/c/746719/18/roles/standalone/tasks/containers.yml@82 runs unconditionally12:25
bogdandomwhahaha: hi, any ideas why there is no log?12:25
ykarelbogdando, https://ea4b2d4f8d5c187742d9-e77ce0ffc4ffea95c693e8c73a486dc2.ssl.cf2.rackcdn.com/747589/3/check/tripleo-ci-centos-8-standalone-provider/b55c9d4/logs/undercloud/home/zuul/containers-prepare-parameters.yaml12:25
ykarelmodify thing is turned off when job.build_container_images is true12:26
bogdandoykarel: that command always logs something, even if run w/o modify12:26
bogdandohttps://opendev.org/openstack/tripleo-operator-ansible/src/branch/master/roles/tripleo_container_image_prepare_default/tasks/main.yml#L512:27
bogdandonot critical ofc, but trying to figure why no logs, again :/12:28
ykarelbogdando, ohkk i thought only -ansible.log missing, need to check why other file is not there12:29
bogdandoyea12:29
bogdandohairy things12:30
bogdandosshnaidm: shouldn't we run full prepare images command in the end https://review.opendev.org/#/c/746719/18/roles/standalone/tasks/containers.yml,unified ?12:32
bogdandoafter all things amended12:32
bogdandohm, or that perhaps for the child jobs to do12:32
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Install latest version of virtualenv and pre-commit  https://review.opendev.org/74783212:33
bogdandonvm, that's https://review.opendev.org/#/c/746719/18/roles/standalone/tasks/standalone-install.yml@10 call image prepare based on the crafted env file IIUC12:34
weshaybogdando, ykarel ya.. so when jobs run w/ parent/child the children should not modify containers12:36
weshayand container updates should not run or log12:36
bogdandobut I think that prep containers log is still missing12:36
weshayoh12:36
bogdandoit always logs something12:36
* weshay looks12:36
sshnaidmykarel, hmm.. any idea why it's in merge conflict with tripleo-common? https://review.opendev.org/#/c/747589/12:36
bogdandothe minimal output is that goes in the env file12:37
bogdandowhat*12:37
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Install pre-commit < 2.7.0  https://review.opendev.org/74783212:37
bogdandolet me show12:37
bogdandoso here is a "full pack" https://24dff9b4c9cba9eb54eb-2b4f2fa211fea8445ec0f4a568f6056b.ssl.cf1.rackcdn.com/745492/4/check/tripleo-ci-centos-8-containers-multinode/71488f6/logs/undercloud/var/log/12:38
bogdandoand the minimal log https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_269/745492/4/check/tripleo-ci-centos-8-standalone-upgrade/269fbf4/logs/undercloud/var/log/tripleo-container-image-prepare.log12:39
ykarelsshnaidm, looking12:40
*** hoonetorg has quit IRC12:41
*** artom has joined #tripleo12:43
bogdandoykarel: it seems that collect logs gets working_dir like that /tmp/undercloud/home/zuul/containers-prepare-parameters.yaml but there is no match for image prepare log?12:44
mwhahahabogdando: https://review.opendev.org/#/c/746719/18/roles/standalone/tasks/containers.yml@82 is the default run. eg to get the sample yaml first12:45
ykarelsshnaidm, doesn't look due to tripleo-common, likely coming from other patches, trying to find12:45
mwhahahathere are no logs12:45
mwhahahaalso, happy trainwreck tuesday12:46
*** Madkiss has quit IRC12:47
bogdandomwhahaha: actually it always logs something12:47
bogdandoto that file12:47
*** Guest34946 has joined #tripleo12:47
bogdandoanyway, there is a 2nd run later, but still no logs12:47
mwhahahano it's there12:48
bogdandowhere?12:48
mwhahahain that log file12:48
mwhahaha2020-08-21 22:19:57,623 5302812:48
mwhahaha2020-08-21 23:07:45,70912:48
bogdandoum12:48
bogdandowhich url sorry12:48
mwhahahahttps://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_269/745492/4/check/tripleo-ci-centos-8-standalone-upgrade/269fbf4/logs/undercloud/var/log/tripleo-container-image-prepare.log12:48
bogdandothat was my example12:49
gfidenteguys, does anyone know when host_prep_tasks is supposed to run? does it run once when the nodes are added to heat or on every update12:49
gfidente?12:49
mwhahahaanyway the fastest way for us to resolve teh 429 bits is to 1) remove mirror config, 2) land my t-q-e patch to switch to source12:49
mwhahahagfidente: during deploy12:49
gfidentemwhahaha hey thanks12:49
gfidentemwhahaha deploy you mean initial deploy only?12:49
bogdandomwhahaha: but I'm talking about no logs captured for https://review.opendev.org/#/q/topic:test-new-ci-job+(status:open+OR+status:merged12:49
mwhahahagfidente: it's part of teh deploy scripts, so it always runs when the deploy bits run12:50
mwhahahagfidente: it may not run during the application of an update or upgrade runs but will run later during converge12:50
*** Guest34946 is now known as redrobot12:50
bogdandono logs/undercloud/var/log/tripleo-container-image-prepare*.log :(12:50
mwhahahabogdando: i haven't looked at those logs, will try and catch up12:50
bogdandoyes, thanks12:50
gfidentemwhahaha ok I'll add you on a review then, but it confirms what we're seeing12:51
bogdandof.e. https://ea4b2d4f8d5c187742d9-e77ce0ffc4ffea95c693e8c73a486dc2.ssl.cf2.rackcdn.com/747589/3/check/tripleo-ci-centos-8-standalone-provider/b55c9d4/logs/quickstart_collect_logs.log shows all collected things that was matched filters12:51
bogdandook thanks for looking into that folks. It's important to have that log for the provider registry job12:52
bogdandoat least the expected log was shown for child jobs after the provider registry set up https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_9ae/747589/3/check/tripleo-ci-centos-8-standalone/9ae77ac/logs/undercloud/var/log/tripleo-container-image-prepare.log:)12:54
cloudnullmornings12:55
openstackgerritSorin Sbarnea (zbr) proposed openstack/tripleo-ansible stable/train: Fixed linting  https://review.opendev.org/74783212:56
sshnaidmbogdando, provider job just build and push to local registry, not sure what we are looking for..12:59
bogdandosshnaidm: the image prepare log... so it should only be for child jobs and without modify images?12:59
bogdandoif so, then it's all fine13:00
sshnaidmbogdando, I think so, neither provider nor child jobs run modify images13:00
*** bnemec has joined #tripleo13:00
sshnaidmbogdando, no need to modify in both of them13:00
*** amoralej|lunch is now known as amoralej13:01
bogdandook then sorry for the fuss, mwhahaha, ykarel, weshay :)13:02
*** jonmasters has quit IRC13:02
*** jonmasters has joined #tripleo13:02
ykarelsshnaidm, didn't find what caused that, so i rechecked to see if it clears13:03
ykareland now i see jobs are triggered13:03
bogdandoso works like a charm then13:04
sshnaidmykarel, interesting...13:12
*** ysandeep|session is now known as ysandeep13:16
openstackgerritAmy Marrich (spotz) proposed openstack/tripleo-docs master: Clean up of wording and layout  https://review.opendev.org/74780913:17
openstackgerritmathieu bultel proposed openstack/tripleo-validations master: Change path for validation Ansible files  https://review.opendev.org/74776013:17
*** ratailor_ has quit IRC13:18
*** cfontain has joined #tripleo13:22
*** cfontain has quit IRC13:27
*** ykarel is now known as ykarel|away13:28
EmilienMweshay, mwhahaha : we need to land the t-q-e patch for 429, do you want me to clear the gate?13:32
mwhahahai would say yes. we need to stop rechecking stuff for now13:35
*** ChanServ sets mode: +o mwhahaha13:35
*** mwhahaha changes topic to "CI status: RED - registry rate limit | community irc meeting 6/23 Tues@1400 UTC - tripleo-ci-community meeting Tues@1330 UTC | https://docs.openstack.org/tripleo-docs"13:36
EmilienMok13:38
EmilienMweshay: ok?13:38
weshayEmilienM, fine by me13:39
mwhahahaEmilienM: let's hold off for a bit13:40
mwhahahato see if the top patches land in the gate13:40
EmilienMk13:40
mwhahahaalso tripleo-ci-centos-7-standalone-upgrade-trainfailure (non-voting) is in the gate13:40
EmilienMI'm ready :D13:40
EmilienMha non voting jobs in gate, it never stops :D13:40
*** TrevorV has joined #tripleo13:41
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-common master: Remove the base layers  https://review.opendev.org/74747613:41
EmilienMmwhahaha: ^13:41
openstackgerritEmilien Macchi proposed openstack/tripleo-ci master: Remove tripleo-ci-centos-7-standalone-upgrade-train from gate  https://review.opendev.org/74795213:41
EmilienMmwhahaha: ^13:41
*** paramite|clone has quit IRC13:44
*** paramite has joined #tripleo13:44
*** tmazur has joined #tripleo13:44
*** ykarel|away has quit IRC13:45
*** jtomasek has joined #tripleo13:47
*** jtomasek has quit IRC13:47
openstackgerritAlex Schultz proposed openstack/tripleo-ci master: Disable docker mirror temporarily  https://review.opendev.org/74795313:48
*** d34dh0r53 has joined #tripleo13:49
openstackgerritMerged openstack/openstack-virtual-baremetal master: Switch from openstackcli to openstacksdk  https://review.opendev.org/74479213:50
*** weshay is now known as weshay|interview13:54
*** jbadiapa has joined #tripleo13:59
*** pcaruana has quit IRC13:59
*** cfontain has joined #tripleo14:04
openstackgerritMerged openstack/tripleo-common stable/train: [TRAIN-Only] Dynamically add centos8 namespace  https://review.opendev.org/74592414:07
openstackgerritMerged openstack/tripleo-validations master: Add new Upgrades validation groups.  https://review.opendev.org/73885714:07
*** pcaruana has joined #tripleo14:12
openstackgerritJose Luis Franco proposed openstack/tripleo-upgrade stable/rocky: Fix conditional for python3-tripleoclient in undercloud pre-reqs  https://review.opendev.org/74796114:13
*** cfontain has quit IRC14:13
openstackgerritTakashi Kajinami proposed openstack/tripleo-ansible master: Show the proper value (0) to set unlimited pids  https://review.opendev.org/74796314:19
mwhahahaEmilienM: clear the gate, the update job timed out on the top change :/14:25
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-common master: Move get network configs action to utils  https://review.opendev.org/73784714:25
mwhahahaEmilienM: keep 746993,914:25
EmilienMmwhahaha: ack14:26
*** pkopec has quit IRC14:29
EmilienMmwhahaha: done14:30
EmilienMALL: on recheck, no +W please14:30
EmilienMi'll send a note on openstack-discuss14:30
openstackgerritFrancesco Pantano proposed openstack/tripleo-heat-templates master: Add post_upgrade_task and remove puppet-ceph pkg  https://review.opendev.org/71905014:31
Tengu;_;14:33
*** ysandeep is now known as ysandeep|away14:37
*** rcernin has quit IRC14:45
openstackgerritSagi Shnaidman proposed openstack/tripleo-ansible master: WIP - tripleo_container_manage: switch to use podman_containers plugin  https://review.opendev.org/71638114:50
*** psahoo has quit IRC15:03
*** pkopec has joined #tripleo15:10
*** xek has quit IRC15:14
openstackgerritChandan Kumar (raukadah) proposed openstack/tripleo-ansible stable/train: Move first set of working mol jobs to CentOS-8  https://review.opendev.org/74772415:16
*** psahoo has joined #tripleo15:18
openstackgerritAlex Schultz proposed openstack/tripleo-ansible master: Add tripleo_nodes_validation role  https://review.opendev.org/74746615:22
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: Use ansible for nodes validation  https://review.opendev.org/74746815:22
*** chandankumar is now known as raukadah15:23
*** amoralej is now known as amoralej|off15:23
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: Remove validation scripts  https://review.opendev.org/74773515:27
*** owalsh has quit IRC15:36
*** jtomasek has joined #tripleo15:42
*** owalsh has joined #tripleo15:45
*** owalsh has quit IRC15:45
*** psahoo has quit IRC15:46
openstackgerritKevin Carter (cloudnull) proposed openstack/tripleo-common stable/train: First pass at script cleanup  https://review.opendev.org/74654315:49
*** sshnaidm is now known as sshnaidm|afk15:59
*** jtomasek has quit IRC16:03
*** weshay|interview is now known as weshay16:04
*** bogdando has quit IRC16:06
openstackgerritAmy Marrich (spotz) proposed openstack/tripleo-docs master: Clean up of wording and layout  https://review.opendev.org/74780916:06
*** lucasagomes has quit IRC16:06
spotzEmilienM: Let me know if you want me to -1 workflow that so it doesn't get checked yet16:07
*** lmiccini has quit IRC16:08
EmilienMspotz: .. include:: ../environments/standalone.rst16:08
EmilienMthe file doesn't exist?16:08
EmilienMand no, doc patches are fine, they don't block anything16:09
spotzEmilienM: I added it16:09
EmilienMspotz: oh right16:09
spotzlet me make sure it got added to the review16:09
EmilienM+216:09
*** udesale has quit IRC16:09
spotzYep we're good:)16:10
spotzI missed patching stuff:)16:10
*** hoonetorg has joined #tripleo16:15
*** dtantsur is now known as dtantsur|afk16:19
*** owalsh has joined #tripleo16:23
*** gfidente has quit IRC16:23
*** bnemec has quit IRC16:30
*** saneax has joined #tripleo16:33
*** bnemec has joined #tripleo16:35
*** noama has quit IRC16:35
openstackgerritMichele Baldessari proposed openstack/tripleo-ansible master: Add tripleo_ha_wrapper role  https://review.opendev.org/72481716:35
*** jpich has quit IRC16:42
*** rh-jelabarre has quit IRC16:43
*** rh-jelabarre has joined #tripleo16:43
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: pcs commands on host: haproxy bundle  https://review.opendev.org/70795316:43
*** ade_lee has quit IRC16:54
lbragstadis anyone else noticing this with centos8 and tripleo-operator-ansible http://paste.openstack.org/show/797136/16:55
lbragstadit appears the play that install tripleo-repos gets skipped16:56
lbragstadleading to that failure16:56
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: pcs commands on host: redis bundle  https://review.opendev.org/70795417:06
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: pcs commands on host: mysql  https://review.opendev.org/70795517:08
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: pcs commands on host: cinder backup/volume  https://review.opendev.org/70795617:10
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: pcs commands on host: haproxy bundle  https://review.opendev.org/70795317:13
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: pcs commands on host: redis bundle  https://review.opendev.org/70795417:16
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: pcs commands on host: mysql  https://review.opendev.org/70795517:16
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: pcs commands on host: cinder backup/volume  https://review.opendev.org/70795617:17
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: WIP pcs commands on host: ovn dbs  https://review.opendev.org/70795717:18
*** derekh has quit IRC17:18
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: pcs commands on host: ovn dbs  https://review.opendev.org/70795717:19
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: pcs commands on host: haproxy bundle  https://review.opendev.org/70795317:20
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: pcs commands on host: redis bundle  https://review.opendev.org/70795417:20
*** ade_lee has joined #tripleo17:21
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: pcs commands on host: mysql  https://review.opendev.org/70795517:21
*** cfontain has joined #tripleo17:21
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: pcs commands on host: cinder backup/volume  https://review.opendev.org/70795617:22
*** cfontain has quit IRC17:26
openstackgerritAlex Schultz proposed openstack/tripleo-quickstart-extras master: Use new modify_only_with_source  https://review.opendev.org/74699317:26
weshayEmilienM, is the trace we're looking for still either of the two here? https://opendev.org/openstack/ansible-role-collect-logs/src/branch/master/vars/sova-patterns.yml#L743-L76117:29
EmilienMweshay: yes17:31
EmilienMi think17:31
weshayEmilienM, k.. tracking it here http://dashboard-ci.tripleo.org/d/wb8HBhrWk/cockpit?orgId=1&fullscreen&panelId=6117:32
*** derekh has joined #tripleo17:33
*** dsneddon has joined #tripleo17:36
jpalanis_cloudnull, hi17:36
cloudnullo/17:37
jpalanis_cloudnull, i have done the setup and working on updating my two patches17:38
openstackgerritDavid Peacock proposed openstack/tripleo-heat-templates master: Expose Innodb_buffer_pool_size  https://review.opendev.org/74799217:39
weshayspotz, thank you :) https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_7a2/747809/3/check/build-openstack-deploy-guide/7a2bec8/docs/environments/standalone.html17:42
weshaynot sure what that error is yet17:42
spotzweshay: Yeah I looked and didn't see anything standing out just a skipped17:42
weshay2020-08-25 17:37:36.591998 | ubuntu-bionic | build finished with problems, 4 warnings.17:42
weshay2020-08-25 17:37:36.824344 | ubuntu-bionic | ERROR: InvocationError for command /home/zuul/src/opendev.org/openstack/tripleo-docs/.tox/deploy-guide/bin/sphinx-build -a -E -W -d deploy-guide/build/doctrees --keep-going -b html deploy-guide/source deploy-guide/build/html (exited with code 1)17:42
*** bnemec has quit IRC17:50
weshayspotz, something between patch set 2 and 3 https://zuul.openstack.org/builds?job_name=build-openstack-deploy-guide%0917:50
weshayunless the test itself just broke17:51
spotzweshay: Maybe the include? The rest is just text17:51
EmilienMmwhahaha: I'll have to step away a bit earlier today, the gerrit query is https://review.opendev.org/#/q/%22The+gate+is+currently+hitting%2217:51
mwhahahak17:51
EmilienM(when we want to restore things)17:51
weshayjebus17:52
*** derekh has quit IRC17:52
*** bnemec has joined #tripleo17:53
openstackgerritAmy Marrich (spotz) proposed openstack/tripleo-docs master: Clean up of wording and layout  https://review.opendev.org/74780917:55
spotzI see one little fix but I would hope it didn't cause the failure and a grammar fix17:55
EmilienMas long as you don't introduce french symbols we're fine :D17:56
weshayseriously.. don't use french, terrible just terrible. It's like speaking w/ rocks in your mouth17:57
EmilienMdon't get me there :)17:58
spotzEmilienM: hehe, My French is basic enough I could order food in Luxembourg and Paris but not answer if they asked if it was for here or to go:)17:58
EmilienMso like weshay's english17:59
EmilienMactually it's the opposite. we usually have no answer when he speaks18:00
weshaypretty much.. english is my second language and I don't have a first18:00
weshaymaybe grunting18:00
spotzhehe18:00
*** suuuper has quit IRC18:04
spotzweshay: COuld this conflict be causing the failure? https://review.opendev.org/#/c/728581/518:04
rlandyEmilienM: https://review.opendev.org/#/c/747414/ very polite message there :)18:05
weshayspotz, I would guess it's the include18:07
spotzweshay: nasty include!18:08
spotzweshay: Gonna remove it for one patch to verify18:09
EmilienMrlandy: working on myself as you can see18:09
openstackgerritAmy Marrich (spotz) proposed openstack/tripleo-docs master: Clean up of wording and layout  https://review.opendev.org/74780918:10
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: pcs commands on host: rabbitmq  https://review.opendev.org/70795818:10
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: pcs commands on host: manila-share  https://review.opendev.org/70835718:13
*** bandini has quit IRC18:15
spotzYep definitely looking like it's related to that include:(18:39
openstackgerritSagi Shnaidman proposed openstack/tripleo-ansible master: WIP - tripleo_container_manage: switch to use podman_containers plugin  https://review.opendev.org/71638118:41
openstackgerritAmy Marrich (spotz) proposed openstack/tripleo-docs master: Clean up of wording and layout  https://review.opendev.org/74780918:41
*** sri_ has quit IRC18:48
*** sri_ has joined #tripleo18:49
*** xek has joined #tripleo18:59
spotzThis makes no sense! My unhelpful comment of the day:)19:02
mwhahahafultonj: what's ceph's take on the docker.io rate limit bits. e.g. will ceph containers move places? (if not they probably should)19:04
fultonjmwhahaha: i don't know that ceph-ansible's CI is affected by the docker.io limits. I can ask19:09
fultonji'm looking at https://bugs.launchpad.net/tripleo/+bug/188912219:10
openstackLaunchpad bug 1889122 in tripleo "mirror timeouts in upstream causing undercloud and standalone failures" [Critical,In progress] - Assigned to Ronelle Landy (rlandy)19:10
fultonjbut I don't see an answer yet as to where openstack containers are moving to deal with the rate limits19:10
mwhahahafultonj: we're still TBD, but CI will likely not be able to fetch teh containers19:11
mwhahahaso we might have to disable ceph integration in upstream CI if we don't have a docker.io alternative19:11
mwhahahacurrently we're investigating switching to building the containers always in CI instead19:11
mwhahahawhich we can't do for ceph/stf bits19:11
fultonjunless we build the ceph containers too19:12
mwhahahawe could probably add them into the tcib stuff if we had too but that assumes rpm availability too19:13
mwhahahajust a heads up, cause it means nov 1st is going to bring pain https://www.docker.com/blog/scaling-docker-to-serve-millions-more-developers-network-egress/19:14
fultonjoh!19:15
fultonjok, thanks19:15
*** cfontain has joined #tripleo19:24
openstackgerritAmy Marrich (spotz) proposed openstack/tripleo-docs master: Clean up of wording and layout  https://review.opendev.org/74780919:27
*** cfontain has quit IRC19:28
*** saneax has quit IRC19:28
*** cfontain has joined #tripleo19:34
*** pkopec has quit IRC19:39
fultonjmwhahaha: i'm going to raise the issue on ceph-devel19:47
mwhahahak19:47
*** rlandy is now known as rlandy|brb19:47
*** cfontain has quit IRC19:48
mwhahahaarg the t-q-e change we need to land failed19:50
*** slaweq has quit IRC20:06
*** rlandy|brb is now known as rlandy20:06
mwhahahaweshay: https://review.opendev.org/#/c/747953/ if we want to go that route while we get the other jobs in place20:12
weshaymwhahaha, they aren't in the gate anymore20:12
weshayand it's not merged20:12
weshayah I see20:12
mwhahahathe t-q-e change failed on ceph20:12
mwhahahathat's a different patch20:12
mwhahahasimilar impact20:12
mwhahahae.g. if we land it, 429s probably go away20:13
weshayok20:13
weshaythis one https://review.opendev.org/#/c/747953/ ya?20:13
mwhahahaya20:13
weshaymakes sense20:14
mwhahahasince that doesn't run the world, it'll probably land :D20:14
weshaylolz20:14
weshayya20:14
weshayci repos are the worst20:14
weshayw/ check + gate20:14
weshaybut better than breaking you guys20:14
mwhahahauntil we to land a fix :D20:15
*** jfrancoa has quit IRC20:17
*** slaweq has joined #tripleo20:18
*** zaneb has quit IRC20:21
openstackgerritAmy Marrich (spotz) proposed openstack/tripleo-docs master: Clean up of wording and layout  https://review.opendev.org/74780920:25
spotzMost patches ever!!!!:)20:25
fultonjthanks spotz20:26
*** jbadiapa has quit IRC20:26
spotzfultonj: clarkb and I have got it down to the heads in the file I created and trying to include. In theory everything is right but it goes from main headers on the one page to subheaders on the other. SO just trial and error until Zuul is happy20:27
*** jfrancoa has joined #tripleo20:30
lbragstadis anyone else noticing issues with tripleo-repos?20:31
lbragstadspecifically - ModuleNotFoundError: No module named 'tripleo_repos'20:32
weshaylocally or in a job?20:33
lbragstadi'm able to recreate it locally20:33
lbragstadthe rpm is installed20:33
mwhahahawrong python version?20:33
mwhahahae.g. installed python2 rpm on py320:34
lbragstadi'm using tripleo-operator-ansible20:34
mwhahahaor vice versa20:34
mwhahahayea that doesn't mean anything :D20:34
weshayspotz, tox -e deploy-guide20:34
mwhahahai've seen that but i forget what the fix was. i thought it was because of the wrong python version being executed/installed20:34
lbragstadhttp://paste.openstack.org/show/797140/20:35
lbragstadi think tripleo-operator-ansible does version detection based on the OS release20:36
lbragstadhttps://github.com/openstack/tripleo-operator-ansible/blob/master/roles/tripleo_repos/tasks/main.yml#L16-L1720:36
mwhahahanot certain, the error seems to poitn to the path not being found20:36
mwhahahalike it's a bad install or something20:36
*** ccamacho has quit IRC20:36
lbragstadit starts executing /usr/bin/tripleo-repos20:37
lbragstadbut it can't find the module20:37
mwhahahayea is it missing from /usr/lib/pythonxx/site-packages ?20:37
mwhahahado you have python 3.8 installed20:37
mwhahahathat's the other time i hit it20:37
mwhahahaif you use stream you'll get python 3.820:37
mwhahahawe build against 3.620:37
lbragstadah20:37
mwhahahaso it can't find it20:37
lbragstadi do have python3.820:37
mwhahahawe should probably have a hard version dep when we build this stuff20:38
mwhahahaseems like a bug in rdo20:38
lbragstadhttp://paste.openstack.org/show/797141/20:38
mwhahaharight you need to switch to the python 3.6 module by default20:39
mwhahahaalso don't use stream20:39
mwhahahaunless you hate yourself20:39
lbragstadyeah - i'm using the CentOS8 generic cloud image20:40
lbragstadwhich i imagine is a moving target20:40
mwhahahait shouldn't be20:41
mwhahahathe generic is different than stream20:41
lbragstadCentOS-8-x86_64-GenericCloud-released-latest (in upshift)20:41
lbragstadwe *just* noticed this within the last 24 hours20:41
mwhahahahttps://cloud.centos.org/centos/8-stream/x86_64/images/20:42
mwhahahathey are named different20:42
mwhahahamaybe someone screwed up20:42
* mwhahaha shrugs20:42
lbragstadah - interesting20:42
lbragstadok - i wasn't aware of the stream image, but i don't think we're using that intentionally20:42
mwhahahaif centos8 the default has moved to 3.8, we're boned20:42
mwhahahaand i'll just not show up for the rest of the month20:43
lbragstadwhelp...20:43
mwhahahaseems about on par for this week20:43
mwhahahayou should be able to check if the stream rpm is installed20:43
mwhahahai forget what it's called20:43
lbragstadstream rpm or the repo?20:44
mwhahahacentos-release-stream20:44
mwhahahais that installed for you?20:44
*** jfrancoa has quit IRC20:44
lbragstadnope20:44
lbragstadhttp://paste.openstack.org/show/797142/20:44
mwhahahaweird20:45
mwhahahado a module list20:45
lbragstadhttp://paste.openstack.org/show/797143/20:45
mwhahahayea yours has 3.8 as default20:46
mwhahahamine doesn't20:46
lbragstadhuh20:46
mwhahahahttp://paste.openstack.org/show/797144/20:46
mwhahahasudo dnf module enable python36 && sudo dnf module disable python3820:48
mwhahahathough we should force that some other way20:48
lbragstadspinning up a fresh instance to see if the module list is the same as yours20:49
mwhahahafeels like /usr/bin/python3 is the next /usr/bin/python20:52
mwhahahapackagers make up your minds :(20:53
lbragstadfresh instance http://paste.openstack.org/show/797145/20:53
mwhahahaweird20:53
mwhahahapython3 --version is 3.8?20:53
lbragstadpython3 command isn't available20:54
lbragstadneither is python20:54
lbragstadwhich seems broken out of the box?20:54
mwhahahano because it's not enabled20:54
mwhahahabut when you install tripleo-repos it should turn it on20:54
lbragstadhm20:55
mwhahahawhich points to us not properly adding the 3.6 python requirement in rdo20:55
mwhahahagood to see the latest centos image still has the bad entry in /etc/resolve.conf20:58
mwhahahahttp://paste.openstack.org/show/797146/20:58
mwhahahayes it's enabling 3.8 now20:58
mwhahahaweshay: -^ something from rdo folks, it's probably going to cause problems somewhere20:59
* mwhahaha files for pto for the remander of the week20:59
cloudnullinterestingly enough https://pasted.tech/pastes/b93ff136011bc9a7b10eda9d2c80210fbe6d2a3d - i'm still seeing py36 as the default but on a recent undercloud build on cent 8 (stable from iso) I'm seeing py 38 available20:59
mwhahahayea you can co-install21:00
mwhahahathe issue is that 36 needs to be the default21:00
mwhahahaand that's not happening for whatever reason21:00
lbragstadlooks like the image i'm using has py36 and py38, but both are disabled21:00
cloudnullI didn't co-install anything intentionally, something pulled it in21:00
mwhahahatripleo-repos enables it on install21:00
mwhahahasee my paste21:00
mwhahahabecause the rpm requirement is likely python3 and not python 3.621:00
lbragstadyeah - that's what's happening in my envs21:01
mwhahahawhich needs to get fixed in the rdo build21:01
mwhahahabecause we're using whatever magical macros21:01
lbragstadhttps://review.rdoproject.org/cgit/openstack/tripleo-repos-distgit/tree/tripleo-repos.spec21:02
lbragstadis that the place ^21:02
lbragstad?21:03
mwhahahano because it likely affects everything21:03
mwhahahait's an issue for #rdo imho21:03
lbragstadok - i'll head over there and see if they're aware21:03
mwhahahabut they are all off for the day21:04
mwhahahacause emea21:04
cloudnullmy env is working, so i never looked to see what python versions were installed before now .21:04
cloudnullOIC21:05
mwhahahaso i booted my centos image21:06
mwhahahaand it enables 3.621:06
mwhahahaCentOS-8.2.200421:06
mwhahahahttp://paste.openstack.org/show/797147/21:07
mwhahahaso the latest image must change it21:07
mwhahahamight also check with #centos-devel21:08
* mwhahaha shrugs21:08
mwhahahai have no idea where the that swithc might happen21:08
*** cfontain_ has joined #tripleo21:10
mwhahahaweird so the latest release one is actually older than what i have21:13
mwhahaha8.1.1911 vs 8.2.200421:13
mwhahahalbragstad: i bet if you dnf update before installing tripleo-repos it'll correctly pull in 3621:14
* mwhahaha checks21:14
lbragstadi tried that and it didn't work for me21:15
lbragstadi updated all packages before installing tripleo-repos21:15
mwhahahaok so i yum updated and it worked correctly21:19
mwhahahayou have to yum update before installing tripleo-repos21:19
mwhahahai think it's abug in an older dnf21:20
mwhahahayou need at least dnf-4.2.17-621:20
mwhahahalatest is dnf-4.2.17-7.el8_2.noarch21:20
lbragstaddnf-4.2.17-7.el8_2.noarch21:21
lbragstadlooks like the version i have21:21
mwhahahayea but you have to have that before installing tripleo-repos21:21
mwhahahaso it properly enables the correct python module21:21
mwhahahahttps://bugzilla.redhat.com/show_bug.cgi?id=1781769 maybe?21:22
openstackbugzilla.redhat.com bug 1781769 in dnf "Dnf does not correctly detect the latest version of the module stream" [Unspecified,Closed: errata] - Assigned to jmracek21:22
lbragstadhm21:23
lbragstadok - so yum update first, then install python3-tripleo-repos21:23
mwhahahayea21:23
lbragstadis what worked for you21:23
mwhahahayea21:23
lbragstadok - trying that on a fresh box quick21:24
mwhahahai just launched a CentOS-8-x86_64-GenericCloud-released-latest21:24
mwhahahaand then i yum updated21:24
mwhahahaand then the yum install <tripleo-repos.rpm> properly got the right version21:24
mwhahahaor, use your own centos image that is 8.2+21:24
mwhahahathe latest released is not the latest released21:25
openstackgerritLuke Short proposed openstack/tripleo-heat-templates master: Expose max_concurrent_builds as a Heat parameter  https://review.opendev.org/74775521:25
lbragstadstrange - i shuffled around a task for updating all the packages on the undercloud21:25
lbragstadand made sure it happened before tripleo-repos was installed21:25
*** cfontain has joined #tripleo21:27
lbragstadweird... http://paste.openstack.org/show/797148/21:28
mwhahahayea that21:28
mwhahahasee21:28
mwhahahahttp://paste.openstack.org/show/797149/21:28
spotzweshay: Yeah, also checking on doc channel for suggestions we can't be the only ones trying this21:28
lbragstadaha - yeah... that's it21:29
mwhahahait's something else than dnf tho21:29
mwhahahai just yum updated dnf and that wasn't enough21:29
*** cfontain_ has quit IRC21:30
*** raildo has quit IRC21:31
sshnaidm|afkmwhahaha, what do you mean by "pull through mirror" registry? Like transparent caching registry? https://docs.docker.com/registry/recipes/mirror/21:34
mwhahahasshnaidm|afk: yea21:35
mwhahahasshnaidm|afk: unfortunately that doesn't let you specify a size limit for teh cache21:35
sshnaidm|afkyeah, this is more suitable for infra to have, so we can configure it in jobs..21:36
mwhahahathe problem is being able to hide docker hub creds in the overall deployment process21:36
mwhahahawe'21:36
mwhahahaso even if we get an account, how do we do that21:36
sshnaidm|afkmwhahaha, with zuul secrets, but why do we need them if we'll have a cache mirror?21:37
mwhahahawe don't have a cache mirror21:37
sshnaidm|afkyeah, when we'll have it with this caching registry ^21:38
mwhahahaso today we do anonymous auth on our end, what we'd want is auth to be handled in a centralized place21:38
mwhahahadocker registry allows you to do that for the proxy config21:38
mwhahahait was just a thought but as i looked into it, the less helpful the thing is21:38
mwhahahaif the docker registry had a cacheing backend that could have a size limit, it'd be great21:39
mwhahahabut it doesn't so it sucks21:39
sshnaidm|afkyeah, well, if infra will have this cache registry, we could use it - they have a plenty of space21:40
mwhahahadidn't sound like it21:40
mwhahahabut who knows21:40
sshnaidm|afkin worst case, just a weekly worker removing old images or kind of21:40
sshnaidm|afkI used this caching registry, worked quite good with docker, pulling only once from docker.io and then only uses cache21:41
sshnaidm|afkbut didn't try with podman21:42
mwhahahawhich caching registry?21:42
sshnaidm|afkhttps://docs.docker.com/registry/recipes/mirror/21:42
mwhahahaah yea it works fine21:42
mwhahahaover all it's decent, but it doesn't help for our use case at the moment21:42
mwhahahaworks with podman if you use the v2 container config21:42
sshnaidm|afkfor other case if we use team/pro account and need to authenticate, we'll need to configure creds in zuul, infra has examples, they publish images via zuul to docker.io21:44
sshnaidm|afkand use credentials for that I suppose21:44
mwhahahathe problem is getting it into tripleo21:44
mwhahahait's not ideal21:44
sshnaidm|afkwe can kind of preparing all containers before with creds, push them to local registry and to point tripleo to work with it instead of docker.io21:45
mwhahahayea possibly21:45
mwhahahawe'21:45
mwhahahawe'll figure it out21:46
* mwhahaha wanders off21:46
mwhahahalbragstad: i have no idea what fixes the module resolution other than getting the latest version of everything first21:46
*** TrevorV has quit IRC21:47
lbragstadmwhahaha yeah - testing it out again now21:48
lbragstadmwhahaha thanks for the help21:48
openstackgerritLuke Short proposed openstack/tripleo-heat-templates master: [WIP][DNM] Add Undercloud tuning template.  https://review.opendev.org/72565821:49
*** rfolco has quit IRC21:57
*** rfolco has joined #tripleo21:57
*** cfontain_ has joined #tripleo22:01
openstackgerritSteve Baker proposed openstack/tripleo-ansible master: WIP Use metalsmith version of metalsmith_instances  https://review.opendev.org/74803722:01
*** rcernin has joined #tripleo22:02
*** rfolco has quit IRC22:02
*** rfolco has joined #tripleo22:03
*** cfontain has quit IRC22:03
*** rfolco has quit IRC22:04
openstackgerritSagi Shnaidman proposed openstack/tripleo-ansible master: WIP - tripleo_container_manage: switch to use podman_containers plugin  https://review.opendev.org/71638122:06
*** rfolco has joined #tripleo22:06
*** rfolco has quit IRC22:08
*** rfolco has joined #tripleo22:12
*** bnemec has quit IRC22:14
*** rfolco has quit IRC22:16
*** rlandy has quit IRC22:18
*** rh-jelabarre has quit IRC22:21
*** rh-jelabarre has joined #tripleo22:21
*** zaneb has joined #tripleo22:22
*** cfontain_ has quit IRC22:22
*** tmazur has quit IRC22:28
*** saneax has joined #tripleo22:35
openstackgerritMerged openstack/tripleo-ci master: Disable docker mirror temporarily  https://review.opendev.org/74795322:56
*** cfontain has joined #tripleo23:00
*** tosky has quit IRC23:04
*** sanjayu_ has joined #tripleo23:09
openstackgerritTakashi Kajinami proposed openstack/tripleo-heat-templates master: Reintroduce keystone_cron container  https://review.opendev.org/73938323:10
*** saneax has quit IRC23:11
*** cfontain has quit IRC23:13
*** rfolco has joined #tripleo23:19
openstackgerritAmy Marrich (spotz) proposed openstack/tripleo-docs master: Clean up of wording and layout  https://review.opendev.org/74780923:25
*** jpalanis_ has quit IRC23:41
« Monday, 2020-08-24 Index Wednesday, 2020-08-26 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!