| *** ekultails has quit IRC | 00:01 | |
| rdogerrit | rdo-trunk created rdoinfo master: Bump rdoinfo train tags to latest upper-constraints https://review.rdoproject.org/r/24590 | 00:03 |
|---|---|---|
| rdogerrit | rdo-trunk created rdoinfo master: Bump rdoinfo ussuri-uc tags to latest upper-constraints https://review.rdoproject.org/r/24591 | 00:05 |
| rdogerrit | rdo-trunk proposed rdoinfo master: Promote CBS tags update for stein-testing https://review.rdoproject.org/r/24546 | 00:05 |
| rdogerrit | rdo-trunk created rdoinfo master: Promote CBS tags update for train-testing https://review.rdoproject.org/r/24592 | 00:07 |
| rdogerrit | Sagi Shnaidman proposed rdo-infra/ci-config master: Add internal projects to zuul builds https://review.rdoproject.org/r/24589 | 00:16 |
| rdogerrit | Merged rdo-infra/ci-config master: Return nginx.conf for local run https://review.rdoproject.org/r/24588 | 00:22 |
| *** tkajinam_ has quit IRC | 00:43 | |
| *** tkajinam has joined #rdo | 00:43 | |
| *** tkajinam_ has joined #rdo | 01:00 | |
| *** tkajinam has quit IRC | 01:00 | |
| *** bnemec has quit IRC | 01:31 | |
| rdogerrit | Merged rdo-infra/ci-config master: Add internal projects to zuul builds https://review.rdoproject.org/r/24589 | 02:47 |
| *** ktdreyer has quit IRC | 02:48 | |
| *** logan- has quit IRC | 02:48 | |
| *** logan_ has joined #rdo | 02:50 | |
| *** logan_ is now known as logan- | 02:50 | |
| *** osmanlicilegi has quit IRC | 02:56 | |
| *** osmanlicilegi has joined #rdo | 02:56 | |
| *** rfolco has joined #rdo | 02:57 | |
| *** rfolco has quit IRC | 03:02 | |
| *** apetrich has quit IRC | 03:13 | |
| *** rh-jelabarre has quit IRC | 03:22 | |
| *** brault has joined #rdo | 03:29 | |
| *** rlandy has quit IRC | 04:06 | |
| *** eggs has quit IRC | 04:12 | |
| rdogerrit | rdo-trunk created openstack/hardware-distgit rpm-master: python-hardware: failed to build 7c4c388 https://review.rdoproject.org/r/24593 | 04:20 |
| *** raukadah is now known as chandankumar | 04:46 | |
| *** udesale has joined #rdo | 04:49 | |
| *** ykarel|afk is now known as ykarel | 04:50 | |
| *** mcornea has joined #rdo | 04:51 | |
| rdogerrit | Chandan Kumar created rdo-jobs master: [DNM] https://review.opendev.org/#/c/703014/ https://review.rdoproject.org/r/24594 | 04:56 |
| *** mcornea has quit IRC | 04:57 | |
| *** zaneb has quit IRC | 05:08 | |
| *** jdandrea has quit IRC | 05:23 | |
| *** rpittau|afk has quit IRC | 05:34 | |
| *** rpittau|afk has joined #rdo | 05:34 | |
| rdogerrit | Yatin Karel proposed openstack/hardware-distgit rpm-master: Remove hardware-cardiff from package https://review.rdoproject.org/r/24578 | 05:35 |
| *** surpatil has joined #rdo | 05:48 | |
| *** soniya29 has joined #rdo | 05:49 | |
| *** udesale has quit IRC | 05:56 | |
| *** udesale has joined #rdo | 05:56 | |
| rdogerrit | Merged rdoinfo master: Pin neutron-lib and python-hardware to py2 compatible tag https://review.rdoproject.org/r/24565 | 05:57 |
| rdogerrit | rdo-trunk created openstack/hardware-distgit queens-rdo: python-hardware-0.23.0-1 https://review.rdoproject.org/r/24595 | 06:00 |
| rdogerrit | rdo-trunk created openstack/hardware-distgit rocky-rdo: python-hardware-0.23.0-1 https://review.rdoproject.org/r/24596 | 06:00 |
| *** soniya29 has quit IRC | 06:00 | |
| rdogerrit | rdo-trunk created openstack/hardware-distgit stein-rdo: python-hardware-0.23.0-1 https://review.rdoproject.org/r/24597 | 06:01 |
| *** marios has joined #rdo | 06:05 | |
| *** aufi has joined #rdo | 06:05 | |
| rdogerrit | Chandan Kumar proposed rdo-jobs master: [DNM] https://review.opendev.org/#/c/703014/ https://review.rdoproject.org/r/24594 | 06:08 |
| *** bandini has joined #rdo | 06:08 | |
| rdogerrit | Yatin Karel created openstack/hardware-distgit rpm-master: Dummy commit to rebuild python-hardware https://review.rdoproject.org/r/24598 | 06:09 |
| rdogerrit | Merged openstack/hardware-distgit rpm-master: Dummy commit to rebuild python-hardware https://review.rdoproject.org/r/24598 | 06:19 |
| rdogerrit | Merged openstack/hardware-distgit stein-rdo: python-hardware-0.23.0-1 https://review.rdoproject.org/r/24597 | 06:20 |
| rdogerrit | Merged openstack/hardware-distgit queens-rdo: python-hardware-0.23.0-1 https://review.rdoproject.org/r/24595 | 06:21 |
| rdogerrit | Merged openstack/hardware-distgit rocky-rdo: python-hardware-0.23.0-1 https://review.rdoproject.org/r/24596 | 06:25 |
| rdogerrit | Yatin Karel proposed rdoinfo master: Bump rdoinfo train tags to latest upper-constraints https://review.rdoproject.org/r/24590 | 06:31 |
| rdogerrit | Yatin Karel proposed rdoinfo master: Bump rdoinfo ussuri-uc tags to latest upper-constraints https://review.rdoproject.org/r/24591 | 06:32 |
| rdogerrit | Yatin Karel proposed rdoinfo master: Bump rdoinfo ussuri-uc tags to latest upper-constraints https://review.rdoproject.org/r/24591 | 06:34 |
| *** SurajPatil has joined #rdo | 06:36 | |
| *** tkajinam__ has joined #rdo | 06:37 | |
| *** SurajPatil has quit IRC | 06:37 | |
| *** tkajinam_ has quit IRC | 06:39 | |
| *** numans has quit IRC | 06:39 | |
| *** surpatil has quit IRC | 06:40 | |
| *** tkajinam_ has joined #rdo | 06:42 | |
| *** soniya29 has joined #rdo | 06:42 | |
| *** tkajinam__ has quit IRC | 06:44 | |
| *** surpatil has joined #rdo | 06:49 | |
| *** jaosorior has joined #rdo | 06:51 | |
| *** jfrancoa has joined #rdo | 06:52 | |
| *** lmiccini has joined #rdo | 07:04 | |
| rdogerrit | Merged rdoinfo master: Bump rdoinfo train tags to latest upper-constraints https://review.rdoproject.org/r/24590 | 07:06 |
| *** zaneb has joined #rdo | 07:09 | |
| *** chem_ has joined #rdo | 07:13 | |
| *** jlibosva has joined #rdo | 07:15 | |
| *** chem has quit IRC | 07:18 | |
| *** ramishra has quit IRC | 07:18 | |
| *** avivgt has quit IRC | 07:18 | |
| *** chasmo77 has quit IRC | 07:18 | |
| *** jrist has quit IRC | 07:19 | |
| *** migarcia has quit IRC | 07:19 | |
| *** lhinds has quit IRC | 07:19 | |
| *** tbarron has quit IRC | 07:19 | |
| *** tobias-urdin has quit IRC | 07:19 | |
| *** ianw has quit IRC | 07:19 | |
| *** mjturek has quit IRC | 07:19 | |
| *** zigo has quit IRC | 07:19 | |
| *** EmilienM has quit IRC | 07:19 | |
| *** rnoriega_ has quit IRC | 07:19 | |
| *** brtknr has quit IRC | 07:19 | |
| *** dmacpher has joined #rdo | 07:20 | |
| *** openstackstatus has quit IRC | 07:20 | |
| *** dmacpher_ has quit IRC | 07:22 | |
| *** slaweq has joined #rdo | 07:35 | |
| *** ykarel is now known as ykarel|lunch | 07:45 | |
| *** brault has quit IRC | 07:56 | |
| *** apevec has joined #rdo | 07:59 | |
| *** ramishra has joined #rdo | 08:05 | |
| *** avivgt has joined #rdo | 08:05 | |
| *** jrist has joined #rdo | 08:05 | |
| *** migarcia has joined #rdo | 08:05 | |
| *** tbarron has joined #rdo | 08:05 | |
| *** lhinds has joined #rdo | 08:05 | |
| *** brtknr has joined #rdo | 08:05 | |
| *** tobias-urdin has joined #rdo | 08:05 | |
| *** ianw has joined #rdo | 08:05 | |
| *** mjturek has joined #rdo | 08:05 | |
| *** zigo has joined #rdo | 08:05 | |
| *** EmilienM has joined #rdo | 08:05 | |
| *** rnoriega_ has joined #rdo | 08:05 | |
| *** maciejjozefczyk has joined #rdo | 08:06 | |
| *** jcapitao has joined #rdo | 08:07 | |
| *** zzzeek has quit IRC | 08:08 | |
| *** nijaba has quit IRC | 08:08 | |
| *** cylopez has joined #rdo | 08:09 | |
| *** rodolof has quit IRC | 08:09 | |
| *** tkajinam_ has quit IRC | 08:11 | |
| *** zzzeek has joined #rdo | 08:11 | |
| *** nijaba has joined #rdo | 08:11 | |
| *** rodolof has joined #rdo | 08:12 | |
| *** tesseract has joined #rdo | 08:16 | |
| *** tmazur has joined #rdo | 08:21 | |
| *** holser has joined #rdo | 08:27 | |
| *** rpittau|afk is now known as rpittau | 08:28 | |
| *** tosky has joined #rdo | 08:29 | |
| *** holser has quit IRC | 08:31 | |
| *** ykarel|lunch is now known as ykarel | 08:31 | |
| *** holser has joined #rdo | 08:32 | |
| *** iurygregory has joined #rdo | 08:32 | |
| *** dmellado has quit IRC | 08:34 | |
| *** dmellado has joined #rdo | 08:35 | |
| *** ccamacho has joined #rdo | 08:35 | |
| *** gfidente has joined #rdo | 08:38 | |
| *** amoralej|off is now known as amoralej | 08:40 | |
| amoralej | ykarel, centos8 images is fixed and I see weirdo jobs passing again, right? | 08:42 |
| ykarel | amoralej, yes those are passing | 08:43 |
| ykarel | amoralej, now new issue with centos7 image, | 08:43 |
| ykarel | i noticed that in tripleo jobs | 08:43 |
| amoralej | what's the new issue? | 08:43 |
| ykarel | amoralej, corrupted cirros image | 08:44 |
| ykarel | so tempest was failing | 08:44 |
| amoralej | that's in upstream jobs or the ones in SF zuul? | 08:44 |
| ykarel | amoralej, sf zuul | 08:44 |
| ykarel | amoralej, https://review.rdoproject.org/r/#/c/24546/ | 08:44 |
| ykarel | and similar train | 08:44 |
| ykarel | also periodic job failed | 08:44 |
| amoralej | are sf guys aware of it? | 08:46 |
| amoralej | Could not open '/var/lib/cinder/conversion/tmpfizqaKstandalone@tripleo_ceph': Image is not in qcow2 format | 08:47 |
| amoralej | mmm | 08:47 |
| amoralej | yeah, looks like corrupted image | 08:47 |
| ykarel | amoralej, yes | 08:48 |
| *** jpena|off is now known as jpena | 08:48 | |
| ykarel | amoralej, i confirmed ^^ by holding node for the job | 08:48 |
| amoralej | jpena, ^ | 08:48 |
| amoralej | could centos7 image be rebuilt in sf nodepool? | 08:48 |
| jpena | which one, upstream-centos-7 ? | 08:49 |
| ykarel | amoralej, http://paste.openstack.org/show/788516/ | 08:49 |
| jpena | Let me check | 08:49 |
| amoralej | cached cirros image is corrupted | 08:49 |
| amoralej | jpena, btw, centos8 image is fixed | 08:49 |
| ykarel | jpena, i just asked in sf-ops for this | 08:49 |
| *** ralonsoh has joined #rdo | 08:50 | |
| *** iurygregory_ has joined #rdo | 09:01 | |
| *** iurygregory has quit IRC | 09:03 | |
| *** rcernin_ has joined #rdo | 09:04 | |
| *** rcernin has quit IRC | 09:04 | |
| *** jpich has joined #rdo | 09:09 | |
| *** ktibi has joined #rdo | 09:17 | |
| *** xek has joined #rdo | 09:21 | |
| *** ktibi has quit IRC | 09:29 | |
| *** chem_ has quit IRC | 09:36 | |
| *** derekh has joined #rdo | 09:36 | |
| *** chem_ has joined #rdo | 09:37 | |
| *** apetrich has joined #rdo | 09:47 | |
| *** paramite has joined #rdo | 09:48 | |
| *** Jeffrey4l has quit IRC | 09:50 | |
| *** alexmcleod has joined #rdo | 09:56 | |
| amoralej | jpena, ykarel wrt https://lists.rdoproject.org/pipermail/users/2020-January/000660.html | 10:00 |
| amoralej | wdyt about doing some puppet promotion for centos8-train until we have oooq promotion on it? | 10:01 |
| amoralej | i think it'd be good for community users needing train on centos8 as starlingx | 10:01 |
| jpena | amoralej: sounds fine, we could at least have some sanity check | 10:01 |
| ykarel | amoralej, ok for me | 10:01 |
| amoralej | ack, i'll reply | 10:02 |
| *** dtantsur|afk is now known as dtantsur | 10:06 | |
| *** aufi has quit IRC | 10:07 | |
| rdogerrit | Alfredo Moralejo created rdo-infra/ci-config master: Add puppet promotion pipeline for centos8-train https://review.rdoproject.org/r/24599 | 10:17 |
| *** ccamposr__ has quit IRC | 10:18 | |
| *** Jeffrey4l has joined #rdo | 10:21 | |
| *** jaosorior has quit IRC | 10:29 | |
| *** ykarel is now known as ykarel|afk | 10:31 | |
| *** rcernin_ has quit IRC | 10:34 | |
| rdogerrit | Sorin Sbarnea (zbr) proposed rdo-jobs master: rdo-tox-molecule: switch nodeset to cloud-centos-8 https://review.rdoproject.org/r/24584 | 10:36 |
| *** apetrich has quit IRC | 10:38 | |
| *** apetrich has joined #rdo | 10:40 | |
| *** ccamposr has joined #rdo | 10:41 | |
| *** fmount|afk is now known as fmount | 10:47 | |
| rdogerrit | Sorin Sbarnea (zbr) proposed rdo-jobs master: rdo-tox-molecule: switch nodeset to cloud-centos-8 https://review.rdoproject.org/r/24584 | 10:50 |
| *** soniya29 has quit IRC | 11:01 | |
| rdogerrit | Sorin Sbarnea (zbr) proposed rdo-jobs master: rdo-tox-molecule: switch nodeset to cloud-centos-8 https://review.rdoproject.org/r/24584 | 11:03 |
| rdogerrit | Alfredo Moralejo proposed rdo-infra/ci-config master: Add puppet promotion pipeline for centos8-train https://review.rdoproject.org/r/24599 | 11:12 |
| *** bhagyashris has joined #rdo | 11:16 | |
| *** ccamel has joined #rdo | 11:16 | |
| *** aufi has joined #rdo | 11:16 | |
| *** camelCaser has quit IRC | 11:16 | |
| *** soniya29 has joined #rdo | 11:27 | |
| *** surpatil has quit IRC | 11:33 | |
| *** ykarel|afk is now known as ykarel | 11:59 | |
| *** pcaruana has joined #rdo | 11:59 | |
| rdogerrit | Sagi Shnaidman created rdo-infra/ci-config master: Read password before starting Grafana services https://review.rdoproject.org/r/24600 | 11:59 |
| *** sshnaidm|afk is now known as sshnaidm|off | 12:00 | |
| *** zbr|rover is now known as zbr|drover | 12:09 | |
| *** dsneddon has quit IRC | 12:12 | |
| *** jcapitao is now known as jcapitao|afk | 12:13 | |
| rdogerrit | Sorin Sbarnea (zbr) proposed rdo-jobs master: rdo-tox-molecule: switch nodeset to cloud-centos-8 https://review.rdoproject.org/r/24584 | 12:13 |
| rdogerrit | Sorin Sbarnea (zbr) proposed rdo-jobs master: rdo-tox-molecule: switch nodeset to cloud-centos-8 https://review.rdoproject.org/r/24584 | 12:13 |
| rdogerrit | Merged rdo-infra/ci-config master: Read password before starting Grafana services https://review.rdoproject.org/r/24600 | 12:17 |
| *** rfolco has joined #rdo | 12:27 | |
| *** akahat has joined #rdo | 12:32 | |
| *** eharney has joined #rdo | 12:37 | |
| *** iurygregory_ is now known as iurygregory | 12:40 | |
| *** dmellado has quit IRC | 12:41 | |
| *** dmellado has joined #rdo | 12:44 | |
| *** udesale_ has joined #rdo | 12:45 | |
| *** jpena is now known as jpena|lunch | 12:46 | |
| *** dsneddon has joined #rdo | 12:48 | |
| *** marios is now known as marios|call | 12:48 | |
| *** udesale has quit IRC | 12:48 | |
| rdogerrit | Merged rdo-infra/ci-config master: Add puppet promotion pipeline for centos8-train https://review.rdoproject.org/r/24599 | 12:49 |
| *** ykarel is now known as ykarel|afk | 12:50 | |
| *** rh-jelabarre has joined #rdo | 12:51 | |
| *** jdandrea has joined #rdo | 12:55 | |
| *** dsneddon has quit IRC | 12:56 | |
| *** jdandrea has quit IRC | 12:57 | |
| *** marios|call is now known as marios | 12:58 | |
| *** dsneddon has joined #rdo | 13:06 | |
| *** rlandy has joined #rdo | 13:11 | |
| *** aufi has quit IRC | 13:17 | |
| *** jcapitao|afk is now known as jcapitao | 13:19 | |
| *** imcleod has joined #rdo | 13:25 | |
| *** gfidente has quit IRC | 13:27 | |
| *** derekh has quit IRC | 13:32 | |
| *** jpena|lunch is now known as jpena | 13:39 | |
| *** gfidente has joined #rdo | 13:42 | |
| *** pcaruana has quit IRC | 13:45 | |
| *** soniya29 has quit IRC | 13:47 | |
| *** ykarel|afk is now known as ykarel | 13:51 | |
| rdogerrit | Sagi Shnaidman proposed rdo-jobs master: WIP: rhel-8 molecule tripleo-ansible job https://review.rdoproject.org/r/23670 | 13:51 |
| *** leanderthal has joined #rdo | 13:53 | |
| rdogerrit | Daniel Alvarez proposed rdoinfo master: Bump OVS/OVN to 2.12 in RDO Train https://review.rdoproject.org/r/23960 | 14:01 |
| *** derekh has joined #rdo | 14:02 | |
| *** nweinber__ has joined #rdo | 14:11 | |
| *** ekultails has joined #rdo | 14:16 | |
| *** pcaruana has joined #rdo | 14:22 | |
| *** amoralej is now known as amoralej|lunch | 14:26 | |
| *** aedc has joined #rdo | 14:36 | |
| *** mcornea has joined #rdo | 14:39 | |
| *** jdandrea has joined #rdo | 14:53 | |
| *** aedc has quit IRC | 14:55 | |
| *** marios is now known as marios|call | 14:57 | |
| rdogerrit | Sorin Sbarnea (zbr) proposed rdo-jobs master: rdo-tox-molecule: switch nodeset to cloud-centos-8 https://review.rdoproject.org/r/24584 | 15:02 |
| *** zbr|drover has quit IRC | 15:08 | |
| *** lmiccini has quit IRC | 15:10 | |
| *** amoralej|lunch is now known as amoralej | 15:12 | |
| *** lmiccini has joined #rdo | 15:12 | |
| *** zbr has joined #rdo | 15:16 | |
| *** zbr is now known as zbr|drover | 15:16 | |
| *** jpich has quit IRC | 15:39 | |
| *** jpich has joined #rdo | 15:39 | |
| *** jcapitao is now known as jcapitao|afk | 15:48 | |
| *** aedc has joined #rdo | 15:52 | |
| *** leanderthal has quit IRC | 15:52 | |
| *** aufi has joined #rdo | 15:58 | |
| *** cylopez has quit IRC | 15:59 | |
| *** udesale_ has quit IRC | 16:01 | |
| *** udesale has joined #rdo | 16:03 | |
| *** jcapitao|afk is now known as jcapitao | 16:03 | |
| *** migarcia is now known as migarcia|away | 16:06 | |
| *** migarcia|away is now known as migarcia | 16:06 | |
| *** udesale has quit IRC | 16:10 | |
| rdogerrit | Sorin Sbarnea (zbr) proposed rdo-jobs master: rdo-tox-molecule: switch nodeset to cloud-centos-8 https://review.rdoproject.org/r/24584 | 16:14 |
| zbr|drover | tristanC: hi! do you have few minutes to chat about the two upstream patches related to docker? | 16:16 |
| *** slaweq has quit IRC | 16:16 | |
| tristanC | zbr|drover: well i'm not familiar with docker, i only use podman... but maybe i can help? what's up | 16:17 |
| zbr|drover | one simple is revoke-sudo chokes on rdo because our username is not called zuul, so I proposed https://review.opendev.org/#/c/703065/8 | 16:18 |
| *** bandini has quit IRC | 16:18 | |
| *** marios|call is now known as marios | 16:18 | |
| zbr|drover | the more complex one is related to install-docker, which fails on centos-8 because docker-ce repository has some outdated dependencies. | 16:19 |
| *** maciejjozefczyk has quit IRC | 16:19 | |
| zbr|drover | it can install succesfully only with --nobest or by manually pre-installing a specific rpm before trying to install docker itself, fix proposed at https://review.opendev.org/#/c/703053/1 | 16:20 |
| zbr|drover | you don't need to know docker, is rpm stuff :D | 16:20 |
| *** lmiccini has quit IRC | 16:24 | |
| *** marios is now known as marios|out | 16:28 | |
| tristanC | zbr|drover: i left new comments on the changes. it would be good to have feedback from the other cores and user affected by the issue you are trying to address. | 16:28 |
| zbr|drover | ouch... so you are against because it breaks your hack :D | 16:30 |
| tristanC | zbr|drover: it's not because of the hack, it's because you made revoke-sudo requires SETUID capabilities, and this is going to break user who do not give SETUID to their jobs using revoke-sudo | 16:31 |
| dmsimard | zbr|drover: docker doesn't have a centos8 repo yet, only 7 | 16:33 |
| dmsimard | https://download.docker.com/linux/centos/ | 16:33 |
| dmsimard | I actually ran into a similar issue recently | 16:33 |
| *** marios|out has quit IRC | 16:33 | |
| zbr|drover | tristanC: that two tasks cannot fail, the only issue i see there is that it would not work with your hack, but I could adapt the code to keep the hardcoded removal of that file. | 16:34 |
| zbr|drover | i am more worried to make the role behave correctly, as now is not working with the newer centos-8 image | 16:34 |
| zbr|drover | probably because zuul is part wheel group (i suspect) | 16:35 |
| zbr|drover | if you update the image to mimic the hack, i would not mind as I would have one less blocking patches | 16:36 |
| tristanC | zbr|drover: well, why zuul is part of the wheel group? | 16:36 |
| zbr|drover | tristanC: no idea, I did not add it myself, but I can confirm that my proposed code does work. | 16:36 |
| zbr|drover | tristanC: look at https://review.rdoproject.org/r/#/c/24584/ (molecule job) | 16:37 |
| *** udesale has joined #rdo | 16:37 | |
| *** ykarel is now known as ykarel|away | 16:37 | |
| zbr|drover | if you try it without using the revoke-sudo patch, it will fail at the task that verifies that you no longer have sudo (last one) | 16:38 |
| zbr|drover | meaning that it removes the file, but that is not enough, sudo still works after. | 16:38 |
| tristanC | zbr|drover: oh, i see, my bad, let me fix the image then | 16:38 |
| zbr|drover | tristanC: thanks. it should buy us more time to properly fix the upstream role | 16:39 |
| zbr|drover | fixing upstream role is our interest, or we will endup drowning in rdo workarounds. | 16:39 |
| tristanC | zbr|drover: i fail to see how zuul is being added to the wheel group, are you sure of that issue? | 16:39 |
| zbr|drover | tristanC: not sure, as I didn't start the image myself, by i only read the logs (look inside json file, contains all details) | 16:40 |
| zbr|drover | the console is kinda brief, but all 3 tasks are with more detail in json file. | 16:40 |
| tristanC | zbr|drover: i'll rebuild the image with https://softwarefactory-project.io/r/17337 | 16:41 |
| tristanC | that's actually a bug on our side, since we refactored the virt-customize stuff into role, we missed that sudoer filename fix | 16:42 |
| zbr|drover | i don't really know why we didn't use the "API" (gpasswd) | 16:42 |
| tristanC | zbr|drover: build is running here: https://softwarefactory-project.io/nodepool-log/cloud-centos-8-0000000005.log | 16:48 |
| rdogerrit | Merged rdoinfo master: Promote CBS tags update for train-testing https://review.rdoproject.org/r/24592 | 16:49 |
| tristanC | once this is completed and updloaded, i think your molecule job will run without depending on https://review.opendev.org/#/c/703065/8 | 16:49 |
| zbr|drover | tristanC: maybe you also happen to know a trick that I could use to avoid being blocked by the install-docker patch. | 16:51 |
| tristanC | then for the docker fix, i can't tell what is the best thing to do. I don't mind your change if other install-docker user are ok with it, otherwise i think we should wait for docker to provide a version that works on centos | 16:51 |
| zbr|drover | i tried to run the task in run-pre but discovered that this runs too late, zuul roles being executed before job pre, thus failing. | 16:52 |
| tristanC | zbr|drover: you can work around the issue by providing your own pre-run playbook and make it install the right thing before including the install-docker role | 16:52 |
| zbr|drover | tristanC: we cannot wait, tell this to weshay_ | 16:52 |
| zbr|drover | they knew about this bug since november, Docker Inc has Red Hat stuff at the back-of-back of their priorities.... | 16:53 |
| *** rpittau is now known as rpittau|afk | 16:53 | |
| tristanC | zbr|drover: let's backup a bit, why are you using docker on centos? | 16:53 |
| zbr|drover | most of molecule scenarios are using docker backend for testing ansible code. | 16:54 |
| tristanC | zbr|drover: why molecule doesn't use podman instead? | 16:54 |
| zbr|drover | podman is not ready to be used as replacement, and I bet the driver author can back my story, sshnaidm|off (sadly off) | 16:54 |
| tristanC | zbr|drover: what is not ready? | 16:54 |
| zbr|drover | in fact even the docker from centos was too old and buggy to be used for testing, and that is what forced us to use the upstream version. | 16:56 |
| amoralej | ykarel|away, we have first puppet promotion for centos8-train | 16:56 |
| tristanC | zbr|drover: what bugs are you referring to? I need to step away shortly for lunch, but i'd be happy to look into why can't use podman after | 16:56 |
| zbr|drover | plan is to also enable podman and to switch scenarios by scenario to podman when we see it working. but that scenario by scenario | 16:57 |
| zbr|drover | the job itself will have to provide both tools/backends. | 16:57 |
| *** dtantsur is now known as dtantsur|afk | 16:58 | |
| ykarel|away | amoralej, cool | 16:58 |
| tristanC | zbr|drover: if podman can't be used in place of docker on centos, then that's a podman issue we need to fix. so please, let me know what is not working and why the docker version you are installing is better suited for molecule | 16:58 |
| *** udesale has quit IRC | 17:00 | |
| zbr|drover | tristanC: just one example: https://github.com/ansible/molecule/pull/2484 | 17:01 |
| zbr|drover | and you know very well which version we ship | 17:01 |
| zbr|drover | we had to disable molecule tests on podman driver, because of that. | 17:02 |
| weshay|ruck | hello | 17:02 |
| weshay|ruck | what do I need to know here? | 17:02 |
| zbr|drover | when we will have 1.6 we can start testing, but until then, molecule needs a working docker. | 17:02 |
| zbr|drover | weshay|ruck: tristan was curious why we really need docker for molecule job | 17:03 |
| rdogerrit | Chandan Kumar created rdo-jobs master: [wip] fs020 with latest master podman https://review.rdoproject.org/r/24601 | 17:04 |
| *** jfrancoa has quit IRC | 17:04 | |
| *** imcleod has quit IRC | 17:04 | |
| weshay|ruck | zbr|drover, like.. docker vs. podman or containers vs. nodepool node | 17:04 |
| zbr|drover | 1st | 17:05 |
| weshay|ruck | what is the status of molecule support for podman | 17:05 |
| zbr|drover | sadly today sagi is off as he is the area knowledge on this | 17:05 |
| weshay|ruck | last I knew about it.. we were helping that along | 17:05 |
| weshay|ruck | right | 17:05 |
| *** alexmcleod has quit IRC | 17:05 | |
| weshay|ruck | afaik folks.. molecule had some issues when running w/ podman and we working to fix it | 17:06 |
| zbr|drover | mainly molecule does not have an active CI with podman and requires 1.5.1+ | 17:06 |
| weshay|ruck | and I'm afaid I don't have all the specifics noted | 17:06 |
| zbr|drover | so use of podman is out of the question for the moment, we need to keep using docker until podman driver becomes a valid replacement | 17:07 |
| weshay|ruck | zbr|drover, last work I see from sshnaidm|off was in october https://github.com/sshnaidm/molecule/commits/master | 17:07 |
| zbr|drover | which will happen, I am sure | 17:07 |
| zbr|drover | in fact podman driver also requires ansible 2.9+ if it remember well | 17:08 |
| weshay|ruck | that merged | 17:08 |
| sshnaidm|off | what is the current issue with podman-molecule? | 17:08 |
| weshay|ruck | there is the man | 17:09 |
| sshnaidm|off | there are various problems with specific versions of podman and cgroups | 17:09 |
| weshay|ruck | sshnaidm|off, are there any remaining issues w/ podman + molecule that you are aware of? | 17:09 |
| zbr|drover | sshnaidm|off: hi! i was trying to explain tristan that we still need docker for rdo-tox-molecule | 17:09 |
| *** jcapitao has quit IRC | 17:09 | |
| *** aufi has quit IRC | 17:09 | |
| sshnaidm|off | weshay|ruck, I know the issue with with cgroups v2 only | 17:09 |
| zbr|drover | maybe tristanC does not know that each molecule scenario/test declares which backend it wants to use, so is not the job that decides what is used, is the tests inside. | 17:10 |
| weshay|ruck | tristanC, dmsimard what is the issue w/ also having docker avail? | 17:10 |
| sshnaidm|off | zbr|drover, can we at least try to run it with podman? to know how much it's bad/good | 17:10 |
| sshnaidm|off | if we don't run it, we'll never have it working | 17:11 |
| zbr|drover | sshnaidm|off: weshay|ruck : read this to underatand what we are debating about https://review.opendev.org/#/c/703053/1 | 17:12 |
| zbr|drover | i have docker-ce on centos-8, since ~2 mo ago, works fine. we just need to hack the install a little bit. | 17:13 |
| zbr|drover | real all my comments, including link to https://github.com/geerlingguy/ansible-role-docker/issues/171 | 17:13 |
| tristanC | weshay|ruck: iiuc, docker doesn't distribute working centos-8 package, https://download.docker.com/linux/centos/ | 17:13 |
| weshay|ruck | oh loveerly | 17:14 |
| weshay|ruck | so they got our flowers re: "hey docker, we think you suck" | 17:14 |
| sshnaidm|off | zbr|drover, I'm not against using docker temporary until we ensure it works good with podman, but if we never run jobs with molecule-podman we'll be stuck with docker forever | 17:14 |
| zbr|drover | tristanC: the el7 repo is working, | 17:14 |
| weshay|ruck | zbr|drover, sounds like it's no docker for centos-8 dude | 17:14 |
| zbr|drover | weshay|ruck: you call me stupid? | 17:15 |
| tristanC | zbr|drover: the molecule issue you link seems to be about a missing podman cp command. iirc podman does have a cp command | 17:15 |
| weshay|ruck | ? | 17:15 |
| sshnaidm|off | tristanC, which issue? | 17:15 |
| tristanC | sshnaidm|off: +1, please test using podman in place of docker and let see what is broken and what needs to be fixed | 17:15 |
| tristanC | sshnaidm|off: the issue zbr|drover linked earlier, it's https://github.com/ansible/molecule/pull/2484 | 17:16 |
| tristanC | (afk lunch now | 17:16 |
| sshnaidm|off | tristanC, I'm for that, but we may require something working meanwhile if there is a problem with podman, always good to have a backup | 17:16 |
| zbr|drover | https://www.google.com/search?client=firefox-b-d&q=instaling+docker+on+centos+8 -- any of the top 10 results will tell you how to do it. | 17:16 |
| sshnaidm|off | zbr|drover, I need more details in this issue | 17:19 |
| sshnaidm|off | zbr|drover, what podman version, cgroups version, ansible version | 17:19 |
| weshay|ruck | tristanC, so.. zbr|drover is saying the centos-7 of docker works on 8 | 17:19 |
| weshay|ruck | perhaps we can check that out | 17:19 |
| *** chandankumar is now known as raukadah | 17:20 | |
| weshay|ruck | tristanC, I guess it's how you feel about https://review.opendev.org/#/c/703053/1/roles/install-docker/tasks/upstream-yum.yaml | 17:21 |
| *** tosky has quit IRC | 17:21 | |
| *** holser has quit IRC | 17:23 | |
| sshnaidm|off | tristanC, the default centos8 podman version 1.4.2-stable2 won't work because of https://github.com/containers/libpod/pull/3391/commits/c962d214e6156bd4d89e0eb20246549eca272ba7 | 17:24 |
| sshnaidm|off | and this commit is only from 1.4.3 :( | 17:24 |
| sshnaidm|off | if there is way to install newer podman package it will help | 17:24 |
| *** holser has joined #rdo | 17:29 | |
| *** holser has quit IRC | 17:29 | |
| *** tmazur has quit IRC | 17:29 | |
| *** jpich has quit IRC | 17:30 | |
| tristanC | zbr|drover: i didn't realize the backend was declared in the molecule scenario. Since docker and podman are similar container-runtime, could molecule use podman instead of docker if the podman command is available? | 17:32 |
| *** aedc has quit IRC | 17:32 | |
| tristanC | weshay|ruck: zbr|drover: in the meantime, i think we can just bake the containerd.io package in the cloud-centos-8 image | 17:33 |
| tristanC | sshnaidm|off: i see, it seems like a nice to have fix in the podman distributed centos. not sure what is the cherry-pick/backport/patch process for the package though. isn't a more recent version available through centos-cr or stream or modularity? | 17:34 |
| zbr|drover | tristanC: just finished talk with wes, we got in idea, can we pre-build an node image and use that? | 17:35 |
| rdogerrit | Chandan Kumar proposed rdo-jobs master: [wip] fs020 with latest master podman https://review.rdoproject.org/r/24601 | 17:35 |
| zbr|drover | like a "centos-8-preloaded" that has all extra deps/hacks on it? | 17:35 |
| tristanC | zbr|drover: yes sure, but that's just postponing the main issue, that is we shouldn't be using docker on centos-8 | 17:35 |
| sshnaidm|off | tristanC, no idea.. there is not even ansible or pip in c8, what's wrong with this distro.. | 17:36 |
| zbr|drover | tristanC: do you want me to propose an ubuntu image? that is what we have upstream, and works fine ;) | 17:36 |
| tristanC | and if we are going to taint the image with docker, then we might as well install the right podman version | 17:36 |
| zbr|drover | tristanC: that image is special for tox/molecule is not for deployments | 17:37 |
| zbr|drover | is a "qa" system, not one that is supposed to match what we ship. | 17:37 |
| sshnaidm|off | tristanC, would be nice | 17:38 |
| zbr|drover | the reality is that for molecule I we will need an image that has: docker, podman, libvirt maybe even vagrant. | 17:38 |
| zbr|drover | all the virtualization backends that may be needed by various functional tests. | 17:39 |
| tristanC | sshnaidm|off: let me have a look, perhaps it's not difficult to get a more recent podman in centos. for pip you need to install python36 ( e.g. https://softwarefactory-project.io/paste/show/1717/ ) | 17:39 |
| zbr|drover | sshnaidm|off: i fixed pip, just remember is called pip3 ;) | 17:40 |
| sshnaidm|off | :E | 17:40 |
| zbr|drover | see my merged fix from https://review.opendev.org/#/c/702978/ | 17:40 |
| sshnaidm|off | and why do they hate ansible? | 17:40 |
| zbr|drover | sshnaidm|off: i think is not hate, is business model ;) | 17:41 |
| tristanC | zbr|drover: iiuc, because molecule test declare the container runtime, then i think you should just skip or fail the scenario that declares docker on centos | 17:42 |
| zbr|drover | the release stream, subscription stuff,... but I am personally not afraid to use pip, in the end YOLO | 17:42 |
| tristanC | zbr|drover: i would fix molecule to not declare the container runtime, so that molecule could choose podman or docker based on what's available | 17:42 |
| zbr|drover | welll, I think I know how to name the magic image, "centos-8-yolo", sounds ok? i am serious. | 17:43 |
| tristanC | zbr|drover: what's the purpose of such image? | 17:43 |
| zbr|drover | tristanC: it is possible to override the driver via an evironment variable but in practice it will work in less than 1/5 cases. | 17:43 |
| zbr|drover | because drivers have different options needed, in very few cases you can get around and switch it with the override env var. | 17:44 |
| zbr|drover | worked fine for few basic cases between podman and docker | 17:44 |
| tristanC | zbr|drover: yeah, that sounds like a limitation... | 17:44 |
| zbr|drover | and my plan is to simplify this, especially between podman and docker. | 17:45 |
| tristanC | zbr|drover: podman and docker are meant to be compatible and swapable | 17:45 |
| zbr|drover | like adding a native fallback between them | 17:45 |
| zbr|drover | tristanC: it seems that marketing worked better for you than for me :D | 17:45 |
| tristanC | zbr|drover: ok, so once molecule can fallback, we can drop docker from centos-8 right? | 17:45 |
| *** jpena is now known as jpena|off | 17:45 | |
| zbr|drover | my desktop is a macos (yes...), and I use docker remoting perfectly (including with molecule), podman remoting system is far from usable. | 17:46 |
| zbr|drover | tristanC: yes, that was the idea. | 17:47 |
| tristanC | zbr|drover: alright, then let's not bother with a special image name, let's bake that docker fix until molecule can use podman instead, then we'll drop docker | 17:47 |
| zbr|drover | if i suceed to make the backend-fallback work, we will be able remove docker without changing any repository or job definition. | 17:47 |
| zbr|drover | tristanC: is risky, because newer podman may go into conflict, I would not want to have of reference image backed with docker. | 17:48 |
| tristanC | zbr|drover: sounds like the right plan to me | 17:48 |
| zbr|drover | i know is easier but i am afraid someone will be really pissed off. | 17:49 |
| tristanC | zbr|drover: why would 'newer podman may go into conflict' ? | 17:49 |
| zbr|drover | for example current version of upstream podman had some bugs which made impossible to install both. they installed man-packages that overlapped. | 17:50 |
| zbr|drover | i think is already fixed, but you got the idea. | 17:50 |
| tristanC | zbr|drover: that is news to me, i never had issue updating centos or fedora packages... | 17:51 |
| zbr|drover | still, if I have a YOLO machine, I could even bake a latest podman from upsream on it to test if newer version version works bettter than vanilla. | 17:51 |
| *** gfidente has quit IRC | 17:51 | |
| zbr|drover | tristanC: read https://github.com/containers/libpod/issues/4791 still open :D | 17:52 |
| tristanC | zbr|drover: looking at this from another angle, when molecule test docker on centos-8, it is testing something nobodies is going to use now because docker on centos-8 does not work yet. Thus I don't understand why we are going through so many work around for such a thing... Fixing molecule to use podman instead sounds like the best things to do now. | 17:52 |
| zbr|drover | tristanC: you are looking at it at the wrong angle: these jobs are not testing software compatibility with OS from the image itself, they are testing virtualized stuff inside docker/podman containers. | 17:54 |
| tristanC | zbr|drover: right, that can happens when you install extra stuff in /usr/share. It sounds like docker-ce should be using /usr/local/share or /opt/docker to avoid conflict with the distribution | 17:54 |
| zbr|drover | AFAIK, we could easily use the ubuntu image that is already used by openstack-tox-molecule job. | 17:54 |
| zbr|drover | which is in fact doing the same thing: installing docker-ce from upstream, and not from ubuntu repos. | 17:55 |
| zbr|drover | but this would not play well with dogfooding, so better to stick on a redhat platfrom | 17:56 |
| tristanC | zbr|drover: is there another issue you had with podman beside the cp directory thing that is already fixed in 1.4.3 ? | 17:57 |
| zbr|drover | tristanC: is NOT fixed, I am 100% sure, read all comments in the bug. The first attempt to fix was in 1.4.3, but the real fix was in 1.5.1 | 17:59 |
| *** aedc has joined #rdo | 17:59 | |
| zbr|drover | i know for sure, as I faces it few weeks back, there is a bz about it. | 17:59 |
| tristanC | zbr|drover: the bug is fixed https://github.com/containers/libpod/issues/3384 | 17:59 |
| *** derekh has quit IRC | 18:00 | |
| tristanC | zbr|drover: i'm curious what is not working and why can't molecule use podman, but until you have implemented the fallback, iiuc, the plan is to install the containerd.io package in the cloud-centos-8 image so that you have some time to do the molecule fallback implementation right? | 18:02 |
| tristanC | zbr|drover: do you need help to add the containerd.io package in the cloud-centos-8 image? | 18:02 |
| zbr|drover | tristanC: i think I can do it. so your idea is to install only this package, so prepare to so the upstream role will work. | 18:03 |
| tristanC | zbr|drover: seems like the easiest thing to do. perhaps also bake the docker-ce.repo in case the containerd.io package depends on some extra requirements provided by docker | 18:03 |
| zbr|drover | this may be a safe hack, as it would not install docker for everyone. sounds like a good compromise to me. | 18:03 |
| zbr|drover | the repository needs to be installed first if I remember well. i will do some tests, and attempt to do minimal alteration to make it work. | 18:04 |
| tristanC | zbr|drover: thanks, let me know when the changes are up for review | 18:05 |
| zbr|drover | all this wasted time because Docker Inc, is slow to fix things. | 18:05 |
| zbr|drover | sure, and thanks a LOT about helping on that. | 18:05 |
| tristanC | zbr|drover: you're welcome :) | 18:05 |
| *** aedc has quit IRC | 18:06 | |
| zbr|drover | cool, apparently i can install that rpm directly without setting the rpm, this will make task easier. | 18:09 |
| tristanC | zbr|drover: this might be helpful too: https://softwarefactory-project.io/r/#/c/17339/1/nodepool/virt_images/README.md | 18:11 |
| *** ralonsoh has quit IRC | 18:11 | |
| *** amoralej is now known as amoralej|off | 18:12 | |
| zbr|drover | i already did something similar. if I have time I will propose some changes to make it even easier to test locally. | 18:12 |
| *** imcleod has joined #rdo | 18:13 | |
| *** ekultails has quit IRC | 18:23 | |
| *** ekultails has joined #rdo | 18:24 | |
| *** social has quit IRC | 18:39 | |
| *** bnemec has joined #rdo | 18:44 | |
| *** bnemec is now known as beekneemech | 18:45 | |
| *** jlibosva has quit IRC | 18:48 | |
| *** pcaruana has quit IRC | 18:51 | |
| *** migarcia is now known as migarcia|away | 18:54 | |
| zbr|drover | tristanC: review https://softwarefactory-project.io/r/#/c/17341/ -- i want to split changes | 19:09 |
| zbr|drover | to avoid introducing bugs | 19:09 |
| tristanC | zbr|drover: wfm | 19:10 |
| zbr|drover | tristanC: if you do not mind i may want to add some extra checks like ansible-lint on that config repo, when I find the time. | 19:11 |
| tristanC | zbr|drover: that would be useful, but add them to https://softwarefactory-project.io/cgit/software-factory/sf-config/tree/ansible/roles/sf-repos/files/config so that all sf users would benefit from them | 19:13 |
| zbr|drover | in fact we use tox-linters to run these, so probably I do not need to add any new playbooks/roles, only to enable the job and add few config files. | 19:14 |
| zbr|drover | but only one line of zuul config, likely. | 19:14 |
| zbr|drover | another option could be to extend the purpose of config-check and make it run extra tests when the right files are detected. | 19:16 |
| zbr|drover | we could even bypass the need for tox. | 19:17 |
| tristanC | zbr|drover: there are ci implication too, we run many config-check/config-update on each software-factory changes as part of the integration test. I would suspect a tox setup to slows down the whole ci. | 19:17 |
| tristanC | and because you need the first config-check to succeed in order to setup nodepool-provider, we run config-check on the executor directly | 19:18 |
| zbr|drover | tristanC: do you have support for caching folder between builds? | 19:18 |
| tristanC | zbr|drover: zuul doesn't let you do that unfortunately | 19:18 |
| zbr|drover | not yet, we will get there. | 19:18 |
| *** tesseract has quit IRC | 19:32 | |
| zbr|drover | tristanC: also updated https://softwarefactory-project.io/r/#/c/17340/ - which could prove useful for future updates | 19:34 |
| *** tosky has joined #rdo | 19:41 | |
| *** aedc has joined #rdo | 19:52 | |
| *** openstackstatus has joined #rdo | 20:02 | |
| *** ChanServ sets mode: +v openstackstatus | 20:02 | |
| *** jtomasek has quit IRC | 20:03 | |
| *** imcleod has quit IRC | 20:17 | |
| *** aedc has quit IRC | 20:26 | |
| *** haleyb has quit IRC | 20:28 | |
| *** haleyb has joined #rdo | 20:31 | |
| *** rh-jelabarre has quit IRC | 20:36 | |
| *** nweinber__ has quit IRC | 20:36 | |
| *** aedc has joined #rdo | 20:36 | |
| *** eharney has quit IRC | 20:59 | |
| *** imcleod has joined #rdo | 21:10 | |
| zbr|drover | tristanC: take a look at https://softwarefactory-project.io/r/#/c/17343/ -- result of last hours | 21:25 |
| zbr|drover | there is a lot of duplication across elements/ and virt_images/ -- i am not sure why. | 21:26 |
| *** rlandy has quit IRC | 21:36 | |
| *** aedc has quit IRC | 21:42 | |
| *** paramite has quit IRC | 21:54 | |
| tristanC | zbr|drover: elements/virt-customize is the legacy, we are migrating to virt_images | 21:55 |
| zbr|drover | tristanC: and what stops us from removing them? | 21:56 |
| zbr|drover | few more images to "port" to the new one? | 21:56 |
| tristanC | zbr|drover: the time it would take to refactor the standalone playbook elements into the new virt_images roles | 21:56 |
| zbr|drover | tristanC: you got yourself a buddy to help, i think i like building images. | 21:57 |
| zbr|drover | i was impressed how fast it it run locally and that it was mostly idempotent | 21:58 |
| tristanC | zbr|drover: that's nice to hear, but we likely need to groom and plan such effort with the team first | 21:58 |
| zbr|drover | sure, small bits | 21:59 |
| tristanC | e.g. before rewritting all the image, we would create a story here https://tree.taiga.io/project/morucci-software-factory/ first | 21:59 |
| *** eharney has joined #rdo | 22:26 | |
| *** rh-jelabarre has joined #rdo | 22:33 | |
| *** xek has quit IRC | 22:36 | |
| *** jbrooks has quit IRC | 22:42 | |
| *** imcleod has quit IRC | 22:45 | |
| *** slaweq has joined #rdo | 22:51 | |
| *** apevec has quit IRC | 22:57 | |
| *** cyberworm54 has quit IRC | 23:10 | |
| *** cyberworm54 has joined #rdo | 23:10 | |
| *** slaweq has quit IRC | 23:14 | |
| *** imcleod has joined #rdo | 23:26 | |
| *** rh-jelabarre has quit IRC | 23:27 | |
| *** mcornea has quit IRC | 23:40 | |
| rdogerrit | Sagi Shnaidman created rdo-infra/ci-config master: Increase interval for zuul stats https://review.rdoproject.org/r/24602 | 23:58 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!