Wednesday, 2020-02-19

*** jmasud has quit IRC		00:26
*** jmasud has joined #oooq		00:28
*** jmasud has quit IRC		00:40
*** jmasud has joined #oooq		01:49
*** skramaja has quit IRC		01:51
*** jmasud has quit IRC		04:12
*** jmasud has joined #oooq		04:14
*** saneax has joined #oooq		04:31
*** saneax has quit IRC		04:35
*** saneax has joined #oooq		04:38
*** udesale has joined #oooq		04:50
*** ratailor has joined #oooq		04:59
*** ykarel\|afk is now known as ykarel		05:10
*** skramaja has joined #oooq		05:25
*** marios\|ruck has joined #oooq		06:05
*** raukadah is now known as chandankumar		06:44
*** jbadiapa has joined #oooq		06:45
*** saneax has quit IRC		06:49
*** saneax has joined #oooq		06:49
*** jfrancoa has joined #oooq		06:51
*** openstack has joined #oooq		07:18
*** ChanServ sets mode: +o openstack		07:18
chandankumar	dpawlik, checking	07:20
chandankumar	dpawlik, it is not related to image server	07:21
chandankumar	dpawlik, check the https://logserver.rdoproject.org/99/708299/3/openstack-check/tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001/8fd71c1/logs/undercloud/var/log/containers/ironic/ironic-conductor.log.txt.gz	07:22
dpawlik	chandankumar, ack	07:23
*** openstack has joined #oooq		07:42
*** ChanServ sets mode: +o openstack		07:42
*** dpawlik has joined #oooq		07:58
*** tesseract has joined #oooq		08:06
*** jmasud has quit IRC		08:13
*** jmasud has joined #oooq		08:14
*** ccamacho has joined #oooq		08:15
*** amoralej\|off is now known as amoralej		08:24
*** Tengu has quit IRC		08:36
*** tosky has joined #oooq		08:37
*** Tengu has joined #oooq		08:38
amoralej	chandankumar, good news, both rhel8 jobs passed in https://review.rdoproject.org/r/#/c/24835/	08:46
amoralej	chandankumar, i'll remove DNM in https://review.opendev.org/#/c/708430/	08:48
amoralej	i'm not sure if we will break rhel8 jobs if we merge that and not https://review.opendev.org/#/c/706527/2	08:49
amoralej	mmm	08:49
amoralej	i'll do a new PS without that	08:49
amoralej	to check if we can merge them independently	08:50
zbr	marios\|ruck: https://review.opendev.org/#/c/703781/ please, is blocking molecule upgrade.	08:50
zbr	and it does not change any production code.	08:50
*** tosky has quit IRC		08:51
*** tosky has joined #oooq		08:51
*** jpena\|off is now known as jpena		08:52
marios\|ruck	zbr: ack will check	08:54
*** bogdando has joined #oooq		08:57
*** holser has joined #oooq		09:00
*** holser has quit IRC		09:00
*** holser has joined #oooq		09:01
chandankumar	amoralej, want to go over fs01 ?	09:03
chandankumar	and test there	09:03
chandankumar	amoralej, I mean standalone job is minimal and may be trying with fs01 will give more constructive result?	09:04
amoralej	chandankumar, it failed without https://review.opendev.org/#/c/706527/2/config/release/tripleo-ci/RedHat-8/promotion-testing-hash-master.yml	09:10
amoralej	so we need both changes together	09:11
amoralej	chandankumar, so, I'd say, restore the depends-on with the fs that you think is better to test	09:11
*** panda has quit IRC		09:11
amoralej	if it goes fine, i'll remove the dnm in https://review.opendev.org/#/c/708430/	09:12
amoralej	when that is merged (currently gentoo job is blocking the gate)	09:12
chandankumar	amoralej, src	09:12
chandankumar	sure	09:12
*** panda has joined #oooq		09:12
amoralej	i can change the link https://trunk.rdoproject.org/rhel8-master/deps/latest/ to point to https://trunk.rdoproject.org/centos8-master/deps/latest/	09:13
amoralej	so that we don't need https://review.opendev.org/#/c/706527/	09:14
amoralej	wdyt?	09:14
chandankumar	soniya29, https://review.opendev.org/#/c/700514/	09:17
chandankumar	amoralej, yes sounds good to me	09:17
amoralej	ok, will you ammend https://review.rdoproject.org/r/#/c/24835/ to change the fs001?	09:19
chandankumar	amoralej, yes will do that	09:20
amoralej	ack	09:21
soniya29	ykarel, please have a look on this review when you have time:- https://review.rdoproject.org/r/#/c/25034/	09:25
chandankumar	amoralej, does the above change https://trunk.rdoproject.org/centos8-master/deps/latest/ is permanent, if yes, it is going to affect rhel8 jobs?	09:34
amoralej	you mean https://review.opendev.org/#/c/708430/ ?	09:35
chandankumar	amoralej, nope, it is about https://review.opendev.org/#/c/706527/2/config/release/tripleo-ci/RedHat-8/promotion-testing-hash-master.yml	09:35
*** holser has quit IRC		09:36
chandankumar	sorry got it	09:36
chandankumar	ignore that	09:36
amoralej	what i was proposing is to fix it in repos server side	09:36
amoralej	instead of jobs config	09:36
*** dtantsur\|afk is now known as dtantsur		09:38
*** holser has joined #oooq		09:39
*** derekh has joined #oooq		09:43
*** ykarel\|lunch is now known as ykarel		09:47
*** holser__ has joined #oooq		09:50
*** holser has quit IRC		09:52
*** holser has joined #oooq		09:53
chandankumar	marios\|ruck, Hello	09:54
marios\|ruck	o/ chandankumar	09:54
chandankumar	marios\|ruck, Did we saw any issue related to downloading overcloud images from new image server?	09:54
chandankumar	on any multinode jobs	09:54
marios\|ruck	chandankumar: not afaik or haven't seen somethign yet anyway	09:55
chandankumar	marios\|ruck, cool, thanks :-)	09:55
chandankumar	dpawlik, I think we can reply back to ML about successful image server migration	09:55
*** holser__ has quit IRC		09:55
dpawlik	chandankumar, it is still two steps required to finish ;) https://review.rdoproject.org/r/#/c/25001/ and https://softwarefactory-project.io/r/#/c/17565/ . In theory it is "safe to merge", but I need to talk with nhicher, tristanC and jpena. If its done, I will send an email :)	09:57
chandankumar	dpawlik, ah, thanks :-)	09:59
sshnaidm	zbr, please remove +w from https://review.opendev.org/#/c/703781/	10:25
zbr	why?	10:25
sshnaidm	zbr, I rebase it for a reason and you don't have 2 +2 there	10:25
sshnaidm	zbr, because I'd like to see results with newer code	10:25
zbr	sshnaidm: interesting, that exactly what you did on https://review.opendev.org/#/c/706797/ at 20:06, added +W on a change that had only one +2.	10:31
zbr	i raised the issue of sova scenario being faulty a month ago, but I seen no fixed proposed.	10:34
zbr	many issues, no fix: use of delegated when it could have being container, due to this it was tested only on ubuntu, when in production is we run on other platforms.	10:35
zbr	use of "pip install --user" on localhost, mainly trying to alter user systems, and failing in some cases.	10:36
zbr	sova module needs pyyaml, but the role does not install it, i said that this must be part of the role, not put inside prepare.yaml	10:36
*** derekh has quit IRC		10:37
zbr	that requirement was never documented in README, but hidden in prepare.yml just to make the test pass in certain conditions.	10:37
sshnaidm	zbr, thanks for your effort	10:39
marios\|ruck	chandankumar: seen that before? ImportError: No module named parse during os_tempest : List tempest tests] * https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-scenario003-standalone-master/3140208/job-output.txt	10:58
chandankumar	marios\|ruck, nope	10:58
chandankumar	marios\|ruck, breanch?	10:58
chandankumar	branch	10:59
marios\|ruck	chandankumar: master	10:59
chandankumar	marios\|ruck, looking into that	10:59
marios\|ruck	chandankumar: only seen once so far so not sure if its a thing yet, noted on etherpad for now anyway	10:59
marios\|ruck	chandankumar: thanks	10:59
chandankumar	marios\|ruck, ImportError: No module named parse	11:00
marios\|ruck	chandankumar: yes	11:00
chandankumar	marios\|ruck, mistral mistral tempest tests broke it	11:00
chandankumar	time ot pin it	11:00
marios\|ruck	chandankumar: sigh k filing a bug then	11:00
chandankumar	marios\|ruck, open a bug I will fix that	11:01
chandankumar	not an issue	11:01
chandankumar	give me some time	11:01
marios\|ruck	chandankumar: do you know what broke it?	11:01
marios\|ruck	13:00 < chandankumar> marios\|ruck, mistral mistral tempest tests broke it	11:01
marios\|ruck	chandankumar: i mean do you have pointer/suspect a patch yet	11:01
chandankumar	marios\|ruck, https://github.com/openstack/mistral-tempest-plugin/commit/3fca72a6589fffc34962aeef6416f52bd860e602	11:01
marios\|ruck	chandankumar: ah i see more py2 ...	11:02
marios\|ruck	thanks chandankumar	11:02
chandankumar	more to come	11:02
marios\|ruck	chandankumar: https://bugs.launchpad.net/tripleo/+bug/1863883	11:06
openstack	Launchpad bug 1863883 in tripleo "periodic scenario3 standalone failing in 'list tempest tests' (py2 related) No module named parse" [Critical,Triaged]	11:06
*** d0ugal has quit IRC		11:17
*** d0ugal has joined #oooq		11:23
* marios\|ruck food biab		11:40
*** sshnaidm is now known as sshnaidm\|afk		11:47
weshay\|ruck	panda++	12:09
weshay\|ruck	panda, thank you https://trunk.rdoproject.org/centos8-master/	12:09
marios\|ruck	o/ weshay\|ruck	12:11
*** jmasud has quit IRC		12:11
marios\|ruck	weshay\|ruck: image deletion cost us promotion again (train this time)	12:13
weshay\|ruck	marios\|ruck, show me	12:13
marios\|ruck	weshay\|ruck: https://etherpad.openstack.org/p/ruckroversprint22	12:13
marios\|ruck	weshay\|ruck: train trying & failing promotion missing image :/ * http://38.145.34.55/centos7_train.log 2020-02-19 06:07:56,344 1022 ERROR promoter ERROR: The promoted hash was not found: https://images.rdoproject.org/centos7/train/rdo_trunk/4b32d316befe0919fd98a147d84086bc0907677a_046903a2	12:13
*** jmasud has joined #oooq		12:14
marios\|ruck	weshay\|ruck: i got a green run there https://review.rdoproject.org/r/#/c/25032/ with v1 but tripleo-ci-testing changed as it was running... so trying again with v2 explicit hash	12:14
weshay\|ruck	what line?	12:14
marios\|ruck	weshay\|ruck: 32	12:14
weshay\|ruck	let's sync	12:15
weshay\|ruck	that's not matching	12:15
marios\|ruck	weshay\|ruck: sure (what not matching?)	12:16
weshay\|ruck	https://meet.google.com/cfe-qxyz-tas?authuser=1	12:16
marios\|ruck	weshay\|ruck: joining	12:16
weshay\|ruck	chandankumar, hold to join this call	12:16
*** rfolco has joined #oooq		12:18
weshay\|ruck	chandankumar, join now please	12:19
marios\|ruck	http://38.145.34.55/centos7_train.log	12:20
marios\|ruck	2020-02-19 10:16:19,901 28012 ERROR promoter ERROR: The promoted hash was not found: https://images.rdoproject.org/centos7/train/rdo_trunk/4b32d316befe0919fd98a147d84086bc0907677a_046903a2	12:21
chandankumar	weshay\|ruck,4b32d316befe0919fd98a147d84086bc0907677a_046903a2	12:22
chandankumar	weshay\|ruck, https://review.rdoproject.org/r/#/c/25001/ and https://softwarefactory-project.io/r/#/c/17565/	12:24
chandankumar	weshay\|ruck, https://review.rdoproject.org/r/#/c/24926/	12:25
*** derekh has joined #oooq		12:25
chandankumar	weshay\|ruck, https://review.rdoproject.org/r/#/c/24926/4/roles/rdo-infra/image-server/templates/purge-images.sh.j2	12:25
*** skramaja has quit IRC		12:30
chandankumar	marios\|ruck, https://trunk.rdoproject.org/centos7-train/4b/32/4b32d316befe0919fd98a147d84086bc0907677a_046903a2/ it is from yesterday	12:32
*** bogdando has quit IRC		12:33
*** jpena is now known as jpena\|lunch		12:34
*** ykarel is now known as ykarel\|afk		12:35
weshay\|ruck	https://logserver.rdoproject.org/64/708264/8/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch/a73049d/job-output.txt	12:35
*** udesale_ has joined #oooq		12:49
*** rlandy has joined #oooq		12:50
*** b3nt_pin has joined #oooq		12:51
*** udesale has quit IRC		12:52
*** jfrancoa has quit IRC		12:52
*** bogdando has joined #oooq		12:54
*** Goneri has joined #oooq		12:54
*** ysandeep is now known as ysandeep\|away		12:57
chandankumar	weshay\|ruck, marios\|ruck dpawlik may be we forgot about syncing c7 images from image server, as I talked only about rhel images	12:58
weshay\|ruck	marios\|ruck, you should join #rhos-infrared	13:00
*** b3nt_pin is now known as beagles		13:01
rlandy	chandankumar: panda: sync meeting	13:01
marios\|ruck	weshay\|ruck: ack	13:02
marios\|ruck	weshay\|ruck: chandankumar: this isn't going to work https://review.rdoproject.org/r/#/c/25032/2/.zuul.yaml we can't do that with dlrn_hash_tag it ignored it and uploaded with the current tripleo-ci-testing	13:03
chandankumar	rlandy, rfolco https://review.rdoproject.org/r/25036	13:06
*** jfrancoa has joined #oooq		13:07
dpawlik	chandankumar, whole iamages were synced from old server. Probably it was removed by the script (but it should write that into the log	13:12
weshay\|ruck	panda, we check the previous 5 tripleo-ci-testing hashes to promote? context is I'm adjusting the image retention	13:13
panda	weshay\|ruck: 10	13:14
weshay\|ruck	so for older branches like queens.. that's quite a long time	13:14
panda	weshay\|ruck: rlandy rfolco here's the list of components I promoted https://review.rdoproject.org/paste/show/217/	13:14
weshay\|ruck	if we adjust the rention to 7 or 10 days panda does that sound right?	13:14
panda	weshay\|ruck: yep	13:15
weshay\|ruck	dpawlik, can we s/3/10 for retention please	13:15
weshay\|ruck	marios\|ruck, ^	13:15
marios\|ruck	weshay\|ruck: ack	13:15
marios\|ruck	weshay\|ruck: side issue... we don't have th eability to specify specific hash with testproject ... see https://github.com/rdo-infra/review.rdoproject.org-config/commit/e657048b5fb742034b7aefb1c5a558a24d930f90#diff-6d137e905483bbad00edc4994712b8ba	13:15
panda	weshay\|ruck: the problem I see it's ok for component, they promote less, we could have a bit more on the others images.	13:16
dpawlik	weshay\|ruck, ok	13:16
marios\|ruck	weshay\|ruck: i.e. that will not work https://review.rdoproject.org/r/#/c/25032/2/.zuul.yaml	13:16
weshay\|ruck	marios\|ruck, more details please	13:17
marios\|ruck	weshay\|ruck: we initially merged that https://review.rdoproject.org/r/#/c/23150 exactly so we can use the actual used hash e.g. when running with testproject	13:18
marios\|ruck	weshay\|ruck: however, there was then a bug https://bugs.launchpad.net/tripleo/+bug/1849284 and during discussion there (and in proposed fix https://review.rdoproject.org/r/#/c/23347/2/playbooks/tripleo-ci-periodic-base/post.yaml ) it was decided to revert https://review.rdoproject.org/r/#/c/23150 with	13:19
openstack	Launchpad bug 1849284 in tripleo "rdo periodic centos 7 featureset 050 upgrades-master fails post task "Error: invalid path 'commits.0.commit_hash', can't query subvalue 'commits' of a leaf"" [Undecided,Fix released]	13:19
marios\|ruck	https://github.com/rdo-infra/review.rdoproject.org-config/commit/e657048b5fb742034b7aefb1c5a558a24d930f90#diff-6d137e905483bbad00edc4994712b8ba	13:19
marios\|ruck	weshay\|ruck: so now it always uses tripleo-ci-testing	13:19
dpawlik	weshay\|ruck, pls check in free time https://review.rdoproject.org/r/#/c/25043/ and https://review.rdoproject.org/r/#/c/24926/	13:19
*** ratailor has quit IRC		13:20
rlandy	https://hackmd.io/4K_NU1PxSDiTYGecnAZNCA	13:30
rlandy	weshay\|ruck: chandankumar: arxcruz: rfolco: zbr: panda: ^^	13:30
weshay\|ruck	panda, thank you	13:30
*** amoralej is now known as amoralej\|lunch		13:31
chandankumar	amoralej\|lunch, jpena\|lunch first this https://review.rdoproject.org/r/25037 and then this https://review.rdoproject.org/r/25038 it will clear master c7 promotion	13:32
* chandankumar headed home		13:32
weshay\|ruck	arxcruz, http://dashboard-ci.tripleo.org/d/cockpit/cockpit?orgId=1&fullscreen&panelId=345	13:35
weshay\|ruck	arxcruz, http://dashboard-ci.tripleo.org/d/cockpit/cockpit?orgId=1&fullscreen&panelId=322	13:36
*** jpena\|lunch is now known as jpena		13:36
rfolco	panda, how do you usually unset an ansible var from one molecule test to the next ?	13:44
rlandy	panda: rfolco: can you join https://meet.google.com/jms-ikvx-kgz?pli=1&authuser=0 re: registry login failure	13:46
*** Goneri has quit IRC		13:47
rfolco	rlandy, ok gimme one min to get my headset	13:47
panda	rlandy: ok gimme one minute to go get my head.	13:48
rfolco	all set	13:49
*** ykarel\|afk is now known as ykarel		13:49
panda	rfolco: not unset specifically, usually reset, but if there's possibility of pollution, yes, unset it	13:50
*** amoralej\|lunch is now known as amoralej		14:00
chandankumar	marios\|ruck, tempest is fixed now, please retest in another 10 mins	14:04
marios\|ruck	chandankumar: thanks	14:04
rlandy	chandankumar: would podman be available by default for install on centos 8?	14:06
rlandy	https://softwarefactory-project.io/cgit/config/tree/nodepool/elements/rhel-8-packages-needs/package-installs.yaml	14:07
rlandy	rhel-8 has it	14:07
chandankumar	rlandy, nope, i think	14:07
rlandy	chandankumar: we need to get that on the centos 8 nodepool nodes	14:08
chandankumar	jpena, ^^	14:08
rlandy	otherwise container login is dead in the water (thanks to panda for that pointer)	14:09
jpena	chandankumar: shouldn't those packages be installed by the job itself? We're using the same elements as opendev.org does for centos-8, and those do not include podman	14:09
chandankumar	rlandy, let me send a patch	14:11
rlandy	https://github.com/rdo-infra/review.rdoproject.org-config/blob/5fb19d8a0f4c8dea4574cf5394de537912579ed6/playbooks/tripleo-rdo-base/container-login.yaml	14:11
rlandy	chandankumar: ^^ that expects it to be there	14:11
rlandy	per the rhel case	14:12
rlandy	we're not at the install-deps yet	14:12
*** jaosorior has joined #oooq		14:16
chandankumar	rlandy, panda https://review.rdoproject.org/r/25046	14:16
chandankumar	rlandy, reusing this https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/roles/tripleo_podman/tasks/tripleo_podman_install.yml	14:17
marios\|ruck	weshay\|ruck: i think might be related to contextlib2 version commented at https://review.opendev.org/#/c/708264/8	14:17
rlandy	chandankumar: looking	14:18
rlandy	chandankumar: let's try it	14:19
rlandy	can always revert :)	14:19
chandankumar	yes	14:19
* rlandy likes centos 8 only patches - less likely to explode		14:20
*** Goneri has joined #oooq		14:25
chandankumar	weshay\|ruck, rlandy I have results from image build https://logserver.rdoproject.org/97/24997/5/check/tripleo-centos-8-buildimage-overcloud-full/bbcbb26/overcloud-full.log	14:34
chandankumar	ModuleNotFoundError: No module named 'yaml'	14:34
rlandy	chandankumar: panda: good news and bad news ... install doesn't fail - but login still does	14:34
rlandy	on with debug	14:35
rlandy	weird	14:35
rlandy	same on ipa?	14:36
chandankumar	rlandy, ipa?	14:37
rlandy	periodic-tripleo-rhel-8-buildimage-ironic-python-agent-master	14:38
*** jaosorior has quit IRC		14:38
rlandy	chandankumar: ^^	14:38
chandankumar	rlandy, yup both places	14:39
rlandy	chandankumar: probably missing another install	14:39
rlandy	checking rhel packages	14:39
chandankumar	let me check tripleo image elemnts	14:40
rlandy	https://softwarefactory-project.io/cgit/config/tree/nodepool/elements/rdo-base/package-installs.yaml	14:40
chandankumar	rlandy, https://github.com/openstack/tripleo-image-elements#instructions	14:42
chandankumar	from readme it talks about pyyaml	14:42
rlandy	pip install dib-utils pyyaml	14:42
chandankumar	let me add a pre task	14:43
rlandy	chandankumar: this is crazy	14:43
rlandy	we are adding missing packages everywhere	14:43
rlandy	we should have a single place	14:43
rlandy	a centos08 pre	14:44
rlandy	to do all the missing installs?	14:44
arxcruz	/whois omaciel	14:44
chandankumar	rlandy, https://github.com/openstack/diskimage-builder/search?q=pyyaml&unscoped_q=pyyaml	14:48
*** TrevorV has joined #oooq		14:48
chandankumar	rlandy, may be I need to add proper elements	14:48
rlandy	looking back at what gets pulled in	14:49
rlandy	base rhel baremetal dhcp-all-interfaces openvswitch overcloud-agent overcloud-base overcloud-controller overcloud-compute overcloud-ceph-storage puppet-modules stable-interface-names grub2 element-manifest dynamic-login iptables enable-packages-install override-pip-and-virtualenv dracut-regenerate remove-machine-id remove-resolvconf openssh disable-nouveau selinux-permissive	14:51
rlandy	https://github.com/openstack/diskimage-builder/blob/master/diskimage_builder/elements/svc-map/package-installs.yaml	14:52
rlandy	chandankumar: ^^ include that?	14:52
*** sshnaidm\|afk is now known as sshnaidm		14:52
rlandy	there is a lot of rhel switching	14:53
rlandy	we need to make that rhel and centos switching	14:53
chandankumar	rlandy, https://github.com/openstack/diskimage-builder/blob/master/diskimage_builder/elements/svc-map/pkg-map	14:53
rlandy	python3-pyyaml should install	14:54
chandankumar	export DISTRO_NAME=rhel	14:55
chandankumar	i got the issue	14:55
*** irclogbot_2 has quit IRC		14:56
rlandy	export NODE_DIST="centos"	14:56
rlandy	ugh registry login	14:58
*** irclogbot_1 has joined #oooq		15:01
marios\|ruck	weshay\|ruck: o/ you joining cix call?	15:31
*** marios\|ruck is now known as marios\|ruck\|call		15:37
*** Trevor_V has joined #oooq		15:40
chandankumar	rlandy, I will try to poke image build tomorrow locally	15:41
rlandy	chandankumar: ack - I think I found the podman issue	15:41
*** Trevor__V has joined #oooq		15:41
rlandy	https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/roles/tripleo_podman/vars	15:41
rlandy	chandankumar: have a good night - will leave you notes	15:42
*** TrevorV has quit IRC		15:43
*** Trevor_V has quit IRC		15:44
migi	rlandy: hey do you have 2min? I have question about osp17	15:51
migi	rlandy: in short there is no passed_phase1 yet in the http://download.eng.bos.redhat.com/rcm-guest/puddles/OpenStack/17.0-RHEL-8/	15:51
rlandy	migi: sure	15:52
migi	rlandy: this is generated repo file: https://sf.hosted.upshift.rdu2.redhat.com/logs/41/192341/3/check/tripleo-ci-rhel-8-standalone-rhos-17/c50caf8/logs/undercloud/etc/yum.repos.d/rhos-release-17-trunk.repo	15:52
migi	by the repo-setup from quickstart	15:52
migi	rlandy: and I thought the rhos_release_args are enough for that: https://code.engineering.redhat.com/gerrit/#/c/192341/3/zuul.d/standalone-jobs.yaml	15:52
migi	rlandy: where else I need to look for proper setup of s/passed_phase1/latest-RHOS_TRUNK-17-RHEL-8	15:53
rlandy	migi: there is no passed_phase_1 ... then we need to pick another build	15:53
migi	rlandy: cause I see it's in at least few places, but maybe you know by heart	15:53
rlandy	getting	15:54
migi	rlandy: yeah this is what I've been trying to do, but both rhos_release_args and current_build are failing to be used	15:54
rlandy	http://git.app.eng.bos.redhat.com/git/tripleo-environments.git/tree/config/release/rhos-15.yml	15:54
*** marios\|ruck\|call is now known as marios\|ruck		15:55
rlandy	migi: ^^ try that in 17	15:55
rlandy	-P 17-trunk	15:55
rlandy	edit	15:55
migi	it's there	15:55
rlandy	http://git.app.eng.bos.redhat.com/git/tripleo-environments.git/tree/config/release/rhos-17.yml#n5	15:55
migi	rlandy: http://git.app.eng.bos.redhat.com/git/tripleo-environments.git/tree/config/release/rhos-17.yml	15:55
rlandy	sec - will put up patch	15:55
migi	rlandy: and there is current_build	15:55
migi	rlandy: which should be taken from https://code.engineering.redhat.com/gerrit/#/c/192341/2/zuul.d/standalone-jobs.yaml	15:56
migi	rlandy: line 178, this didn't work so added the second version, still wrong and I don't want to hardcode it	15:56
rlandy	migi: hold on - let me experiment	15:58
migi	rlandy: ok, as you said it's possible to modify env file but that's a hack imo	15:59
rlandy	migi: go ahead change release file	15:59
rlandy	no big deal	15:59
weshay\|ruck	sshnaidm, need to chat you avail?	15:59
weshay\|ruck	re: vex	15:59
migi	rlandy: ok	15:59
rlandy	migi; not with current	16:00
rlandy	with trubk	16:00
rlandy	trunk	16:00
rlandy	let's start there	16:00
sshnaidm	5 min	16:01
sshnaidm	weshay\|ruck, ^	16:01
rlandy	migi: you're putting in the review or should I?	16:03
rlandy	you don't have to merge	16:03
rlandy	just depends-on	16:03
migi	rlandy: https://code.engineering.redhat.com/gerrit/192386	16:03
weshay\|ruck	sshnaidm, k	16:04
rlandy	migi: yep - just add that as a depends on and let's see what happend	16:04
migi	yep done	16:05
migi	rlandy: https://code.engineering.redhat.com/gerrit/#/c/192341/	16:05
rlandy	migi: Depends-On	16:05
*** bogdando has quit IRC		16:06
migi	rlandy: I think o will work, but changed	16:06
rlandy	idk	16:06
rlandy	we'll see	16:06
rlandy	2020-02-19 16:06:46.563707 \| TASK [Assert role has worked in (CentOS\|RedHat) 8]	16:07
rlandy	2020-02-19 16:06:46.632112 \| primary \| skipping: Conditional result was False	16:07
rlandy	yes yes!!!	16:07
rlandy	hello centos 8 container reg login	16:07
weshay\|ruck	nice	16:08
weshay\|ruck	rlandy, so did compute promote upstream? did we figure that bit out yet?	16:08
rlandy	weshay\|ruck: no centos vars in https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/roles/tripleo_podman/vars	16:08
rlandy	weshay\|ruck: promote from what to what?	16:08
weshay\|ruck	rlandy, will catch you later.. sorry to interrupt	16:09
migi	rlandy: https://opendev.org/zuul/zuul/src/branch/master/zuul/driver/gerrit/gerritconnection.py#L474	16:09
migi	rlandy: IGNORECASE	16:09
rlandy	migi: ok - I'm sorry	16:09
rlandy	you're right	16:09
migi	rlandy: that's ok better off having O to be consistent - corrected	16:09
rlandy	now we're back to ERROR! the role 'get_hash' was not found in	16:10
rlandy	fixing	16:11
*** TrevorV has joined #oooq		16:13
weshay\|ruck	marios\|ruck, panda FYI.. disabling rocky on the promoter... so it can run in vexhost	16:15
weshay\|ruck	will add to the etherpad	16:15
*** Trevor__V has quit IRC		16:15
*** ysandeep\|away is now known as ysandeep		16:17
*** jfrancoa has quit IRC		16:17
weshay\|ruck	marios\|ruck, so you thought.. https://review.rdoproject.org/r/#/c/25032/ is NOT going to work right?	16:18
marios\|ruck	weshay\|ruck: ack on rocky	16:18
marios\|ruck	weshay\|ruck: yeah... i'm trying v3 with both _hash and _hash_tag but i think it wont work because we originally merged that https://review.rdoproject.org/r/#/c/23150/ so we could report with the right hash, but then we reverted it with https://review.opendev.org/#/c/570207/	16:19
marios\|ruck	sorry, reverted it with https://review.rdoproject.org/r/#/c/23350/	16:20
sshnaidm	rlandy, weshay\|ruck I'm checking another patch on bm	16:20
rlandy	sure	16:20
weshay\|ruck	sshnaidm, aye.. just ping rlandy usually just in case	16:20
weshay\|ruck	bm rulz	16:20
weshay\|ruck	sshnaidm, ~/ci-config/ci-scripts/dlrnapi_promoter/dlrn-promoter.sh where the branches are defined...	16:23
weshay\|ruck	is in src control.. I've made that file immutable in the production promoter.. but it's in source control so.. may be an issue	16:23
weshay\|ruck	rocky is disabled	16:23
*** ysandeep is now known as ysandeep\|gone		16:24
sshnaidm	weshay\|ruck, ack	16:24
rlandy	sshnaidm: weshay\|ruck: just fyi - master bm stalls on tempest ... hadn't had a chance to fix it yet	16:24
sshnaidm	rlandy, ok	16:24
rlandy	will do after centos 8 is alive and kicking	16:24
weshay\|ruck	marios\|ruck, context on the sync is knowledge transfer.. show and tell.. so you are welcome to stay for as much or as little as you like	16:25
marios\|ruck	weshay\|ruck: which sync?	16:26
marios\|ruck	weshay\|ruck: you mean tomorrow ruck\|rover sync?	16:27
weshay\|ruck	ya..	16:27
weshay\|ruck	marios\|ruck, back to train	16:27
marios\|ruck	weshay\|ruck: k	16:27
weshay\|ruck	so the testproject job is not picking up dlrn_hash_tag and dlrn_hash as it's set in the test project job	16:27
weshay\|ruck	test project has	16:27
*** TrevorV has quit IRC		16:27
weshay\|ruck	force_periodic: true	16:27
weshay\|ruck	dlrn_hash_tag: 4b32d316befe0919fd98a147d84086bc0907677a_046903a2	16:27
weshay\|ruck	dlrn_hash: 4b32d316befe0919fd98a147d84086bc0907677a_046903a2	16:27
weshay\|ruck	the job has	16:27
rlandy	weshay\|ruck: rfolco: fyi ... https://review.rdoproject.org/r/#/c/25056/	16:28
weshay\|ruck	+ DISTRO_HASH=9ee5e8de3d2d8c89809ff8b1536f91321f81a099	16:28
weshay\|ruck	2020-02-19 15:02:37.900733 \| primary \| + FULL_HASH=82cd572931b95f0dc30a948fb8a5fea2b35627e3_9ee5e8de	16:28
weshay\|ruck	2020-02-19 15:02:37.900796 \| primary \| + cat	16:28
marios\|ruck	weshay\|ruck: yeah it is ignoring those	16:28
marios\|ruck	weshay\|ruck: i was poking at * https://github.com/openstack/tripleo-quickstart/blob/master/roles/repo-setup/tasks/get-dlrn-hash.yml	16:28
weshay\|ruck	maybe the new role doesn't allow an override	16:28
weshay\|ruck	rfolco, ^	16:28
weshay\|ruck	not sure if it's using the new role.. tbh	16:28
weshay\|ruck	2020-02-19 15:02:25.815960 \| PRE-RUN START: [trusted : review.rdoproject.org/config/playbooks/tripleo-ci-periodic-base/pre.yaml@master]	16:29
rfolco	weshay\|ruck, no that get-dlrn-hash isn't using any new code	16:29
weshay\|ruck	aye.. thanks	16:29
weshay\|ruck	rfolco, probably want to ensure the new role has an override option if you haven't already	16:30
rfolco	weshay\|ruck, ok	16:30
weshay\|ruck	marios\|ruck,	16:31
weshay\|ruck	vars:	16:31
weshay\|ruck	force_periodic: true	16:31
weshay\|ruck	featureset_override:	16:31
weshay\|ruck	dlrn_hash_tag:	16:31
weshay\|ruck	try it w/ featureset_override: -> dlrn_hash_tag https://review.rdoproject.org/r/#/c/24397/3/.zuul.yaml	16:31
marios\|ruck	weshay\|ruck: ah !	16:31
marios\|ruck	damn ...	16:31
*** jfrancoa has joined #oooq		16:32
weshay\|ruck	rlandy, +2	16:32
marios\|ruck	weshay\|ruck: i guess that means i have to document that :/ :)	16:32
*** TrevorV has joined #oooq		16:33
marios\|ruck	weshay\|ruck: https://review.rdoproject.org/r/#/c/25032/4/.zuul.yaml lets see now then	16:33
weshay\|ruck	marios\|ruck, I knew you were going to say that	16:33
marios\|ruck	weshay\|ruck: ;)	16:33
marios\|ruck	weshay\|ruck: new bug https://bugs.launchpad.net/tripleo/+bug/1863920 seems to be not consistent but seen twice now so...	16:39
openstack	Launchpad bug 1863920 in tripleo "(intermittent?) cli-enable-ssh-admin.yaml fails during the overcloud deploy" [Critical,Triaged]	16:39
weshay\|ruck	marios\|ruck, I think that's also in osp	16:41
zbr	weshay\|ruck: ok to reconsider https://review.rdoproject.org/r/#/c/24972/ ?	16:43
weshay\|ruck	maybe not	16:43
weshay\|ruck	zbr, oh knoze.. we move tox-molecule back to non-voting?	16:43
zbr	that one is nv, nothing to move.	16:44
zbr	the issue is not the job, is the test.	16:44
weshay\|ruck	zbr, I thought is was voting previously	16:44
weshay\|ruck	the failing test should now pass	16:44
weshay\|ruck	zbr, https://trunk.rdoproject.org/centos8-master/consistent/delorean.repo.md5	16:44
zbr	at some point in the past yep, but as soon we fix it, we make it voting again.	16:44
weshay\|ruck	is avail now	16:45
weshay\|ruck	zbr, ah this failed too	16:45
weshay\|ruck	TASK [Attempt to use docker command as normal user] ****************************	16:45
weshay\|ruck	what's up w/ that one?	16:45
weshay\|ruck	zbr, nrmind.. reading comments	16:46
weshay\|ruck	zbr, voted	16:46
zbr	panda will probably fix it	16:46
migi	rlandy: passed that part: https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/stream/9db88104b46240f5b9847077d3129e1f?logfile=console.log	16:46
weshay\|ruck	panda, have time fix or slammed? ^	16:47
migi	rlandy++	16:47
zbr	weshay\|ruck: there reason for which I am pushing with migration is that it needs molecule.yml changes and is a PITA not to be able to run the tool directly on each repo.	16:47
rlandy	migi: cool	16:47
rlandy	migi: go ahead and change whatever you need - I won;t touch 17 until centos 8 is sorted	16:48
migi	rlandy: I hope I am doing correct path - first get those standalone rhel8 jobs up and running and then add the component pipeline on top of them	16:48
rlandy	migi: yep - there are a LOT of changes in component pipeline atm	16:49
marios\|ruck	weshay\|ruck: going in few need something ?	16:49
migi	rlandy: cool	16:49
rlandy	we're changing stuff left and right upstream - that might impact downstream jobs	16:49
rlandy	honestly, I haven't even began to look at fixing that	16:49
rlandy	migi: but yeah - first get regular standalone going	16:50
rlandy	then we can talk component pipeline	16:50
migi	rlandy: ok	16:50
zbr	@oooq i need to you tell me which zuul-jobs roles you suspect to have problems w/ centos-8, so I can priorietize them.	16:53
weshay\|ruck	marios\|ruck, hrm.. master ovb centos .. think we have a problme	16:53
* weshay\|ruck checking		16:53
weshay\|ruck	marios\|ruck, hrm.. it's nhicher	16:54
weshay\|ruck	and vexx	16:54
marios\|ruck	weshay\|ruck: link ?	16:55
marios\|ruck	weshay\|ruck: oh 18:54 < weshay\|ruck> marios\|ruck, hrm.. it's nhicher	16:55
marios\|ruck	weshay\|ruck: you mean its a test job? ^	16:55
weshay\|ruck	http://dashboard-ci.tripleo.org/d/cockpit/cockpit?orgId=1&fullscreen&panelId=442	16:56
weshay\|ruck	https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master	16:56
weshay\|ruck	marios\|ruck, but.. the job is bloody red.. in rdo as well	16:56
marios\|ruck	weshay\|ruck: 2018?	16:56
marios\|ruck	18:56 < weshay\|ruck> https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master	16:56
*** udesale_ has quit IRC		16:56
weshay\|ruck	not sure what ur saying	16:57
marios\|ruck	https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master	16:57
marios\|ruck	2018-10-16T20:01:33	16:57
marios\|ruck	2018-10-16T16:08:58FAILURE	16:57
panda	weshay\|ruck: slammed, we moved all the fixes to jobs to the next sprint.	16:57
marios\|ruck	2018-10-16T13:07:31FAILURE	16:57
marios\|ruck	weshay\|ruck: ^	16:57
weshay\|ruck	oh wtf	16:57
weshay\|ruck	how is that happening?	16:58
marios\|ruck	weshay\|ruck: i don't see tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master in https://review.rdoproject.org/zuul/jobs	16:58
weshay\|ruck	http://dashboard-ci.tripleo.org/d/cockpit/cockpit?orgId=1&fullscreen&panelId=426	16:59
panda	anyone up for https://review.rdoproject.org/r/24678 ? this will need patient reviews.	16:59
marios\|ruck	weshay\|ruck: https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001	16:59
weshay\|ruck	maybe a more sane place to look	16:59
marios\|ruck	weshay\|ruck: its with -master on the name ^	16:59
marios\|ruck	weshay\|ruck: its that https://bugs.launchpad.net/tripleo/+bug/1863920 e.g. there https://logserver.rdoproject.org/43/704443/8/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/7e53ed6/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz	17:00
openstack	Launchpad bug 1863920 in tripleo "(intermittent?) cli-enable-ssh-admin.yaml fails during the overcloud deploy" [Critical,Triaged]	17:00
weshay\|ruck	ah k	17:01
weshay\|ruck	Data could not be sent to remote host "192.168.24.9". Make sure this host can be reached over ssh: Warning: Permanently added '192.168.24.9' (ECDSA) to the list of known hosts.	17:01
weshay\|ruck	2020-02-19 07:47:16 \| Permission denied (publickey,gssapi-keyex,gssapi-with-mic).	17:01
marios\|ruck	weshay\|ruck: yeah the new bug	17:01
marios\|ruck	weshay\|ruck: so perhaps not intermittent then	17:01
weshay\|ruck	rlandy, you have anther bm env?	17:01
marios\|ruck	weshay\|ruck: k i will dig into that some more tomorrow	17:01
weshay\|ruck	sshnaidm, you're not testing fs001 centos are you?	17:01
weshay\|ruck	marios\|ruck, https://code.engineering.redhat.com/gerrit/#/c/190672/	17:02
rlandy	weshay\|ruck: yeah - you can stop the stein jobs	17:02
rlandy	and take that env	17:02
rlandy	or take env D back from rhel	17:02
rlandy	wow - bm just got popular	17:02
weshay\|ruck	rlandy, it's ok.. sshnaidm is already testing it	17:02
marios\|ruck	weshay\|ruck: context? 19:02 < weshay\|ruck> marios\|ruck, https://code.engineering.redhat.com/gerrit/#/c/190672/	17:02
weshay\|ruck	marios\|ruck, it's not code	17:02
weshay\|ruck	it's infra	17:02
weshay\|ruck	marios\|ruck, https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/build/1b98bbdfdcdb4f0681e7d8fe3788dd13	17:02
weshay\|ruck	not sure what we want to do w/ it.. I'll add comments to the cix	17:03
marios\|ruck	weshay\|ruck: sorry i'm not sure what you're saying here...	17:03
weshay\|ruck	marios\|ruck, let me show you something before you leave	17:03
weshay\|ruck	https://meet.google.com/dno-qfiu-jmf?authuser=1	17:04
rfolco	weshay\|ruck, rlandy any problems if I trigger periodic-tripleo-centos-8-master-promote-promoted-components-to-tripleo-ci-testing in testproject ?	17:04
weshay\|ruck	will just take a minute marios\|ruck	17:04
rlandy	rfolco: should be ok - expect we don;t have the previous tag	17:05
rfolco	weshay\|ruck, rlandy I want to test the new get_hash for "integration" use case (aggregated hash)	17:05
*** ykarel is now known as ykarel\|away		17:06
rfolco	apparently the last periodic failed on missing md5, so next time it will work	17:06
rlandy	k	17:07
rlandy	component-ci-testing	17:09
rlandy	^^ that is what we are looking for	17:09
rlandy	rfolco: if I reran periodic-tripleo-centos-8-master-component-compute-promote-consistent-to-component-ci-testing - would the promote work?	17:10
rfolco	the last failure suggests http 404 on md5 file	17:10
rfolco	so yes I guess so	17:10
* rlandy reruns		17:10
*** dtantsur is now known as dtantsur\|afk		17:11
weshay\|ruck	sshnaidm, cockpit down	17:11
weshay\|ruck	maybe I'll restart	17:11
*** jmasud has quit IRC		17:12
*** marios\|ruck is now known as marios\|out		17:13
*** tosky has quit IRC		17:16
*** marios\|out has quit IRC		17:16
weshay\|ruck	zbr, do you mind trying to patch / fix the non-voting molecule job?	17:26
weshay\|ruck	we need all molecule voting :)	17:26
zbr	i can try tomorrow with panda	17:27
zbr	we are 100% on the same page regarding "nv" part.	17:27
weshay\|ruck	rlandy, sshnaidm re: env E.. if the job fails.. please let us look at it to debug https://bugs.launchpad.net/tripleo/+bug/1863920	17:31
openstack	Launchpad bug 1863920 in tripleo "(intermittent?) cli-enable-ssh-admin.yaml fails during the overcloud deploy" [Critical,Triaged]	17:31
weshay\|ruck	thanks	17:31
rlandy	k	17:31
sshnaidm	weshay\|ruck, Out of memory: Kill process 7148 (influxd) score 885 or sacrifice child	17:31
rlandy	weshay\|ruck: are we 1-on-1'ing?	17:31
rlandy	or too busy	17:31
weshay\|ruck	sshnaidm, oh nose	17:32
weshay\|ruck	sshnaidm, it's back up	17:32
weshay\|ruck	but ugh	17:32
weshay\|ruck	rlandy, let's 1-1	17:32
sshnaidm	weshay\|ruck, yeah, it repaired itself..	17:32
weshay\|ruck	sshnaidm++	17:32
*** TrevorV has quit IRC		17:33
rlandy	weshay\|ruck: https://logserver.rdoproject.org/openstack-component-compute/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-8-master-component-compute-promote-consistent-to-component-ci-testing/7af802a/job-output.txt	17:47
weshay\|ruck	rlandy, https://trunk.rdoproject.org/centos7-master/	17:49
*** jmasud has joined #oooq		17:52
*** derekh has quit IRC		18:00
*** jpena is now known as jpena\|off		18:00
*** TrevorV has joined #oooq		18:01
*** jmasud has quit IRC		18:04
weshay\|ruck	rfolco, what's the status on the centos-8 container push?	18:11
rfolco	weshay\|ruck, don't know, chandankumar or rlandy maybe?	18:12
weshay\|ruck	really	18:12
weshay\|ruck	fak	18:12
weshay\|ruck	is chandankumar still here?	18:12
* weshay\|ruck looks		18:12
weshay\|ruck	rfolco, what are you working on?	18:13
rfolco	get_hash and promoted_components dlrn report	18:14
*** jmasud has joined #oooq		18:14
weshay\|ruck	k	18:14
rfolco	periodic-tripleo-centos-8-master-promote-promoted-components-to-component-ci-testing	18:15
rfolco	does not exist ?? whaaat?	18:15
rfolco	nm .... periodic-tripleo-centos-8-master-component-compute-promote-consistent-to-component-ci-testing	18:18
*** jmasud has quit IRC		18:18
*** holser has quit IRC		18:22
rfolco	weshay\|ruck, get_hash seems to be working for components --> https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-centos-8-master-component-compute-promote-consistent-to-component-ci-testing&job_name=periodic-tripleo-centos-8-master-component-security-promote-consistent-to-component-ci-testing	18:22
rfolco	now checking if it works for integration job	18:23
*** holser has joined #oooq		18:24
migi	anyone have seen issue with ansible-playbook-3 not found during standalone deployment ? asking because maybe it's already known issue	18:26
rfolco	migi, not exactly the same issue, but I had a similar problem w/ 2 ansible installs from system/pip	18:28
weshay\|ruck	migi, ya.. if ur getting files not found.. probably something failing in the setup.py install	18:31
migi	weshay\|ruck: ack, see it's coming from ./tripleo_common/image/image_uploader.py	18:32
migi	will investigate further	18:32
*** jbadiapa has quit IRC		18:32
weshay\|ruck	rlandy, rfolco oh ya.. already merged	18:33
weshay\|ruck	https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-master	18:33
weshay\|ruck	periodic-tripleo-centos-8-master-promote-promoted-components-to-tripleo-ci-testingopenstack/tripleo-cimasteropenstack-periodic-mastermaster792020-02-19T08:10:19FAILURE	18:33
weshay\|ruck	periodic-tripleo-centos-8-master-containers-build-pushopenstack/tripleo-cimasteropenstack-periodic-mastermasterSKIPPED	18:33
rlandy	what, what?	18:33
rfolco	weshay\|ruck, I found a bug in the job that promotes to tripleo-ci-testing	18:34
rfolco	2020-02-19 18:02:31.619093 \| primary \| + dlrnapi --url https://trunk.rdoproject.org/api-centos-master-uc --username review_rdoproject_org repo-promote --commit-hash --distro-hash --promote-name tripleo-ci-testing	18:34
rfolco	2020-02-19 18:02:31.779021 \| primary \| usage: dlrnapi repo-promote [-h] --commit-hash COMMIT_HASH --distro-hash	18:34
rfolco	2020-02-19 18:02:31.779086 \| primary \| DISTRO_HASH --promote-name PROMOTE_NAME	18:34
rfolco	2020-02-19 18:02:31.779095 \| primary \| dlrnapi repo-promote: error: argument --commit-hash: expected one argument	18:34
rfolco	we need to omit distro_hash	18:34
rfolco	if that is even possible in dlrn client, dunno	18:34
rlandy	Testing https://review.rdoproject.org/r/#/c/25063/	18:34
*** jmasud has joined #oooq		18:35
*** holser has quit IRC		18:36
weshay\|ruck	rfolco, s/https://trunk.rdoproject.org/api-centos-master-uc/https://trunk.rdoproject.org/api-centos8-master-uc	18:40
*** jmasud has quit IRC		18:40
chandankumar	hmm	18:41
weshay\|ruck	rfolco, then look at the unit tests for dlrn	18:41
weshay\|ruck	hey chandankumar	18:41
chandankumar	weshay\|ruck, Hello	18:41
weshay\|ruck	rlandy, rfolco https://review.rdoproject.org/r/#/c/25064/	18:48
rlandy	k	18:49
weshay\|ruck	rlandy, that may also be pending your patches	18:49
rlandy	hopefully that will push containers	18:49
weshay\|ruck	rlandy, because promote will fail	18:49
weshay\|ruck	because centos vs centos8	18:49
weshay\|ruck	but we have the job	18:49
weshay\|ruck	rfolco, did my comment make sense?	18:49
weshay\|ruck	sshnaidm, your bm failed https://sf.hosted.upshift.rdu2.redhat.com/logs/72/190672/7/check/periodic-tripleo-ci-centos-7-bm_envE-3ctlr_1comp-featureset001-master/c1d8da6/	18:51
weshay\|ruck	sshnaidm, see ur patch	18:51
weshay\|ruck	https://review.opendev.org/#/c/708717/2/tripleo_ansible/playbooks/cli-enable-ssh-admin.yaml	18:51
weshay\|ruck	sshnaidm, shall we try bm w/ your patch?	18:51
chandankumar	weshay\|ruck, any help needed?	18:51
weshay\|ruck	sshnaidm, or I can show you how to log into the box	18:51
weshay\|ruck	chandankumar, no but we should sync up as pm/ua soonish	18:52
weshay\|ruck	chandankumar, alot going on and we need our tc/ua's	18:52
chandankumar	weshay\|ruck, yup sure	18:53
sshnaidm	weshay\|ruck, I wanted to test import nodes and seems like it works: https://sf.hosted.upshift.rdu2.redhat.com/logs/72/190672/7/check/periodic-tripleo-ci-centos-7-bm_envE-3ctlr_1comp-featureset001-master/c1d8da6/logs/undercloud/home/zuul/overcloud_prep_images.log	18:53
rlandy	"repo_url": "https://trunk.rdoproject.org/centos7/a9/d3/a9d30b2f6dca34bc5278bb7eba024e40c17d0dd2_e3a3586e", - still doing that	18:53
chandankumar	weshay\|ruck, may be tomorrow after scrum. will work	18:53
weshay\|ruck	maybe.. maybe next year	18:53
weshay\|ruck	lolz	18:53
sshnaidm	weshay\|ruck, it's from https://review.opendev.org/#/c/704561/	18:53
chandankumar	okok	18:54
rlandy	ugh - idiot - fixing	18:54
weshay\|ruck	sshnaidm, k.. so next question do you still need envE, fine if you do	18:55
sshnaidm	weshay\|ruck, not atm, you can take it over	18:56
weshay\|ruck	sshnaidm, thanks	18:57
weshay\|ruck	rfolco, so.. dlrnapi --url https://trunk.rdoproject.org/api-centos-master-uc --username review_rdoproject_org repo-promote --commit-hash --distro-hash --promote-name tripleo-ci-testing	18:57
weshay\|ruck	2020-02-19 18:50:17.643220 \| primary \| usage: dlrnapi repo-promote [-h] --commit-hash COMMIT_HASH --distro-hash	18:57
weshay\|ruck	https://logserver.rdoproject.org/64/25064/2/check/periodic-tripleo-centos-8-master-promote-promoted-components-to-tripleo-ci-testing/f07f3b6/job-output.txt	18:57
weshay\|ruck	rfolco, rlandy is working on a fix there	18:57
rfolco	weshay\|ruck, ah ok good to know	18:57
rfolco	thanks rlandy	18:57
weshay\|ruck	rlandy, ping me with a patch and we'll try w/ testproject	18:57
rlandy	rfolco: https://review.rdoproject.org/r/#/c/25063	18:57
weshay\|ruck	rlandy, don't use the standalone deploy until we have containers	18:58
weshay\|ruck	it's a waste	18:58
rlandy	weshay\|ruck: ^^ patch and testproject runs promote-to-component-ci-testing	18:58
rlandy	but it still uses 7	18:58
rfolco	wait, old get-hash ? component jobs use the new one	18:59
rfolco	rlandy, weshay\|ruck what am I missing on rlandy patch ?	18:59
weshay\|ruck	well.. that's a problem	18:59
rlandy	rfolco: ugh .. new get-hash ... pls point	18:59
rfolco	ci-config/roles/get_hash	19:00
weshay\|ruck	rfolco, rlandy periodic-tripleo-centos-8-master-promote-promoted-components-to-tripleo-ci-testing	19:00
weshay\|ruck	should be using the new api of course	19:00
rfolco	weshay\|ruck, rlandy can I fix the new api ?	19:00
weshay\|ruck	and ya..	19:01
weshay\|ruck	--commit-hash --distro-hash	19:01
weshay\|ruck	are blank	19:01
rlandy	what are we fixing now?	19:01
weshay\|ruck	rlandy, so many things	19:01
weshay\|ruck	rlandy, you fix the centos vs centos8	19:01
rlandy	fine	19:01
weshay\|ruck	rfolco, you figure out for the new api.. how to pass build_id to the new api	19:02
mjturek	is there something I need to do to ensure that I run tripleo-ci playbooks with python3? Hitting this error currently https://logserver.rdoproject.org/ci.centos.org/tripleo-upstream-containers-build-master-ppc64le/2036/logs/logs/build.log	19:09
rlandy	rfolco: https://review.rdoproject.org/r/#/c/25063/ - pls review	19:11
*** tesseract has quit IRC		19:16
rfolco	rlandy, not a nit, just to make it more readable	19:22
rfolco	rlandy, 2 tasks w/ when: would be better	19:22
rlandy	rfolco: it doesn;t work	19:22
*** amoralej is now known as amoralej\|off		19:22
rfolco	what ?	19:23
rfolco	set_fact:	19:23
rlandy	still taking centos7	19:23
rlandy	put a debug in there	19:23
rlandy	otherwise splitting tasks for sure	19:23
rfolco	rainstorm, lights blinking	19:24
rfolco	omg	19:24
rfolco	taking centos7?	19:25
rfolco	these %- markers are so annoying and confusing	19:26
rfolco	put it in a when clause and that's it, I can create the molecule test to verify it	19:27
rlandy	I think that config playbook does not take depends on	19:27
rlandy	not dsplaying debug	19:27
rlandy	rfolco: fine - taking those marker sout	19:28
rlandy	rfolco: edited https://review.rdoproject.org/r/#/c/25063/ pls check the when conditions	19:34
rlandy	all centos8	19:35
rlandy	only component?	19:35
rfolco	rlandy, integration and component (all centos8) --> new api	19:37
rfolco	I think its good other than name of task	19:37
rlandy	rfolco: pls comment	19:37
rlandy	and I will fix	19:38
rfolco	rlandy, as long as linters don't complain	19:39
rfolco	+2	19:39
chandankumar	rlandy, rfolco https://review.rdoproject.org/r/24948 and https://review.rdoproject.org/r/25036 want to merge these two? or move to weekend pipeline?	19:40
rlandy	rfolco: updated task name - pls revote	19:41
rlandy	we will have to merge to test this	19:41
rlandy	chandankumar: yeah - eventually - just trying to fix the promote jobs first	19:42
rlandy	otherwise nothing will work	19:42
rlandy	I hate config	19:42
rfolco	chandankumar, rlandy lets focus on compute/security and then merge all components	19:43
rfolco	my 2 cent	19:44
rlandy	we're not even at that point yet	19:44
rlandy	no component-ci-testing	19:44
rlandy	no containers	19:44
rlandy	^^ need that first	19:44
rlandy	rfolco: molecule on get-hash - all yours - not touching that now	19:45
rfolco	ack	19:46
mjturek	weshay\|ruck: do you know how we're supposed to handle dropped py2 support in tripleo-ci jobs? the ppc64le job is broken by it	19:53
*** sshnaidm is now known as sshnaidm\|afk		19:54
rlandy	weshay\|ruck: rfolco: woohoo ... https://trunk.rdoproject.org/centos8-master/component/security/component-ci-testing/	19:55
rlandy	https://trunk.rdoproject.org/api-centos8-master-uc	19:56
weshay\|ruck	mjturek, which part	19:56
weshay\|ruck	broke	19:56
weshay\|ruck	container-build?	19:57
mjturek	yes	19:57
mjturek	https://logserver.rdoproject.org/ci.centos.org/tripleo-upstream-containers-build-master-ppc64le/2036/logs/logs/build.log	19:57
weshay\|ruck	mjturek, I think you need to move to centos-8	19:57
rlandy	weshay\|ruck: retrying https://review.rdoproject.org/r/#/c/25064/	19:57
weshay\|ruck	we have one centos-8 job up on build containers	19:57
mjturek	weshay\|ruck: so you guys are stopping centos-7 builds?	19:57
weshay\|ruck	https://review.opendev.org/#/c/708218/	19:57
weshay\|ruck	mjturek, for ussuri yes.. asap.. then train.. not sure after that	19:58
rlandy	ugh - we need to vars fix	19:58
rlandy	putting that in	19:58
mjturek	weshay\|ruck okay fair enough	19:58
weshay\|ruck	mjturek, tripleo-build-containers-centos-8SUCCESS in 47m 38s (non-voting)	19:58
weshay\|ruck	mjturek, ya.. it sucks	19:58
weshay\|ruck	we're getting centos-8 awfully late in the py27 cycle	19:58
mjturek	weshay\|ruck: speaking of which are you guys moving image builds to centos-8?	19:58
weshay\|ruck	we're here :)	19:58
mjturek	awesome	19:59
mjturek	baha ^	19:59
mjturek	so that's good news bad news for us	19:59
weshay\|ruck	mjturek, so no kolla patches required.. should work from head	19:59
weshay\|ruck	rfolco, rlandy myself and marios have been involved	19:59
mjturek	bad news is that we need to move to centos-8 builds... which I think means we'll need to bug cico to provide centos-8 nodes?	19:59
baha	Thank you for the ping!	19:59
weshay\|ruck	mjturek, ah ya.. fak.. how could they not have centos-8	20:00
weshay\|ruck	we'll need to start doing the same	20:00
baha	As far as I'm aware cico is all centos 7 right now.	20:00
mjturek	yep	20:00
weshay\|ruck	aint that a bitch	20:00
rfolco	oh boy	20:00
mjturek	and you can't build centos-8 containers on centos-7 right?	20:00
weshay\|ruck	baha, mjturek it's like everyone is tryhing to make this really hard on purpose	20:00
baha	I know right?	20:00
weshay\|ruck	mjturek, no.. don't think so	20:00
mjturek	weshay\|ruck okay fair enough	20:00
baha	The bright side is that the cloud image problem solves itself with centos 8	20:00
mjturek	yep! that's the good news	20:01
baha	The not-so-bright side is most everything else	20:01
mjturek	we were struggling to build images because centos-7 ppc64le cloud images were never published	20:01
weshay\|ruck	rlandy, don't you need depends on?	20:02
rlandy	weshay\|ruck: for https://review.rdoproject.org/r/#/c/25064/? broken on	20:02
rlandy	2020-02-19 19:58:59.780160 \| primary \| + dlrnapi --url https://trunk.rdoproject.org/api-centos8-master-uc --username review_rdoproject_org repo-promote --commit-hash --distro-hash --promote-name tripleo-ci-testing	20:03
rlandy	2020-02-19 19:58:59.906543 \| primary \| usage: dlrnapi repo-promote [-h] --commit-hash COMMIT_HASH --distro-hash	20:03
weshay\|ruck	oh you merged	20:03
rlandy	I merged my fix	20:03
* rlandy thinks ^^ requires rfolco's fix?		20:03
weshay\|ruck	yes	20:03
weshay\|ruck	rfolco, ?	20:03
rlandy	just submitting the tripleo-ansible vara fix	20:04
rlandy	vars	20:04
weshay\|ruck	I see	20:04
weshay\|ruck	020-02-19 19:58:53.522572 \| TASK [get_hash : read md5]	20:04
weshay\|ruck	2020-02-19 19:58:54.047401 \| primary \| d15d61afc27948b715fc611aa20636dd	20:04
weshay\|ruck	2020-02-19 19:58:54.047456 \| primary \| [Zuul] No trailing newline	20:04
weshay\|ruck	2020-02-19 19:58:54.083001 \| primary \| ok: Runtime: 0:00:00.002370	20:04
weshay\|ruck	in the job	20:04
rfolco	yes hold on hold on....	20:04
rfolco	distro-hash is required	20:05
rfolco	and there is no aggregate-hash yet afaik,	20:05
rfolco	so we need to make commit-hash the new aggregate-hash, pretend like I mean	20:05
weshay\|ruck	rfolco, rlandy [13:05:41] <rfolco> so we need to make commit-hash the new aggregate-hash, pretend like I mean	20:08
weshay\|ruck	oops	20:08
weshay\|ruck	sorry	20:09
weshay\|ruck	rfolco, rlandy meant https://dlrn.readthedocs.io/en/latest/api.html#post-api-promote	20:09
weshay\|ruck	When the projects.ini use_components option is set to true, an aggregated repo file will be created, including the repo files of all components that were promoted with the same promotion name. The hash of that file will be returned as aggregated_hash. If the option is set to false, a null value will be returned.	20:09
weshay\|ruck	so same api.. same call.. we just need to pass the build_id or aggregated_hash to both commit-hash and distro-hash.. and one is ignored I think	20:09
rfolco	weshay\|ruck, try omitting distro-hash	20:10
weshay\|ruck	rfolco, rlandy let's chat	20:10
weshay\|ruck	let's do this together	20:10
rfolco	dlrnapi repo-promote: error: the following arguments are required: --distro-hash	20:10
rfolco	client is not ready I guess ?	20:10
weshay\|ruck	it works.. panda did it	20:11
weshay\|ruck	https://meet.google.com/vgk-oeuy-oqi?authuser=1	20:11
mjturek	weshay\|ruck: so there is no timeline for centos-8 availability on cico for ppc64le. It's planned, but there's no clue as to when it will even happen. I would be lying if I said this isn't immensely frustrating.	20:12
weshay\|ruck	mjturek, I hear ya.. and I'll one up you.. they are trying to have us change clouds when doing all this shit too	20:13
mjturek	fun	20:14
mjturek	weshay\|ruck: so as it stands we do not have a path to restore the ppc64le job, I'm not sure what we can do	20:17
weshay\|ruck	jebus	20:18
mjturek	if you think of anything please let us know	20:20
rfolco	weshay\|ruck, rlandy https://review.rdoproject.org/r/25065 Set aggregate_hash as commit_hash	20:22
weshay\|ruck	https://review.rdoproject.org/r/#/c/25064/	20:23
rlandy	https://review.rdoproject.org/r/#/c/24948/	20:24
rlandy	rfolco: weshay\|ruck: ^^	20:24
rlandy	weshay\|ruck: https://review.rdoproject.org/r/#/c/24995/11/.zuul.yaml	20:27
mjturek	weshay\|ruck: I'm confused, does this mean all centos-7 jobs are failing?	20:27
*** jmasud has joined #oooq		20:27
mjturek	rfolco ^	20:28
weshay\|ruck	mjturek, no they are still working atm	20:29
rfolco	mjturek, sorry, need context	20:29
mjturek	weshay\|ruck: how are you guys achieving that?	20:30
mjturek	rfolco: our job is failing because of py3 support getting dropped from kolla	20:30
mjturek	py27* sorry	20:30
rfolco	hmm	20:31
mjturek	rfolco: are your centos-7 jobs using py3?	20:33
mjturek	rfolco: or you guys pinning kolla?	20:33
mjturek	or are you guys no longer building centos-7 containers? we're very confused here	20:35
weshay\|ruck	rfolco, ssh moter.rdoproject.org	20:39
weshay\|ruck	rfolco, ssh promoter.rdoproject.org	20:39
rlandy	dlrnapi --url https://trunk.rdoproject.org/api-centos8-master-uc --username review_rdoproject_org repo-promote --commit-hash a9d30b2f6dca34bc5278bb7eba024e40c17d0dd2 --distro-hash e3a3586e5986d324ceb2b0cf4d66455d032fff1a --promote-name component-ci-testing	20:43
rfolco	mjturek, not pinning afaik... here is one example https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_349/706962/3/check/tripleo-build-containers-centos-7-buildah/349dfb3/	20:43
rfolco	sorry for the delay @mtg	20:43
mjturek	rfolco: okay	20:44
mjturek	okay you guys are still using py27 for these jobs	20:46
rfolco	mjturek, apparently yes no changes	20:46
rlandy	https://softwarefactory-project.io/r/#/c/17091/	20:46
mjturek	rfolco is there any centos7 container build runs that happened after 18:00 UTC? The breaking change didn't merge until then/	20:50
*** jfrancoa has quit IRC		21:04
mjturek	yeah there hasn't been a run since the breaking change	21:05
weshay\|ruck	rfolco, rlandy https://gist.github.com/weshayutin/c77a2aa33c0d7a4c9b93481dc2f789f4	21:30
*** yolanda has quit IRC		21:38
weshay\|ruck	rfolco, before you go	21:44
weshay\|ruck	rfolco, give me access to edit https://hackmd.io/5uYRmLaOTI2raTbHWsaiSQ?both	21:44
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
rfolco	calm	21:45
weshay\|ruck	rfolco, ping	21:45
rfolco	keep	21:45
weshay\|ruck	rfolco, ping	21:45
rfolco	calm	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:45
weshay\|ruck	rfolco, ping	21:46
weshay\|ruck	rfolco, ping	21:46
weshay\|ruck	rfolco, ping	21:46
weshay\|ruck	:)	21:46
weshay\|ruck	now I have access	21:46
weshay\|ruck	weird	21:46
rfolco	I just did	21:46
rfolco	you asked	21:46
weshay\|ruck	ok.. thanks	21:46
weshay\|ruck	spec coming up	21:46
rfolco	while you were flooding the channel I was working	21:46
rfolco	:)	21:46
rfolco	lol	21:46
*** yolanda has joined #oooq		21:52
*** tosky has joined #oooq		22:00
*** yolanda has quit IRC		22:06
*** saneax has quit IRC		22:16
*** TrevorV has quit IRC		22:21
*** dpawlik has quit IRC		22:21
*** ysandeep\|gone is now known as ysandeep		23:27
rlandy	weshay\|ruck: rfolco: have to go now ... bbl .. this works for me locally ... https://review.rdoproject.org/r/25069	23:27
rfolco	cool	23:28
*** rlandy is now known as rlandy\|bbl		23:28
*** tosky has quit IRC		23:34

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!