Thursday, 2021-10-21

« Wednesday, 2021-10-20 Index Friday, 2021-10-22 »
*** rlandy|ruck|bbl is now known as rlandy|ruck00:56
*** carloss_ is now known as carloss01:09
*** pojadhav is now known as pojadhav|ruck04:57
*** ykarel|away is now known as ykarel05:15
*** ysandeep|away is now known as ysandeep05:34
*** jpena|off is now known as jpena06:53
*** ysandeep is now known as ysandeep|trng06:59
bhagyashris_chandankumar, re run the container build push job for c9 and here we dont have centos9.repo https://logserver.rdoproject.org/50/36350/1/check/periodic-tripleo-ci-build-containers-centos-9-push-master/225929e/etc/yum.repos.d/07:46
*** bhagyashris_ is now known as bhagyashris07:46
chandankumarbhagyashris: awesome07:46
chandankumarbhagyashris: patch link please07:47
bhagyashrischandankumar, do you want me to run the image build jjobs as well?07:47
bhagyashrisbecause i just ran the container build jobs07:47
chandankumarbhagyashris: yes, please07:47
bhagyashrisok i will update the patch 07:48
bhagyashrischandankumar, here the patch https://review.opendev.org/c/openstack/tripleo-ci/+/814749 , rerunning the image build jobs https://review.rdoproject.org/r/c/testproject/+/3546507:54
*** ykarel is now known as ykarel|lunch08:53
*** ysandeep|trng is now known as ysandeep|lunch08:55
jbadiapa has anyone tried to use deploy with tripleo-quickstart recently?09:18
jbadiapaI got a weird error on the undercloud.qcow2 convert_image.sh script09:19
jbadiapathe error is when creating the swapfile09:21
*** ysandeep|lunch is now known as ysandeep09:31
ysandeepjbadiapa, I have not tried manual deployment lately but could you please share the error snippet, may be we have noticed same in CI.09:32
akahatchandankumar, backport is worked for victoria: https://review.rdoproject.org/r/c/openstack/octavia-tempest-plugin-distgit/+/36335 testproject:https://review.rdoproject.org/r/c/testproject/+/34602/4/09:42
akahatchecking with ussuri09:42
chandankumarakahat: awesome09:43
chandankumarakahat: paste the result in the review09:43
chandankumarand with patch09:43
akahatokay09:44
arxcruzakahat: chandankumar https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/809979 done 09:45
jbadiapasure, it's a long log09:46
jbadiapaysandeep, https://paste.opendev.org/show/810135/ 09:46
jbadiapaysandeep, this is where I think is failing09:49
jbadiapabut removing it the creation of the swapfile does not help.09:49
jbadiapathere is an issue regarding the disk full when injecting the agents.... I'm goint to tried disabling the whole_disks_images and see09:50
ysandeep~~~09:51
ysandeep+ dd if=/dev/zero of=/swapfile bs=1024 count=419430409:51
ysandeepdd: error writing '/swapfile': No space left on device09:51
ysandeep~~~09:51
ysandeepjbadiapa, ^^ looks like this is causing the issue 09:52
jbadiapaysandeep, yes, but I removed it from the conver_images.sh script and still failing due to disk full on a different place09:53
ysandeeppojadhav|ruck, ^^ are we hitting any issue on image build/ ovb jobs in component line during image build step?09:57
chandankumarmarios: please have a look at this patch https://review.opendev.org/c/openstack/tripleo-ci/+/813618 when free, thanks :-)09:57
marioschandankumar: ack 09:58
pojadhav|ruckysandeep, nope.. if i notice, will ping you10:00
ysandeepjbadiapa, fyi.. hardprov is making some changes at various places for making whole-disk image as default10:01
ysandeepjbadiapa, https://review.opendev.org/q/topic:%22bp%252Fwhole-disk-default%22+(status:open%20OR%20status:merged) 10:01
ysandeepBut i don't see any recent merged change that could cause this.. 10:01
ysandeepjbadiapa, we are not seeing same issue in CI, It is possible to report a bug with reproducer steps, I can try to do a manual deployment next week to confirm if I am also hitting this, and then we can collaborate to fix this10:03
jbadiapaI started to have the issue yesterday, But I havent deployed for a whole week before that.10:03
jbadiapaysandeep, sure...10:03
jbadiapathx10:03
ysandeepjbadiapa, :) you are welcome.10:04
jbadiapaysandeep, https://bugs.launchpad.net/tripleo/+bug/1948001 10:13
jbadiapaI aslo added the whole stderr of the command on a comment as it didnt let me added on the description10:13
ysandeepjbadiapa: ack, thanks for filing the bug10:14
*** ykarel|lunch is now known as ykarel10:20
jbadiapanp :)10:21
jbadiapaysandeep, one question https://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/CentOS-8/master.yml#L33 does not need the whole_disk_images|bool ? 10:23
*** rlandy is now known as rlandy|ruck10:33
rlandy|ruckakahat: hello10:34
rlandy|ruckready to meet?10:34
akahatrlandy|ruck, yes10:35
akahatrlandy|ruck, https://meet.google.com/mbg-pzsw-rgt?authuser=010:35
rlandy|ruckakahat: https://meet.google.com/qjg-yvrt-bfg?pli=1&authuser=010:38
akahatjoining10:38
rlandy|ruckakahat: lol - joined both - you weren't in either10:40
rlandy|ruckwhich gmeet are you on?10:40
akahatrlandy|ruck, one which you shared.10:40
rlandy|ruckk - joining that one10:41
akahatrlandy|ruck, https://review.rdoproject.org/r/c/testproject/+/3460210:41
akahatrlandy|ruck, https://review.rdoproject.org/r/q/4c153372e404a7e5d75edd815a5deba74843e4ef10:42
akahathttps://review.opendev.org/c/openstack/tripleo-quickstart/+/81474010:44
akahatrlandy|ruck, ^^10:44
chandankumarakahat: please test it10:47
chandankumarakahat: https://review.rdoproject.org/r/c/openstack/octavia-tempest-plugin-distgit/+/3636810:47
chandankumaras we tested victoria10:47
*** dviroel|rover|out is now known as dviroel|rover10:51
rlandy|ruckdviroel|rover: hey10:53
rlandy|ruckdviroel|rover: pooja has an  outage10:53
rlandy|ruckshe will join when we gets access again10:53
dviroel|roverrlandy|ruck: hi, morning10:53
dviroel|roverrlandy|ruck: ok10:53
akahatchandankumar, yes. it's running here: https://review.rdoproject.org/r/c/testproject/+/34602, i've updated the depends-on patch.10:54
chandankumarakahat: once finish ping me with result, will do further action10:54
chandankumarrlandy|ruck: ^^10:54
akahatchandankumar, okay10:54
akahatrlandy|ruck, chandankumar please take a look when you are free, thanks. https://review.opendev.org/c/openstack/tripleo-ci/+/81361910:58
ysandeepjbadiapa: hey sry was afk for few mins, yes would be safer to add |bool there.. otherwise it can be wrongly evaluated.. same is mentioned in this article: https://stackoverflow.com/questions/52458247/how-to-fix-false-being-evaluated-to-true-in-ansible-jinja2-templates  10:59
rlandy|ruckakahat: one more scenario010 on octavia ... https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/81485511:03
rlandy|ruckpls see comments here11:04
rlandy|ruckand touch base with pooja11:04
chandankumarsshnaidm: this one is good to go https://review.opendev.org/c/openstack/tripleo-ansible/+/81402611:04
akahatrlandy|ruck, ack11:04
rlandy|ruckchandankumar: joining11:04
sshnaidmchandankumar, done11:06
ysandeepjbadiapa, hi just did a quick test https://paste.openstack.org/show/810140/ , from testing looks like.. issue can only arise if whole_disk_image is defined as 'false' instead of false .. But yeah as a thumb rule we should add |bool filter 11:07
dviroel|roverrlandy|ruck: https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/81482211:12
rlandy|ruckdviroel|rover: thanks - merging11:18
rlandy|ruckdviroel|rover: pls close out the bug and the CIX card - thank you11:18
dviroel|roverack, tks11:18
mariosjpena: no why? ! what did we ever do to you?!11:23
jpenamarios: X-D. Nothing special, but I needed to try something different11:24
mariosjpena: yeah i get that ;)11:24
rlandy|ruckarxcruz: hey - fyi  merging https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/80997911:28
arxcruzrlandy|ruck: ack 11:28
*** jpena is now known as jpena|lunch11:29
rlandy|ruckbhagyashris:hey - can you join https://meet.google.com/vzm-nrah-qqf?authuser=011:38
rlandy|ruckbhagyashris: hi - pls will you make a doc that chandankumar and I can access with a list of everything you do as TC and instructions to us on how to do the JIRA sprint work11:41
rlandy|ruckjpena|lunch: best of luck - we will miss you! thank you for all you did for the CI team 11:46
chandankumarjpena|lunch: Thank you for everything, All the best for new adventure :-) I am going to bug you again time to time :-)11:58
rlandy|ruckcompany meeting time12:02
bhagyashrisrlandy|ruck, sure12:11
bhagyashrisrlandy|ruck, hey you around? i went for tea break 12:12
rlandy|ruckbhagyashris: ack - on meeting though12:13
bhagyashrisrlandy|ruck, np 12:13
*** amoralej is now known as amoralej|lunch12:14
rlandy|ruckdviroel|rover: https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-8-standalone-on-multinode-ipa12:16
rlandy|rucktrouble with 1.1.1.1 on other clouds12:16
dviroel|roveroh, lets move to opendns default for all12:19
*** jpena|lunch is now known as jpena12:22
jpenathank you rlandy|ruck chandankumar :)12:22
rlandy|ruckdviroel|rover: we'll have to spin test that on the various clouds12:23
rlandy|ruckthere are multiple forwarders and 1.1.1.1 fails out12:23
dviroel|roverthe problem with 1.1.1.1 and 1.0.0.1 is that they are not responding correctly the ipa-server query, to test them against extended dns, so we can focus only on ipa jobs12:27
*** pojadhav is now known as pojadhav|ruck12:33
bhagyashrischandankumar, fyi standalone-full-tempest-scenario is passing periodic-tripleo-ci-centos-9-standalone-full-tempest-scenario-master https://review.rdoproject.org/zuul/build/b8eec43bebf142468a34c216522703e8 : SUCCESS in 1h 48m 52s12:34
chandankumarbhagyashris:  thank you :-)12:34
chandankumarbhagyashris:  we are waiting for series of patches to land12:34
bhagyashrischandankumar, yeah ...12:34
chandankumaron standalone patches merges, we can merge other reviews12:34
bhagyashrisack12:43
jbadiapaysandeep, I think the issue might be related to this https://review.opendev.org/c/openstack/tripleo-common/+/79804113:00
jbadiapaBut I dont know how to test it, tbh13:00
rlandy|ruckdviroel|rover: let's try replace them with open dns13:03
rlandy|ruckand see how we go13:03
ysandeepjbadiapa, hmm.. that seems old, it was merged in july.. not sure.. 13:04
rlandy|ruckPTG time13:04
ysandeepjbadiapa, i can check on Monday.13:04
rlandy|ruckhackfest13:04
rlandy|ruckhttps://etherpad.opendev.org/p/tripleo-directord-hackfest13:04
jbadiapaysandeep: np, using the whole_disk_images=false works13:04
jbadiapaysandeep, thx13:05
ysandeepwhole_disk_images=false works : nice!13:07
jbadiapaafter applying my submitted patch for tq13:09
*** amoralej|lunch is now known as amoralej13:12
dviroel|roverrlandy|ruck: only changing 'forwarder_address_default' won't work for some clouds13:35
dviroel|roverbecause https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_bc1/814824/1/gate/tripleo-ci-centos-8-standalone-on-multinode-ipa/bc1931c/logs/198.72.124.68/etc/unbound/forwarding.conf13:35
dviroel|roverthey will also use those other ips from unbound, according with https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/ipa-multinode/tasks/ipaserver-subnode-install.yml#L90-L9713:36
dviroel|roveras we can see here https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_bc1/814824/1/gate/tripleo-ci-centos-8-standalone-on-multinode-ipa/bc1931c/logs/quickstart_install.log13:36
dviroel|rover"--forwarder 1.0.0.1  --forwarder 8.8.8.8    --forwarder 1.1.1.1 "13:36
dviroel|roveripa-server will test all against EDNS and if just one fails, installation will be aborted13:38
ade_leedviroel|rover, nice - the zuul-jobs change merged14:20
dviroel|roverade_lee: yes :)14:20
rlandy|ruckdviroel|rover: yeah the forwarder unbound if using should be the best for the cloud14:42
rlandy|ruckunless we added some logic to remove 1.1.1.114:43
rlandy|ruckade_lee: ^^ any thoughts?14:43
rlandy|ruck1.1.1.1 is failing to resolve14:43
rlandy|ruckon many clouds now14:43
dviroel|rover1.0.0.1 also fails14:43
rlandy|ruckstarted on vexx14:43
rlandy|rucknow we are seeing it elsewhere14:43
rlandy|rucksshnaidm: pls join tripleo-interview on internal14:45
rlandy|ruckdviroel|rover: we could not use unbound14:46
rlandy|ruckforwarders14:46
rlandy|ruckthat is actually where we started14:46
rlandy|ruckthen we added unbound forwarders back14:46
rlandy|ruckif we add logic to get rid of 1.1.1.1 out of forwarders,14:47
rlandy|ruckwe may make situation worse14:47
rlandy|ruckidk14:47
ade_leerlandy|ruck, sorry - just got back in -- whats up?14:51
rlandy|ruckysandeep: have you been able to reprovision hardware boxes? I am getting a network error connecting to console from drac14:52
ade_leerlandy|ruck, is the problem that 1.1.1.1 and 1.0.0.1 does not resolve?  can we check before we add it to the forwarder?14:52
rlandy|ruckade_lee: pls see the failures in multinode-ipa https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-8-standalone-on-multinode-ipa14:53
rlandy|ruckwe switched to use open DNS on vexxhost in RDO14:53
rlandy|ruckbut in upstream we are now hitting this on other upstream clouds14:53
* ade_lee looking14:55
ysandeeprlandy|ruck: are you opening drac directly / idrac from container? last time when I reprovision some boxes I still need to use old java from drac containers which we have.14:56
rlandy|ruckysandeep: from container14:57
rlandy|ruckit opens the console14:57
rlandy|ruckand then closes it due to network error14:58
rlandy|ruckI need to update the box from c7 to c8 stream14:58
rlandy|ruckmight just have to update repos14:58
rlandy|ruckneed to join interview14:58
rlandy|ruckwill show you later14:58
ysandeeprlandy|ruck: ack14:59
*** ysandeep is now known as ysandeep|afk15:04
*** ysandeep|afk is now known as ysandeep15:25
*** jpena is now known as jpena|off15:52
chandankumarrlandy|ruck: logging out now, see ya on monday!, Happy weekend :-)15:54
rlandy|ruckchandankumar: thanks - see you monday15:55
rlandy|ruckon interview call15:55
*** amoralej is now known as amoralej|off15:56
*** ykarel is now known as ykarel|away15:58
dviroel|roverade_lee: so, ipa-server does an initial check for all forwarders if they support EDNS, and if one of them fails, the server install fails. And now, 1.1.1.1 and 1.0.0.1 are failing in this validation.16:16
dviroel|roverade_lee: not sure if something change at ipa-server query or if is something at DNS side16:16
ade_leedviroel|rover, rlandy|ruck sorry -been in ptg meetings16:19
rlandy|ruckade_lee: no worries16:19
dviroel|roverade_lee: np16:19
rlandy|ruckdviroel|rover dn I are in interview right now16:19
ade_leedviroel|rover, rlandy|ruck and I have a meeting in 10 :)16:20
ade_leeso - we'll figure it out and answer when we can16:20
rlandy|ruckade_lee: let's touch base this afternoon16:20
ade_leeok16:20
* ysandeep out see you next week, Have a wonderful recharge day and weekend! 16:22
*** marios is now known as marios|out16:22
*** ysandeep is now known as ysandeep|out16:23
*** bhagyashris_ is now known as bhagyashris16:27
* bhagyashris leaving for the day Have a nice weekend !16:28
* dviroel|rover lunch16:41
dviroel|roverrlandy|ruck: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/815046 - trying a different fix here18:19
* rlandy|ruck looks18:26
rlandy|ruckdviroel|rover: lol - that condition is becoming a whole logic unto itself18:28
rlandy|ruckbu ok- let's try it18:28
rlandy|ruckdviroel|rover: how stable are the introspection change jobs now?18:29
* rlandy|ruck checking on why wallaby change is not in tripleo-ci-testing18:29
rlandy|ruckhttps://wimp.usersys.redhat.com/?change_id_input=I0a73443630faf25de870a7878a9568ac8319aee918:30
dviroel|roverlets see fs03518:30
rlandy|ruckstill says promoted-components18:30
dviroel|roveryeah18:30
rlandy|ruckevery day we move promoted-components to tripleo-ci-testing18:30
dviroel|roverhumm18:31
rlandy|ruckopenstack-tripleo-heat-templates-14.3.1-0.20211020082010.aa4de7d.el8.noarch.rpm2021-10-20 08:23 581K 18:31
rlandy|ruckhttps://trunk.rdoproject.org/centos8-wallaby/component/tripleo/tripleo-ci-testing/18:32
rlandy|ruckdviroel|rover: it is ^^18:32
rlandy|ruckI think Tengu's tool is just not reporting that  condition18:32
rlandy|ruckI think it must just report promoted-components and current-tripleo18:33
dviroel|roveryeah, maybe 18:33
rlandy|ruckopenstack-tripleo-heat-templates-14.3.1-0.20211016064329.5ede212.el8.noarch.rpm2021-10-16 06:46 583K 18:33
rlandy|ruckhttps://trunk.rdoproject.org/centos8-wallaby/component/tripleo/current-tripleo/18:33
rlandy|ruck^^ still old18:33
rlandy|ruckso  ok - we need to promote this hash18:33
rlandy|ruckdviroel|rover: periodic-tripleo-centos-8-buildimage-overcloud-hardened-full-wallaby  18:35
rlandy|ruckpython3 roles/rrcockpit/files/telegraf_py3/ruck_rover.py --release wallaby18:35
rlandy|ruckthat job is removed18:35
rlandy|ruckwe need to update conditions18:35
* rlandy|ruck gets18:35
dviroel|roveroh, yeah, o only see periodic-tripleo-centos-8-buildimage-overcloud-hardened-uefi-full-wallaby running18:36
rlandy|ruckhttps://github.com/rdo-infra/rdo-jobs/commit/5ab4d2804da898fc053f7aa94a84e4726661890b18:36
* rlandy|ruck puts in patch18:36
rlandy|rucksec18:36
Tengurlandy|ruck: hmm I don't remember if there are 2 or 3 status for that.... I'll check on Monday18:39
rlandy|ruckTengu: hey - yeah - no worries - we can show you on monday18:39
rlandy|ruckwe go promoted-components -> tripleo-ci-testing -> current-tripleo18:39
rlandy|ruckstate 2 is missing18:39
Tenguhmmm yeah. I think I don't have that one indeed.18:39
Tengurlandy|ruck: ah, yeah. I have consistent, promoted-components and current-tripleo18:43
rlandy|ruckTengu: no rush - but yeah, we check that additional state18:43
rlandy|ruckhttps://review.rdoproject.org/r/c/rdo-infra/ci-config/+/36373 Add uefi-full-wallaby to criteria -- readd fs03518:44
rlandy|ruckdviroel|rover: ^^ pls check me18:44
Tenguso I can add that new step in-between.18:44
rlandy|ruckplease18:44
TenguI'm checking in reverse - so current-tripleo, if not -> tripleo-ci-testing, if not -> promoted-components, if not -> consistent18:45
TenguI'll do that Monday if it's OK for you18:45
rlandy|ruckTengu: sure - have a good recharge weekend18:46
Tengusame for you :)18:46
dviroel|roverthanks Tengu, have a good weekend18:47
Tengusee you next week folks ;)18:48
rlandy|ruckbye18:48
dviroel|roverso.. https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train+18:49
dviroel|roverhttps://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train/5db8f21/logs/quickstart_install.log 18:49
dviroel|roverlastest one ^18:49
dviroel|rovertook 6 min18:49
rlandy|ruckperiodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-trainopenstack/tripleo-cimasteropenstack-periodic-integration-stable43 hrs 35 mins 3 secs18:50
rlandy|ruckworking again :)18:50
rlandy|ruckhow??18:50
rlandy|rucklol18:50
dviroel|rover(ಠ_ಠ)18:50
rlandy|ruckyeah and train promoted18:50
rlandy|ruckmiracle18:50
rlandy|rucktestprojected the two jobs missing from victoria18:52
rlandy|ruckarxcruz: hey - just checking in on update/upgrades patch18:52
rlandy|ruckcan we help there?18:52
rlandy|ruckdviroel|rover: when https://jenkins-cloudsig-ci.apps.ocp.ci.centos.org/view/phase-1-pipelines/ goes all green, we hold a LARGE party18:53
*** sshnaidm is now known as sshnaidm|afk18:53
dviroel|roveroh, only one missing, we will get this one green too18:54
rlandy|ruckthat is the wallaby promotion we are chasing18:54
arxcruzrlandy|ruck: https://review.opendev.org/c/openstack/tripleo-ci/+/81362919:00
arxcruzdone 19:00
rlandy|ruckarxcruz: nice ...19:00
rlandy|ruckdo we need more testing here?19:00
rlandy|ruckie: do we know an upgrade job works w/o a provider?19:01
rlandy|rucklike in periodics?19:01
rlandy|ruckarxcruz: ^^?19:08
arxcruzrlandy|ruck: not sure if in tripleo, but pretty sure in other projects we might have it 19:08
rlandy|ruckarxcruz: sorry - meaning?19:09
arxcruzas marios said, it's very hard to test, so it's more like "introducing a new bug" approach 19:09
rlandy|rucklol19:09
arxcruzrlandy|ruck: some puppet job ?19:09
arxcruzrlandy|ruck: maybe better leave it to monday ?19:09
rlandy|ruckarxcruz: update/upgrage jobs run in periodic w/o provider19:10
arxcruzso we can monitor if it will fail because of the patch 19:10
rlandy|ruckarxcruz: 2 things ...19:10
rlandy|ruckpls testproject with an update/upgrade job in RDO 19:10
arxcruzrlandy|ruck: if it will run without provider, it will not have problem 19:10
arxcruzrlandy|ruck: ok 19:10
rlandy|ruckarxcruz: and ack19:10
rlandy|rucklet's merge on monday19:10
rlandy|ruckwhen we can watch the fallout19:10
rlandy|ruckthank you for taking care of this19:11
* dviroel|rover power outage19:55
rlandy|ruckdviroel|rover: oh gosh19:56
rlandy|ruckyou  ok?19:56
dviroel|roveryeah, ups should hold for a whil20:03
rlandy|ruckk - if you need to drop, I am here20:03
rlandy|ruckwallaby is promoting20:19
dviroel|rover\o/20:23
rlandy|ruckhttps://review.rdoproject.org/r/c/rdo-jobs/+/3612120:23
rlandy|ruckdviroel|rover: ^^ and the test definitions20:23
rlandy|rucksorry - typing on two channels20:23
rlandy|ruckhttps://review.opendev.org/c/openstack/tripleo-quickstart/+/81308320:23
rlandy|ruckto enable RBAC testing20:24
rlandy|ruckrechecking https://review.rdoproject.org/r/c/testproject/+/3612220:26
dviroel|roverack, lets wait for the results of this recheck20:27
rlandy|ruckI think it's not done20:28
rlandy|ruckbut we could merge the job definitions20:28
* rlandy|ruck trying to close out work20:28
rlandy|ruckso much open stuff20:28
dviroel|rover+120:28
dviroel|roverright, just job definition20:29
ade_leerlandy|ruck, dviroel|rover yo20:52
ade_leehey -so quick question -- I want to set IscsidCHAPAlgorithms in the fips jobs --- whats the best way to do this?20:53
ade_leerlandy|ruck, dviroel|rover I want to do this in https://review.opendev.org/c/openstack/tripleo-ci/+/80821520:56
* rlandy|ruck looks20:56
ade_leerlandy|ruck, I think its just a matter of me setting the right variable20:57
rlandy|ruckdepends where you want it set20:57
ade_leewhatever needs to be set to set a THT variable20:57
rlandy|ruckah20:57
rlandy|ruckenvironment file20:57
rlandy|ruckthat you pass to deployment20:57
ade_leeah - ok20:58
* rlandy|ruck gets20:58
rlandy|ruckyou can put it in THT or extras20:58
ade_leeok so maybe I need to create an environment file for fips settings -- would need that to do a fips deployment in any case.20:58
ade_leeand then reference that 20:59
ade_leeyeah - I'll do that20:59
rlandy|ruckhttps://github.com/openstack/tripleo-heat-templates/tree/master/ci/environments21:00
rlandy|rucksomething like that21:00
rlandy|ruckand then in the deploy ...21:00
dviroel|roverade_lee: hey o/21:02
ade_leerlandy|ruck, standalone_environment_files ?21:02
rlandy|ruckhttps://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-deploy/tasks/deploy-overcloud.yml21:02
rlandy|ruckyou can pass it as an extra to deployment21:02
rlandy|ruckor add in standalone if that is where you want it21:03
rlandy|ruckhttps://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-deploy/tasks/pre-deploy.yml21:03
rlandy|ruckor something like fips_args21:03
rlandy|ruckhttps://github.com/openstack/tripleo-quickstart-extras/commit/cf98a2f811d52b09970fe4b7c0b2c50f24b8c7a221:04
rlandy|ruckexample change21:04
* rlandy|ruck checks standalone21:06
dviroel|roverade_lee: btw, won't you start using 'multinode-fips' for all your jobs? avoid late rebooting is always good :)21:07
rlandy|ruckhttps://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/standalone/templates/standalone.sh.j221:07
rlandy|ruck -e {{ standalone_custom_env_files | join(' -e ') }} \21:07
rlandy|ruckold way21:07
rlandy|ruckhttps://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/standalone/tasks/standalone-install.yml#L2921:07
rlandy|ruck^^ standalone21:08
ade_leedviroel|rover, yes indeed :)21:08
rlandy|ruckhttps://opendev.org/openstack/tripleo-ci/src/branch/master/roles/run-test/tasks/main.yaml#L5621:09
rlandy|ruckyou can even override them in your job21:09
rlandy|ruckfeatureset_override.standalone_environment_files21:09
rlandy|ruckhttps://github.com/rdo-infra/rdo-jobs/blob/7a256eee05123a78877e1936f7277e4bb069f56a/zuul.d/standalone-jobs.yaml#L10821:10
rlandy|ruck^^ example21:10
rlandy|ruckany of those ways will work21:10
ade_leerlandy|ruck, ok - cool thanks - that gives me enough to start with -- I'll ping with more questions as needed21:10
rlandy|ruckade_lee: sure21:11
ade_leesome of this is stuff I planned to do in any case to allow deploy with a fips enabled system to begin with21:11
rlandy|ruckright21:11
rlandy|ruckade_lee: we merged the change to add the image for rhel21:11
rlandy|ruckno idea if it works :)21:11
rlandy|ruckbut it's there21:11
rlandy|ruckrhel 8.421:12
ade_leeI'll put up something that sets the chap algorithms and also uses the new multinode-fips21:12
ade_leenice :)21:12
ade_leeI do need to get back to the job that uses the upstream image and figure out why the test failed21:12
ade_leerlandy|ruck, dviroel|rover ok - so switching back to the ipa job stuff21:13
rlandy|ruckade_lee: dviroel|rover added a patch to skip using 1.1.1.121:13
rlandy|ruckfor some reason, that resolver is erroring now21:13
dviroel|roverhttps://review.rdoproject.org/r/c/testproject/+/35017/16#message-96f0d657d87c2846dc82c4ed76f8098914cc51a021:14
ade_leerlandy|ruck, dviroel|rover mentioned something about no edns?21:14
dviroel|roverade_lee: yes21:14
dviroel|roverthey fail in ipa-server edns validation21:14
rlandy|ruckright21:14
rlandy|ruckso initially we only saw this in vexxhost jobs21:15
ade_leeso what happens when we don't have 1.1.1.1? do we always have a valid forwarder?21:15
rlandy|ruckbit now it's showing up everywhere21:15
rlandy|ruck1.1.1.1 shows up in two cases21:15
rlandy|ruckwhen it's the forwarder from unbound 21:16
rlandy|ruckand when we have to default21:16
rlandy|ruckbecause unbound is using a forwarder that is ipv621:16
rlandy|ruckwhich iirc, ipa does not work with?21:16
ade_leerlandy|ruck, not sure about that -- I would think it should work21:17
rlandy|ruckso now we are trying use open DNS21:17
rlandy|ruckand skip 1.1.1.1 and 1.0.0.121:17
rlandy|ruckif that is what unbound offers up21:18
dviroel|roverhttps://logserver.rdoproject.org/17/35017/16/check/periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-master/1dd9790/logs/38.102.83.6/var/log/ipaserver-install.log.txt.gz21:18
dviroel|roveruboungd also offers 8.8.8.8, which works with ipa-server-install21:18
rlandy|ruck^^ we get rate limited on 8.8.8.821:19
rlandy|ruckso we should use that sparingly 21:19
dviroel|roverwe can add to skip list if needed too21:19
rlandy|ruckhttps://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-8-standalone-on-multinode-ipa21:19
rlandy|ruckade_lee: ^^ current stats21:20
rlandy|ruckwe will need to make the job non-voting again21:20
rlandy|ruckif we don;t sort this out21:20
rlandy|ruckfailing the gate21:20
rlandy|ruckdviroel|rover: how are the testing stats on your skiplist patch?21:20
dviroel|roverall green21:21
rlandy|ruckdviroel|rover: pls post patch21:21
dviroel|roverhttps://review.rdoproject.org/r/c/testproject/+/35017/16#message-96f0d657d87c2846dc82c4ed76f8098914cc51a021:21
rlandy|ruckade_lee: ^^ thoughts?21:21
dviroel|roverjsut added a comment in the patch https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81504621:21
rlandy|ruckhttps://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/81504621:21
ade_leerlandy|ruck, dviroel|rover so is the problem that 1.1.1.1 exists and is available but doesn't support edns?21:21
dviroel|roverade_lee: yes, as far as i can see in the logs21:22
dviroel|roverade_lee: https://92d0f4c4fe5e48d3e242-906d96d1d59ea771e6d0d6132c25fd8a.ssl.cf5.rackcdn.com/814763/1/check/tripleo-ci-centos-8-standalone-on-multinode-ipa/455cee9/logs/198.72.124.69/var/log/ipaserver-install.log21:23
dviroel|rover"ERROR DNS server 1.0.0.1: query '. SOA': All nameservers failed to answer the query . IN SOA: Server 1.0.0.1 UDP port 53 answered A DNS query response does not respond to the question asked."21:23
* ade_lee chatting with ipa folks to see what options are available21:24
dviroel|roverthere is an option to disable this check :)21:24
dviroel|roverbut doesn't look right21:24
ade_leethats what I'm wondering about21:24
dviroel|rover'no_dnssec_validation': False21:25
dviroel|rovershould be this one ^21:25
ade_leedviroel|rover, yup - just confirmed that ..21:30
dviroel|roverade_lee: i'm following sec-interop21:30
dviroel|roveryeah, so there is an option to skip the validation21:31
ade_lee--no-dnssec-validation21:31
ade_leemischief managed :)21:31
dviroel|roverso, I didn't try that cause i'm not sure how this impact in the rest of the instalation and configuration21:32
ade_leedviroel|rover, I don't think we're using dnssec .. but only one way to find out :)21:33
dviroel|roveryeah :)21:33
* dviroel|rover create a new patch21:34
* ade_lee going for run. see you guys tomorrow21:34
dviroel|roverade_lee: see you monday o/21:35
rlandy|rucksee you then21:39
rlandy|ruckdviroel|rover: ^^ if that passes - let's try merge early on monday21:43
rlandy|ruckto clear gate failures21:43
dviroel|roverack, testing now21:43
dviroel|roverhttps://review.rdoproject.org/r/c/testproject/+/3637421:44
dviroel|rovertripleo-quickstart-promote-wallaby-current-tripleo-delorean-minimal deploying overcloud21:54
rlandy|ruckyay21:55
dviroel|rovergoing out for the day21:57
dviroel|roverrlandy|ruck: have a good weekend o/21:58
rlandy|ruckyou too21:58
rlandy|ruckenjoy recharge21:58
* dviroel|rover will check later how things are going21:58
dviroel|roveryou too21:58
*** dviroel|rover is now known as dviroel|rover|afk21:58
rlandy|ruckno worries22:00
rlandy|ruckI'll check in on friday22:00
rlandy|ruckit should be quiet22:01
*** rlandy|ruck is now known as rlandy|ruck|afk22:22
*** rlandy|ruck|afk is now known as rlandy|ruck23:54
« Wednesday, 2021-10-20 Index Friday, 2021-10-22 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!