Thursday, 2021-02-18

*** tosky has quit IRC00:30
*** jmasud has quit IRC01:47
*** jmasud has joined #oooq02:43
*** gouthamr has quit IRC02:59
*** jmasud has quit IRC03:02
*** jmasud has joined #oooq03:05
*** apetrich has quit IRC03:10
*** udesale has joined #oooq04:19
*** ratailor has joined #oooq04:35
*** jmasud has quit IRC04:58
*** ykarel has joined #oooq05:00
*** skramaja has joined #oooq05:03
*** ysandeep|away is now known as ysandeep|ruck05:04
*** jmasud has joined #oooq05:15
*** gouthamr has joined #oooq06:01
*** rlandy|bbl has quit IRC06:17
*** marios has joined #oooq06:18
*** matbu has quit IRC07:10
*** matbu has joined #oooq07:11
*** jmasud has quit IRC07:12
*** slaweq has joined #oooq07:13
*** udesale_ has joined #oooq07:35
*** udesale has quit IRC07:35
*** jpena|off is now known as jpena07:36
*** slaweq has quit IRC07:39
*** slaweq has joined #oooq07:42
chandankumarzbr: hello, need some help on this patch https://review.rdoproject.org/r/#/c/32019/ I am writting to rsync few files from different dir to log server but "msg": "Accessing files from outside the working dir /var/lib/zuul/builds/d00e8aa4ee5b414fa7d818efd000b156/work is prohibited" please have a look07:43
chandankumarsome dirs are created by different users07:43
zbrchandankumar: add become and hope that this task never runs on zuul worker as it may not allow you that.07:52
zbrsorry, executor07:52
chandankumarzbr: sure, thanks!07:54
zbrrsync requires root in most cases, as chmod/chown required it, in addition to needed read access for source.07:55
zbryou also need to be careful if you want to sync those or not, likely not for logs.07:55
*** apetrich has joined #oooq08:17
*** jpodivin has joined #oooq08:19
*** jpodivin has quit IRC08:20
*** jpodivin has joined #oooq08:22
*** ysandeep|ruck is now known as ysandeep|lunch08:26
zbri am attending several devconf.cz talks these today and tomorrow, maybe you find some interesting too https://devconfcz2021.sched.com/08:27
zbrlast night I found a design issue with tq/tqe ansible code, at least the second one fails to pass ansible syntax check on several playbooks.08:37
zbrmainly because the try to include playbooks that do not exist in current repository.08:38
chandankumarjpena: hello, I am working on fixing post log collection for promoter, the log files are created by promoter user, I need to copy it to zuul log exeecutor dir, https://review.rdoproject.org/r/#/c/32019/ but it "msg": "Accessing files from outside the working dir /var/lib/zuul/builds/7847c070a53547d7baaed01dad1376e1/work is prohibited" giving, please have a look when free.08:38
zbrin production it "happens" to work because we install both inside the same folder, mixing them.08:38
zbrbut the code itself of tqe has invalid imports, so both ansible syntax check is failing and the new linter, where ability to load files become mandatory.08:39
zbransible has no support for playbook-import-paths, it has paths for roles, collections, modules, but not for playbooks.08:40
zbrin order to avoid those import failures i had to create 8 placeholder files: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/77639908:41
zbrstill, this will likely override the tq real files, so is not really a solution. Any ideas?08:42
*** tosky has joined #oooq08:45
chandankumarzbr: moving playbooks from tq to tqe is also not going to help as we will hit same problem there also09:04
*** ykarel_ has joined #oooq09:05
zbri wonder how crazy it would be to combine tq and tqe into one repo.09:06
*** ykarel has quit IRC09:08
*** ysandeep|lunch is now known as ysandeep|ruck09:15
chandankumarzbr: may be we can bring this topic today during scrum call? cc bhagyashris09:20
zbrclearly we should, but we may also have to think about the subject before, as the meeting is short.09:21
zbri know that the issue was opened several times in the past, but was defered each time due to worries about implications or effort09:23
zbri wonder if that would endup as a benefit that could save us time as "extras" is always added on top of tq, so it may not be so hard.09:24
*** dtantsur|afk is now known as dtantsur09:31
*** derekh has joined #oooq09:38
*** ykarel_ is now known as ykarel09:49
bhagyashrischandankumar, zbr in case if we have extra time then will discuss in today's meeting or else we discuss this in community call09:51
bhagyashrishttps://hackmd.io/MMg4WDbYSqOQUhU2Kj8zNg?both#2021-02-23-Community-call09:52
bhagyashrisfeel free to add topic in agenda section09:53
zbrsshnaidm, marios: what is your take on the subject ^? does it make sense or not? at least from a high-level.09:53
sshnaidmit's definitely a bad idea09:56
marioszbr: it has been discussed before ... i mean thre will be a whole lot of pain to update all the places currently using those roles as 2 different repos , for starters09:58
marioszbr: but at a high level it seems like a sane idea09:58
marioszbr: and note it would break anyone that isn't us who may be using those roles too10:00
zbran alternative would be to add all files that cause include failures, to exclude list for linting. it lowers to coverage but is low-effort.10:00
marioszbr: so there is a wider issue of how to do it, deprecation period for example and publicity10:00
marioszbr: you can have a press conference10:01
zbrmaybe is too late to spend effort in fixing these design issue if tq, and we should only go the "add to exclude" way.10:01
sshnaidmI think the job should be not to make linter pass by hacks, but configure them to help us to keep our code w/o errors, seems like the main goal of linters is lost here10:09
marioschandankumar: sshnaidm: zbr: please folks when you next have time for reviews add these to your list https://review.opendev.org/c/openstack/tripleo-ci/+/775858 https://review.opendev.org/c/openstack/tripleo-ci/+/775858 https://review.opendev.org/c/openstack/python-tripleoclient/+/774580 (from https://review.opendev.org/q/topic:tripleo-ci-reduce )10:27
mariosthank you10:27
mariosmissed one (duplicated tripleo-ci/+/775858 above) there please https://review.opendev.org/c/openstack/tripleo-ci/+/77457510:28
zbrsshnaidm: let me put it this way: the linter is not at fault here, even running ansible-playbook --syntax-check ... here would fail in an epyc way.10:32
sshnaidmzbr, doesn't matter in this case10:33
zbrwhat happens is that I uncovered a very old and poor design decision. In fact likely syntax check already existed at that point but was not considered.10:34
zbrit does matter, claiming that "linter is useless" is not a constructive approach, we should be able to recognize our mistakes.10:35
zbrif ansible tells you that import_playbook: garbage.yml is an error, it a useful feature, not a bug.10:36
jpenachandankumar: sorry I was busy with another issue. I've been looking at https://review.rdoproject.org/r/32019, and I think the issue is that you're trying to fetch logs from outside the zuul user home. As part of the promoter code (or somewhere before post-run), you should copy the logs you want to fetch somewhere in /home/zuul10:42
*** ksambor is now known as ksambor|lunch10:52
*** ksambor|lunch is now known as ksambor10:53
chandankumarjpena: thanks!11:04
*** rlandy has joined #oooq12:28
*** jpena is now known as jpena|lunch12:30
rlandyweshay|ruck: we're a go on 16.2 build containers on 8.412:34
rlandyrunning a last test on image builds12:34
rlandyysandeep|ruck: ^^12:34
ysandeep|ruckrlandy, ack..12:34
rlandyif that works, I will put in a patch to move the integration line to 8.412:34
ysandeep|ruckrlandy, fyi.. baremetal job are hitting an issue.. i am working on that12:35
ysandeep|ruckfolks can i please get some reviews on https://review.opendev.org/c/openstack/tripleo-quickstart/+/776391 to clear a promotion blocker.12:36
rlandyysandeep|ruck: ack - thanks12:40
weshay|ruckakahat, bhagyashris want to go for it before the scrum or after?12:46
bhagyashrisweshay|ruck, after scrum12:47
akahatweshay|ruck, after scrum12:47
weshay|ruckbhagyashris, ok.. did we get the basic standalone to pass or skip?12:47
bhagyashrisweshay|ruck, standalone job is passing12:47
bhagyashrishttps://review.rdoproject.org/r/#/c/31612/12:48
weshay|ruckbhagyashris, rock on12:48
bhagyashris:)12:48
weshay|ruckTODAY along side the mars rover landing.. WE SHALL LAUNCH THE NEW PROMOTER!!!12:49
weshay|ruckWE SHALL BROADCAST THE PROMOTER IN TMUX!! alongside this video https://mars.nasa.gov/mars2020/timeline/landing/watch-online/12:49
* zbr hopes that our promoter would have a better success ratio than musk rockets12:49
weshay|ruckah.. here comes twitter12:50
weshay|ruckrlandy, I have a recruiting call, will miss the 1/2 of scrum12:51
chandankumarrlandy: marios sshnaidm I have emailed the tripleo-repo cli interface design https://hackmd.io/L2vZMEUtRoy-ZhHqM5GIeQ?view feel free to add comments12:53
marioschandankumar: thanks12:53
sshnaidmchandankumar, ack12:53
ysandeep|ruckfolks can i please get some reviews on https://review.opendev.org/c/openstack/tripleo-quickstart/+/776391 to clear a promotion blocker.12:54
ysandeep|ruckweshay|ruck, arxcruz, ^^12:54
weshay|ruckah dang.. look at that12:55
weshay|ruckI thought we had guaranteed python3 on all upstream / rdo nodes12:55
rlandychandankumar++12:56
ysandeep|ruckweshay|ruck, seems like some corruption12:56
ysandeep|ruck~~~12:56
ysandeep|ruck[zuul@standalone ~]$ rpm -V python3-pip12:56
ysandeep|ruckmissing /usr/bin/pip312:56
rlandywill check it out12:56
rlandyweshay|ruck: ack12:56
weshay|ruckysandeep|ruck, ah for f's sake12:56
rlandybhagyashris; hey - you around to run scrum today?12:57
weshay|ruckwtf12:57
weshay|ruckysandeep|ruck, ideally we can revert that patch12:57
weshay|ruckand get to the root cause of the rpm issue12:57
ysandeep|rucksshnaidm, fyi.. we have a green run with depends on ^^ this patch - periodic-tripleo-ci-centos-7-standalone-stein https://review.rdoproject.org/zuul/build/8f71a447bbc544b9a11cdf79e1f59bcc : SUCCESS in 1h 15m 20s12:59
sshnaidmysandeep|ruck, ack, great12:59
bhagyashrisrlandy, yes13:00
ysandeep|ruckweshay|ruck, yes.. I am trying to get the root cause13:00
chandankumarrlandy: marios sshnaidm https://imgur.com/c3duIuB more colorful13:02
weshay|ruckysandeep|ruck++13:02
marioshttps://review.opendev.org/q/topic:tripleo-ci-reduce13:03
mariossshnaidm: weshay|ruck: rlandy: chandankumar: zbr: wdyt folks https://review.opendev.org/c/openstack/tripleo-ci/+/776429 please add to reviews list when you next have time thanks13:15
rlandymarios: ack - post 8-4 move13:15
*** ratailor has quit IRC13:16
sshnaidmmarios, tripleo-tox-molecule is actually quite redundant job and doesn't check something important, I don't think it's worth to add it to run in every patch13:17
mariossshnaidm: well we have it in our layout13:17
mariossshnaidm: but sure13:17
sshnaidmmarios, not in every template13:17
mariossshnaidm: i mean i was considering just adding the litners one13:17
sshnaidmmarios, given the fact it's very breakable, we add another instability13:18
mariossshnaidm: i can make it just depend on linters. this was actually my original use case because it happened on one of my patches13:18
mariossshnaidm: you mean linters?13:18
sshnaidmmarios, tripleo-tox-molecule13:18
mariosis breakable?13:18
mariossshnaidm: ah kk13:18
mariossshnaidm: sure so i can make it just depend on linters13:18
sshnaidmmarios, do linters check anything there?13:18
mariossshnaidm: yeah13:19
sshnaidmmarios, what exactly?13:19
mariossshnaidm: eg end of line or yaml etc i guess?13:19
sshnaidmidk13:20
sshnaidmmarios, but you add it to general templates, that run on all tripleo project13:20
sshnaidmis this the idea?13:20
mariossshnaidm: well i thikn openstack-tox-linters is already in most places13:20
mariossshnaidm: but yeah13:20
sshnaidmmarios, I don't know if in all13:22
sshnaidmmarios, and it's not only linters, every repo may be different13:22
sshnaidmmarios, maybe better to do it in each repo like: https://github.com/openstack/tripleo-common/blob/master/zuul.d/layout.yaml#L22-L2713:22
mariossshnaidm: ok yeah i have wf-1 there so i can check13:22
mariossshnaidm: yeah tripleo-upgade also has an override13:22
sshnaidmmarios, for example in link you can see openstack-tox-py36 and others13:22
mariossshnaidm: but in general you agree to the idea then13:22
sshnaidmmarios, yeah13:23
mariossshnaidm: like tripleo-upgrade https://opendev.org/openstack/tripleo-upgrade/src/commit/0e1398a1bc73b6d3916891457114b701eacacfca/zuul.d/layout.yaml#L913:23
mariossshnaidm: ack ok cool13:23
mariossshnaidm: i'll look into the repos and fix up ..13:23
mariossshnaidm: wil also remove th etox-molecule from dependencies13:23
sshnaidmactually it's different in each repo and even in each branch13:24
sshnaidmlike tox jobs for various python versions13:25
mariossshnaidm: well, let's fix it where we can, and how we can in each case13:25
mariossshnaidm: i mean, we have lots more jobs running in master vs queens for example so ... we might just leave q alone13:25
sshnaidmyeah, agree13:26
sshnaidmwe can just make it in master only13:26
sshnaidmthere is a little chance linters will fail on a backport..13:26
*** jpena|lunch is now known as jpena13:31
sshnaidmjpena, can you please fill a ticker for vexxhost as mnaser asked yesterday to track the upstream issue?13:34
sshnaidms/ticker/ticket/13:34
jpenasshnaidm: yes, will do13:34
sshnaidmjpena, thanks13:34
jpenasshnaidm: do you have access to the vexxhost ticketing system, to add you in cc?13:34
weshay|ruck-02-18 13:55:08,630 206395 INFO     promoter Containers promote 'aggregate: 506ab006b10154bd16a3dd23f663f30d, commit: 526bb3e7cac37266def07e7ddeee85da46281ab2, distro: 81425d7de5e2d7d0fbbbb580279f04d24d099ab7, component: validation, timestamp: 1613621402' to current-tripleo: Attempting promotion13:56
weshay|ruckmark rober14:01
ysandeep|ruckweshay|ruck, rlandy downstream baremetal job broke on 15th after alan stopped rdo mirror as a part of decommissioning, we should use internal mirror instead.. this will fix jobs - https://code.engineering.redhat.com/gerrit/#/c/228319/14:03
rlandyysandeep|ruck: ack - merging that14:04
ysandeep|ruckrlandy, thanks!14:05
*** ysandeep|ruck is now known as ysandeep|afk14:11
*** ykarel_ has joined #oooq14:14
*** ykarel has quit IRC14:17
*** marios is now known as marios|call14:18
akahatweshay|ruck, ussuri is promoting14:26
*** ykarel_ has quit IRC14:27
weshay|ruckakahat, bhagyashris nice.. /me following http://10.0.148.74/promoter_logs/container-push/20210218-142559.log14:28
bhagyashrisweshay|ruck, ok14:29
weshay|ruckakahat, k.. the pull failed14:30
weshay|ruckakahat, which container failed?14:30
akahatweshay|ruck, are you sure there is tripleou namespace in rdo registry14:31
akahat?14:31
weshay|ruckakahat, let's check14:31
akahatweshay|ruck, pm you some links14:31
weshay|ruckakahat, https://registry.rdoproject.org:8443/oapi/v1/namespaces/tripleoussuri/imagestreamtags/14:32
akahatweshay|ruck, bhagyashris this is because of this constructor: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/config.py#L37414:35
akahatwe have different names for ussuri on different registry14:36
weshay|ruckakahat, so.. let me show you one thing14:36
chandankumarakahat: you can also set source and target namespace in config itslef14:36
weshay|ruckakahat, bhagyashris http://promoter.rdoproject.org/config/CentOS-8/ussuri.ini14:36
chandankumarinstead of this hardcode14:36
weshay|rucklook at the overrides for ussuri14:36
weshay|ruckakahat, see it?14:36
akahatweshay|ruck, yes.. let us try with it14:37
weshay|ruckok14:37
weshay|ruckakahat, you have typo14:37
weshay|ruckI'll fix14:38
weshay|ruckhold14:38
weshay|ruckbah.. I can't access akahat, wq14:38
akahatthat window got freeze. :D14:38
weshay|ruckcrazy14:38
weshay|ruckakahat, k.. go ahead14:39
weshay|ruckakahat, k.. go 4 it14:40
akahatweshay|ruck, okay14:40
bhagyashrisweshay|ruck, http://10.0.148.74/promoter_logs/container-push/20210218-144021.log akahat14:41
akahatweshay|ruck, bhagyashris great.. now it's pulling.14:41
weshay|ruck:)14:42
bhagyashrisweshay|ruck, we are taking dinner break14:42
weshay|ruckbhagyashris, akahat have dessert :)14:42
bhagyashrisand will be back after 1 and half hr14:42
* akahat will be back after some time.14:43
*** tosky has quit IRC14:51
*** tosky_ has joined #oooq14:51
*** tosky_ is now known as tosky14:51
*** ysandeep|afk is now known as ysandeep|ruck15:02
ysandeep|ruckweshay|ruck, rlandy i have a mtg invite in my calendar about feedback15:02
ysandeep|ruckweshay|ruck, rlandy only i have joined, is that meeting canceled?15:03
rlandyysandeep|ruck: we are still on the last call15:03
rlandywill be there in a few15:03
ysandeep|ruckack, thanks!15:04
*** marios|call is now known as marios15:28
weshay|ruckbhagyashris, akahat promotion died15:30
weshay|ruckI think we're missing access to the images.rdoproject.org15:34
ysandeep|ruckleaving for the day, rlandy good luck with 8.415:35
ysandeep|ruckarxcruz, cya tommorow15:35
rlandyysandeep|ruck: thank you15:35
*** udesale__ has joined #oooq15:35
*** ysandeep|ruck is now known as ysandeep|away15:35
weshay|ruckakahat, bhagyashris containers worked15:37
weshay|ruckakahat, bhagyashris https://hub.docker.com/u/tripleou15:38
*** rlandy is now known as rlandy|training15:39
*** udesale_ has quit IRC15:39
arxcruzysandeep|away: namaste15:40
weshay|ruckakahat, bhagyashris fyi.. http://paste.openstack.org/show/802790/15:43
weshay|rucktrying again w/ new ssh keys15:43
*** ykarel_ has joined #oooq15:48
bhagyashrisweshay|ruck, ack15:49
zbrhelper for ansible terminology https://paste.debian.net/1185977/ -- not official but coming from a core member.15:51
weshay|ruckbhagyashris, akahat we need to make a note.. that any new promoter server needs public key access to the image servers15:54
weshay|ruckbhagyashris, akahat e.g. http://images.rdoproject.org/15:54
bhagyashrisweshay|ruck, ack15:57
weshay|ruckbhagyashris, akahat bah.. I think it failed again in the qcow2 routines15:59
weshay|ruckbhagyashris, akahat key works.. something else is busted16:01
weshay|ruckbhagyashris, akahat https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/qcow_client.py#L4516:03
* weshay|ruck wonders where this is set..16:04
weshay|ruck    def __init__(self, server_conf):16:04
weshay|ruck        self._host = server_conf['host']16:04
weshay|ruck        self._user = os.environ.get("USER")16:04
weshay|ruck        self._client_type = server_conf['client']16:04
weshay|ruck        self._keypath = server_conf['keypath']16:04
weshay|ruck        self._client = os16:04
zbrIT HAPPENED: https://libraries.io/pypi/ansible/3.0.0 -- embrace for impact on CI :p16:04
weshay|ruckzbr, wait.. talk to me goose16:04
weshay|ruckzbr, meet.google.com/oiu-xuws-hpb16:05
zbrweshay|ruck: i am joking, i do not expect impact. joining16:05
*** udesale__ has quit IRC16:06
*** skramaja has quit IRC16:08
*** rlandy|training is now known as rlandy16:16
bhagyashrisweshay|ruck, from here it gets that value https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/config_environments/rdo/defaults.yaml#L1816:17
*** ykarel_ is now known as ykarel16:21
*** ykarel has quit IRC16:29
*** jmasud has joined #oooq16:30
*** jpodivin has quit IRC16:35
*** jmasud has quit IRC16:36
*** jpena is now known as jpena|off16:51
weshay|ruckbhagyashris, akahat rlandy I think we have a successful promotion :) http://10.0.148.74/promoter_logs/centos8_ussuri.log16:53
rlandyweshay|ruck; woohoo16:54
akahat\o/16:55
bhagyashris:)16:56
weshay|ruckbhagyashris, akahat let's sync tomorrow17:02
weshay|ruckbecause we have to figure out if we can shut down the promoter in rdo17:02
weshay|ruckk?17:03
bhagyashrisweshay|ruck, sure17:03
*** jlarriba has joined #oooq17:04
*** marios is now known as marios|out17:05
*** jmasud has joined #oooq17:06
akahatweshay|ruck, ack17:07
* bhagyashris out17:10
*** marios|out has quit IRC17:19
*** jmasud has quit IRC17:46
*** dtantsur is now known as dtantsur|afk18:00
*** rlandy is now known as rlandy|training18:01
weshay|ruckrlandy|training, bhagyashris akahat HA... we're promoting again :)18:08
rlandy|trainingweshay|ruck: review requested before merge ... https://code.engineering.redhat.com/gerrit/#/c/225518/18:10
weshay|rucklooks good18:13
rlandy|trainingugh - error - will correct18:13
rlandy|trainingweshay|ruck: ok - check jobs running now18:19
rlandy|trainingif that passes - we merge and hope18:19
rlandy|trainingnext run will tell us18:19
*** derekh has quit IRC18:24
*** jmasud has joined #oooq19:51
weshay|ruckhrm... promotion problem20:05
*** slaweq has quit IRC20:21
*** jmasud has quit IRC20:29
*** rlandy|training is now known as rlandy20:33
*** jmasud has joined #oooq20:36
rlandyproblem? how so?20:38
*** chem has quit IRC20:40
rlandyweshay|ruck: https://code.engineering.redhat.com/gerrit/228372 Start 16.2 pipeline early  for rhel 8.4   - so we can watch fallout now20:43
rlandywill revert20:43
*** tosky has quit IRC23:31

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!