Friday, 2020-02-21

« Thursday, 2020-02-20 Index Saturday, 2020-02-22 »
*** jmasud has quit IRC00:04
*** tosky has quit IRC00:09
*** jmasud has joined #oooq00:24
*** rfolco|doctor is now known as rfolco01:18
*** rfolco has quit IRC01:34
*** jmasud has quit IRC02:04
*** rlandy has quit IRC02:04
*** ykarel|away is now known as ykarel03:51
*** raukadah is now known as chandankumar04:11
*** jmasud has joined #oooq05:27
*** ratailor has joined #oooq05:30
*** jmasud has quit IRC05:33
*** marios|ruck has joined #oooq06:25
*** d0ugal has quit IRC06:44
*** d0ugal has joined #oooq06:44
*** jmasud has joined #oooq06:50
*** jfrancoa has joined #oooq06:52
*** beagles_pto has quit IRC07:31
marios|ruckzbr: mergy? https://review.opendev.org/#/c/708020/907:41
marios|ruckzbr: sorry didn't get to *11:16 <@zbr> marios|ruck: chandankumar arxcruz: please help me merge https://review.opendev.org/#/c/708613/ - i need it in order to07:41
marios|ruckzbr: so have that one instead ;)07:42
marios|ruckchandankumar: o/07:48
chandankumarmarios|ruck, \o07:48
marios|ruckchandankumar: can we catch up about that https://bugs.launchpad.net/tripleo/+bug/1864018/comments/6 when you have few mins? maybe quick call07:48
openstackLaunchpad bug 1864018 in tripleo "periodic master container build fails ImportError: No module named queue" [Critical,Triaged]07:48
chandankumarmarios|ruck, yes sure07:49
marios|ruckchandankumar: i am confused about the revert and what more is needed there. the job is still broken07:49
marios|ruckchandankumar: k in 10 mins work for you?07:49
chandankumarmarios|ruck, the job is still broken?07:49
marios|ruckchandankumar: well i see it as failure on https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-master07:49
marios|ruckchandankumar: but lets talk on phone in bit07:49
marios|ruckchandankumar: if you want later also works for me if you're busy now don't mean to interrupt you07:49
chandankumarperiodic-tripleo-centos-7-master-containers-build-push -> passed07:50
marios|ruckchandankumar: ah...07:50
marios|ruckchandankumar: ok07:50
marios|ruckchandankumar: i think i see the issue...07:50
marios|ruckchandankumar: well, not sure but there might be new issue now07:50
marios|rucknERROR: No matching distribution found for python-heatclient===2.0.007:50
marios|ruckchandankumar: its what i wrote in the bug description https://bugs.launchpad.net/tripleo/+bug/1864018 looks like it might be another bug...07:51
openstackLaunchpad bug 1864018 in tripleo "periodic master container build fails ImportError: No module named queue" [Critical,Triaged]07:51
*** ccamacho has joined #oooq07:53
ykarelmarios|ruck, heatclient issue still there?07:54
ykarelit should be already fixed07:54
marios|ruckykarel: i saw it once yesterday and once just now (from midnight run today)07:55
marios|ruckykarel: do you have something to point to please for fix?07:55
ykarelmarios|ruck, fix was https://review.opendev.org/#/c/708780/07:55
marios|ruckykarel: thanks checking07:55
marios|ruckykarel: ok will try a testproject to check it07:56
ykarelu seen when?07:56
ykarellink?07:56
ykarelfor failure07:56
marios|ruckhttps://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build-push/66f34b3/job-output.txt07:56
marios|ruckykarel: ^07:56
ykarellooking07:56
ykarelmarios|ruck, but this looks old then the patch merged07:57
ykarel2020-02-20 00:22:2107:57
ykarelmore than 24 hr ago07:58
marios|ruckykarel: ack yea was the first run for today at midnight so lets see at https://review.rdoproject.org/r/#/c/25080/ thanks07:58
marios|ruckchandankumar: ^^^07:58
marios|ruckchandankumar: we can skip the call then... if that is green it means both are fixed07:58
marios|ruckchandankumar: unless you wanted to still talk07:58
chandankumarmarios|ruck, nope07:58
chandankumarmarios|ruck, thanks!07:58
marios|ruckchandankumar: k thanks07:58
ykarelso last runs are green https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-master&job_name=periodic-tripleo-centos-7-master-containers-build-push07:59
marios|ruckykarel: ah that one was from 20th not 21st @ 09:56 < marios|ruck> https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build-push/66f34b3/job-output.txt08:00
ykarelyes08:01
* marios|ruck refresh zuul page08:01
marios|ruckykarel: cool so containers build done thanks guys08:01
ykarelmarios|ruck, so what's plan for upstream container builds08:03
ykarelcentos7 job08:03
marios|ruckchandankumar: so do we still need that then? I mean the job is green now. https://review.rdoproject.org/r/#/c/25114/ . or that doesn't affect job now cos we no longer use venv. however that change is needed for ... ?08:04
ykareli mean http://zuul.openstack.org/builds?job_name=tripleo-build-containers-centos-7 and other equivalent jobs08:04
chandankumarmarios|ruck, we need that patch, we will be receiving latest contents of kolla in container build08:04
marios|ruckykarel: not sure yet... we could do the same use packages or try something like that if needed https://review.opendev.org/708846 or we drop centos7 containers build and only use centos 8 \o/08:05
marios|ruckykarel: cos this isn't going to stop (py2 issues)08:06
ykarelmarios|ruck, u saw mwhahaha comment that i referenced yesterday?08:06
marios|ruckchandankumar: do you mean in check jobs?08:06
chandankumarmarios|ruck, nope in periodic jobs08:07
marios|ruckykarel: which one you mean https://bugs.launchpad.net/tripleo/+bug/1864018/comments/508:07
openstackLaunchpad bug 1864018 in tripleo "periodic master container build fails ImportError: No module named queue" [Critical,Triaged]08:07
chandankumarmarios|ruck, in periodic container build c7, packages will come from tripleo-ci-testing repo what is build by dlrn and will be used while installing kolla08:07
ykarelmarios|ruck, i meant comment in https://review.opendev.org/#/c/691316/, but comments 5 in lp is same08:08
*** bogdando has joined #oooq08:09
marios|ruckykarel: k yeah i mean we have to at some point and this is just costing us we have new issues often now for that and its only going to get worse. I guess we're already working to that but we need another sprint i think08:09
ykarelhmm the sooner centos8 get's ready, better for us08:10
marios|ruckchandankumar: so i thought the pin is what protects us so that the packages built by rdo will be 'compatible' and not bring in the new breaking change08:11
marios|ruckchandankumar: and then we switched to use packages in job.08:11
marios|ruckchandankumar: so still not clear why we want to remove that pin :/ sorry i am slow08:11
chandankumarmarios|ruck, call?08:11
marios|ruckchandankumar: yes :) please08:11
marios|ruckplugging headphones08:11
chandankumarmarios|ruck, pass me the link08:12
marios|ruckk going to get one08:12
marios|ruckhttps://meet.google.com/asq-qeye-bau chandankumar08:12
*** jmasud has quit IRC08:13
*** jmasud has joined #oooq08:15
chandankumarmarios|ruck, https://trunk.rdoproject.org/centos7-master-head/report.html08:16
*** tesseract has joined #oooq08:16
*** marios|ruck is now known as marios|ruck|call08:20
*** marios|ruck|call is now known as marios|ruck08:20
*** amoralej|off is now known as amoralej08:20
*** tosky has joined #oooq08:27
*** jpena|off is now known as jpena08:43
*** ykarel is now known as ykarel|lunch08:47
*** ykarel|lunch is now known as ykarel08:55
zbrpanda: morning!09:00
pandazbr: =_=09:01
pandazbr: 'morning09:01
pandazbr: good dawn to you.09:02
zbrwhat do we need to do to fix mo' on https://review.rdoproject.org/r/#/c/25121/ ?09:02
pandazbr: one thing that I realized yesterday, is that rdo-tox-molecule still uses docker from package09:03
pandazbr: I needed to set use_upstream: true in the pre-run playbook09:04
pandazbr: not sure if it solves anything, but we can try, maybe it's working bettere with systemd service files09:04
zbrclearly docker-ce has improved support09:05
zbrdo you have a change open?09:05
pandazbr: yes, but it's the big one with all the rest09:05
pandazbr: it didnt' solve completely the other problems I had09:06
zbrpanda: something is not right rdo-tox-molecule is already using centos-8 node, which means that it already uses docker-ce09:09
zbrafaik, that is the only docker on centos-809:09
pandazbr: uh ?09:09
pandazbr: I didn't know. Another thing I found weird yesterday, is that ansible is not availabe in cento8 .... if I try to install it says "no package"09:10
zbrstill using rpms?09:11
pandazbr: not sure, anyway, the next step is to fix that docker info > /dev/null in command: module09:12
zbryou should never done that test, let me explain why09:13
pandazbr: probably moving rdo-tox-molecule to centos8 already unblocked the problem with apache start09:13
zbrmolecule can use remote docker servers, so you can run it on machines where there is no "docker" cli installed, and have it work fine.09:13
zbrdoing a test like "docker info" would fail in these cases.09:14
zbrdocker may be usable, even if you do not have a docker cli.09:14
zbransible docker modules do not require docker presence, they only need docker python-module09:15
pandazbr: that task is there to try and understand if a normal user can use docker commands, and the add to docker group worked.09:16
pandazbr: I really wonder what it's taking 34 minutes to run.09:17
*** dpawlik has joined #oooq09:17
zbrthe add to docker group can be tricky, you need to re-login, pipelining can affect it09:17
zbras ansible can keep connections caches09:18
pandazbr: yes that's why in the middle of the pipeline I send a reset command to ansible09:18
zbrinterestingly for me, i did not had any issue on pabelanger instance09:18
zbrthis is what we have there: https://github.com/ansible-community/molecule/blob/master/tools/test-setup.sh#L1609:19
pandazbr: maybe tehy too are sending reset_connection somewhere int he middle09:19
*** holser has joined #oooq09:22
*** holser has quit IRC09:22
zbrwth is the "Mount external volume"... part about?09:23
*** derekh has joined #oooq09:23
pandazbr: originally this playbook was create to provision the promoter server from scratch09:23
pandazbr: comtainers images are reserved an entire partition, mounted to /var/lib/docker09:24
pandazbr: to tst that part, in test environment we don't have another partition to mount, so I create a small loop device09:24
pandazbr: so even with additional partition, we get coverage for that part09:25
*** ratailor has quit IRC09:25
pandawithout*09:25
zbrthat will designed to fail, i said DoD will not work.09:25
panda....09:25
zbri think i would waste my time trying to make it work with docker driver.09:25
pandawhat will not work ?09:25
pandawhy not ?09:25
pandait worked before09:25
pandait's working now09:26
pandain docker driver09:26
zbr  msg: 'Error while getting device information with parted script: ''/usr/sbin/parted -s -m /dev/loop0 -- unit ''KiB'' print'''09:26
pandaI developed it with molecule and docker driver, and it worked for a while09:27
pandaI notice after that the loop device was created on the host09:28
*** ratailor has joined #oooq09:28
pandazbr: where are you seeing that error ?09:28
zbrit may have worked in certain context only by chance, different os, docker, docker config would break it.09:28
zbri think that the only practical approach on this is to switch it to delegated09:29
pandazbr: well it needs to work only on a single configuration09:29
zbris broken on CI, so is not working even on that "single". but developers are supposed to be able to run molecule scenarios, so is not really "one config"09:30
zbrpanda: i have nothing against docker driver, is my fav one, but i know its limits.09:30
pandazbr: it worked for me, and in CI for a while09:31
zbrwhen you reach systemd you are in trouble, if you reach DoD you are in deep.09:31
pandaDoD ?09:31
pandadefinition of done ?09:31
zbrDoD = docker-on-docker09:31
pandadead on delivery ?09:31
pandadistribution of denials ?09:31
zbrsynonims09:31
pandadevil on deck ?09:32
pandadirector at the dock ?09:32
pandaI think docker was bind mounting /dev/ on the container, and it was working09:33
pandaanyway, let's use delegated and goodnight09:33
zbrhow many scenarios do i need to convert to delegated?09:35
amoralejmay i get some help on the tripleo-build-containers-centos-8 failure in https://review.opendev.org/#/c/707338/ ? it failed to install a package in ironic-base09:37
amoralejbut i can't find why09:37
amoralejaren't the individual container build logs stored in the log server for those jobs?09:37
marios|ruckamoralej: they should be in there https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_bc2/707338/4/check/tripleo-build-containers-centos-8/bc234e0/logs/buildah-builds/ but missing09:39
pandazbr: only the promoter I think, the get_hash can be left there.09:39
zbrtrying now09:40
pandazbr: o you know of any plan to move tox-27 and tox-36 jobs to centos8 ?09:40
marios|ruckamoralej: i also saw this earlier on another patch. 2 thinsgs... 1 not sure what is going on with the logging and 2. i suspect might be something like that https://bugs.launchpad.net/tripleo/+bug/1863246  but without the logs we can't confirm (i.e. during yum error with centos mirrors)09:40
openstackLaunchpad bug 1863246 in tripleo "(intermittent) gate buildah container builds master/train failing - infra related "Could not retrieve mirrorlist"" [Critical,Won't fix] - Assigned to Marios Andreou (marios-b)09:40
zbrobviously, in fact 36 must be c8, not sure about 27. but YES09:40
amoralejok, i'll recheck09:40
amoralejit's non-voting but i don't want to break tripleo c8 jobs because of this09:41
zbrpanda: in how many places we use testinfra as verifier in ci-config repo?09:46
pandazbr: not familiar with testinfra09:48
zbraha, so is there only because it used to be default in the template09:48
zbris cool, but I also happen to prefer ansible, is much easier to understand for people to write test using ansible.09:48
zbrbut once you go-pro, you probably want to use it.09:49
*** holser has joined #oooq09:51
zbrpanda: get_hash is broken too, https://logserver.rdoproject.org/21/25121/3/check/rdo-tox-molecule/71afda3/tox/reports.html09:59
zbrtbh, i do not understand why you do not grab the url on localhost directly.10:00
zbrthat warning may explain the problem10:01
pandazbr: rfolco was working on it, I got involved only yesterday and only partially10:01
pandazbr: I'll work ith him to make it work10:01
zbrcool. better if we fix these in incremental steps.10:02
zbrdo you have 5mins? i want to show you something cool about newer molecule10:03
zbrhttps://meet.google.com/bsr-eitx-tqt  others welcome.10:04
pandazbr: if it's not a plugin for pycharm I'm not interested10:04
zbri stopped using pycharm more than an year ago, vscode python support went so good that made pycharm only a fat cow.10:05
*** jbadiapa has joined #oooq10:31
zbrpanda: btw, you should start using verify.yml to testing outcomes instead of playbook/converge ones.10:40
pandazbr: you mean  converge to call roles and verify.yml to check that the output/artifacts are sane ?10:43
zbryeah10:49
zbrconverge is supposed to only call role10:49
zbrverification of outcomes, should be in verify10:50
zbrthis ease testing, as you can run verify again, w/o running role again.10:50
zbrfrom CI point, there is no much difference.10:51
zbralso, coverage is supposed to be idempotent, verify does not.10:51
pandazbr: why would I want to run just verify ? I the first attemp failed, it's probable that I had to change something on the role, and I have to call converge again10:52
zbrpanda: when developing? failed first time, you change something, run again,....10:53
zbrit can fail due to broken code in verify part, w/o anything needing changes in role inclusion.10:53
pandazbr: but not just verify, I need to redeploy, maybe After I'm sure the deployment is correct and I just want to work on the expectations10:53
zbrthat is why you have destroy or the full "test" sequence.10:54
zbrCI uses "test", but when developing a role, you don't want to re-run each step when you resolve a warning in one task.10:55
pandamarios|ruck: ok to merge https://review.rdoproject.org/r/24678 ?11:07
*** ykarel is now known as ykarel|afk11:16
chandankumararxcruz, soniya29 do we have anything to disccuss?11:20
chandankumararxcruz, soniya29 kopecmartin agenda added here https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ?view11:22
marios|ruckpanda: sorry was getting food... looking11:31
marios|ruckpanda: yeah no objections here11:32
marios|ruckpanda: you want me to +A?11:32
pandamarios|ruck: ok, potentially disrupive change coming up on a Friday then :)11:32
marios|ruckpanda: ack revert is but a click away ;)11:33
pandamarios|ruck: yes, please +A, so if something breaks I can tell you I tried to sop but you wouldn't listen11:34
marios|ruckpanda: i don't care i'm going to merge it anyway YOLO !11:35
marios|rucklike that?11:35
pandayeeep11:35
* marios|ruck runs to pub11:35
pandaleeerooooy jenkiiiins11:36
arxcruzchandankumar: i have nothing to say without my lawyer11:38
pandayour honor, my client choose to invoke the fifth amendment11:39
*** ykarel|afk is now known as ykarel11:42
*** ratailor has quit IRC11:51
zbrpanda: marios|ruck: do you know to which extend we can move jobs to vexx?12:02
marios|ruckamoralej: fyi https://bugs.launchpad.net/tripleo/+bug/1864108  and for the missing logs that https://review.opendev.org/#/c/709039/212:03
openstackLaunchpad bug 1864108 in tripleo "centos-8 container builds fail in promotion on base container" [Critical,Triaged]12:03
marios|ruckzbr: don't know what the plan with jobs is ... i recall weshay|ruck saying we are waiting for some more tests before moving12:03
zbrmarios|ruck: i guess moving py37/36 from ci-config to rdo-centos-8-vexxhost would be good/bad idea?12:04
marios|ruckzbr: i guess all of them moving is a good idea but i don't know if that is a good idea _today_12:05
*** dtantsur|afk is now known as dtantsur12:06
amoralejmarios|ruck, what repos are you using in that job?12:20
amoraleji man, the container builds12:20
marios|ruckamoralej: well for kolla/tripleoclient we use install from git source if that is what you mean12:22
amoralejmarios|ruck, https://logserver.rdoproject.org/64/25064/6/check/periodic-tripleo-centos-8-master-containers-build-push/3550d66/logs/buildah-builds/kolla-zyg2howw/docker/base/base-build.log is missing some repos12:22
amoralejdelorean and deps12:22
marios|ruckamoralej: yeah its in the bug ( Error: Unable to find a match: crudini dumb-init openstack-tripleo-common-container-base puppet-tripleo12:23
marios|ruckamoralej: not sure what was tagged as 'current-tripleo' for centos8, checking12:24
marios|ruckamoralej: (cos obviously we don't have the promotion pipeline yet)12:24
ykarelamoralej, marios|ruck see latest patch12:25
ykareli revised, that fixes the issue12:25
ykarelissue were repos were not set in that job12:26
ykarels/were/was12:26
ykarelhttps://review.rdoproject.org/r/#/c/25064/12:26
marios|ruckykarel: looking12:26
ykareli had to do like that as it needed fix in config project12:28
marios|ruckykarel: sorry not following. which one is the fix? https://review.opendev.org/#/c/709034/ DNM and https://review.opendev.org/#/c/708967/ for logs12:28
ykarelmarios|ruck, since fix is needed in config project, i had to push https://review.opendev.org/#/c/709034/ for testing12:29
marios|ruckykarel: ok where is the config project fix then i don't see it12:29
ykarelmarios|ruck, fix is just need to remove https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleoci.yaml#L39412:29
marios|ruckykarel: but that is for periodic only ^^ shouldn't affect the check.. unless we have smae in the tripleo-ci definition12:30
ykarelmarios|ruck, yes only periodic has issue12:30
ykarelcheck is fine12:30
marios|ruckykarel: not sure i've seen same sec12:31
marios|ruck    * https://4d6da1b390c8df1f8e50-bd895733351047fc01b6164e721bda2c.ssl.cf5.rackcdn.com/708893/1/check/tripleo-build-containers-centos-8/3af86c3/logs/build.log12:31
marios|ruck    https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_bc2/707338/4/check/tripleo-build-containers-centos-8/bc234e0/logs/build.log12:31
marios|ruckadding to bug12:31
marios|ruckykarel: so you posted config change already? lets add it to the bug then12:32
marios|ruckykarel: or lets post it12:32
ykarelmarios|ruck, nope i didn't12:32
ykarelu can post, or if not i can post12:32
chandankumarweshay|ruck, we are waiting for you12:33
chandankumarsoniya29, meeting time12:33
marios|ruckykarel: ok fine i can do it thanks we will need for both periodic & check12:34
ykarelmarios|ruck, but check should be fine i think12:34
ykarellet me cross check then12:34
marios|ruckykarel: do you want to add comment on the bug ? (pasting some of the chat from here)12:34
marios|ruckykarel: see https://bugs.launchpad.net/tripleo/+bug/1864108/comments/712:34
openstackLaunchpad bug 1864108 in tripleo "centos-8 container builds fail in promotion on base container" [Critical,Triaged]12:34
arxcruzsoniya29: around?12:35
ykarelmarios|ruck, in check issue is different12:35
ykarelthere 121 images build fine12:35
ykarelbut in periodic 012:35
marios|ruckykarel: hmm ok... nice and confusing.12:37
marios|rucki hate container build jobs right now12:38
marios|ruckykarel: seems pretty similar trace but until we get logs we won't know12:38
marios|ruckykarel: i mean with https://review.opendev.org/#/c/709039/212:39
ykarelmarios|ruck, ack yes it fails same for each container build12:41
ykarelwithout logs it would be hard12:41
*** jpena is now known as jpena|lunch12:42
marios|ruckykarel: ok i'll post for the periodic to start with12:42
ykarelmarios|ruck, ack12:42
*** ysandeep is now known as ysandeep|afk12:52
*** rfolco has joined #oooq12:53
chandankumararxcruz, https://review.rdoproject.org/r/#/c/24922/12:55
*** rlandy has joined #oooq12:59
rlandychandankumar: weshay|ruck: panda: sync meeting13:01
rlandyarxcruz: zbr: ^^ if you want13:02
rlandycentos0813:02
chandankumarrlandy, coming in few mins13:02
migiweshay|ruck: there is no component mtg today? I am on pto but wanted to attend it and saw it dissapeared from cal13:03
*** Goneri has joined #oooq13:09
*** ykarel is now known as ykarel|afk13:22
chandankumarrlandy, rfolco this one https://bugs.launchpad.net/tripleo/+bug/1864108/13:24
openstackLaunchpad bug 1864108 in tripleo "centos-8 container builds fail in promotion on base container" [Critical,Triaged]13:24
*** ysandeep|afk is now known as ysandeep13:27
ysandeepweshay|ruck, hey o/ our 1:113:30
*** dpawlik has quit IRC13:40
rlandymarios|ruck: hi ... can https://review.rdoproject.org/r/#/c/25182/ merge?13:41
*** ysandeep is now known as ysandeep|away13:41
rfolcozbr, can you clone https://github.com/rdo-infra/ci-config and try running molecule locally on roles/get_hash  ?13:41
rfolcozbr, and tell me the steps you do13:41
zbrcan you open a tmate?13:41
rfolcozbr, sure, pvt boy13:42
marios|ruckrlandy: yeah i think so. ykarel|afk had a successful test see https://bugs.launchpad.net/tripleo/+bug/1864108/comments/813:42
openstackLaunchpad bug 1864108 in tripleo "centos-8 container builds fail in promotion on base container" [Critical,Triaged]13:42
*** jpena|lunch is now known as jpena13:42
zbrrfolco: you want to keep meet open, or no need?13:42
marios|ruckthanks rlandy13:43
rfolcozbr, yes better, give one sec13:43
zbri do have several changes related to cents8 to make to roles zuul-jobs, who can help me reviewing them? (before I ping infra)13:43
weshay|ruckrlandy, holy crap :)13:44
zbrit could prove very useful to have them confirmed by someone else before I start poking infra13:44
rlandyweshay|ruck: what now???13:44
weshay|ruckrlandy, you have containers :) https://review.rdoproject.org/r/#/c/25064/13:45
rlandyweshay|ruck: yeah - I swa the job went green13:45
rlandyshocker :)13:45
rlandyweshay|ruck: chandankumar also made good edits to release files13:46
rlandywe may be closer here13:46
weshay|ruckchandankumar, ykarel|afk so it's https://review.opendev.org/#/c/709034/ ?13:46
weshay|ruckykarel|afk, chandankumar https://review.opendev.org/#/c/708956/13:47
marios|ruckweshay|ruck: yeah but that was just testing by ykarel|afk rlandy just merged that one https://review.rdoproject.org/r/#/c/25182/ which should fix13:47
*** dpawlik has joined #oooq13:48
weshay|ruckmarios|ruck, ok.. thanks.. that's what I was wondering13:48
marios|ruckweshay|ruck: noted in comments at https://bugs.launchpad.net/tripleo/+bug/186410813:48
openstackLaunchpad bug 1864108 in tripleo "centos-8 container builds fail in promotion on base container" [Critical,Triaged]13:48
marios|ruckweshay|ruck: now, not sure if the check issue we're seeing for centos8 is the same ... may not be. but we need EmilienM patch for logging to make it clearer13:49
weshay|ruckrlandy, did you check where they pushed and w/ what id?13:49
marios|ruckweshay|ruck: https://bugs.launchpad.net/tripleo/+bug/1864108/comments/713:49
rlandyweshay|ruck: trying to decipher all that now13:50
rlandya1bd02d30481c514e0a16c6d6ce5ea458f7d1001_750cce0213:50
rlandy^^ wrong hash13:50
weshay|ruckmarios|ruck, will get there.. need a few13:51
weshay|ruckrlandy, rfolco would be nice to update this to log something13:51
weshay|ruck2020-02-21 04:28:12.888624 | POST-RUN START: [trusted : review.rdoproject.org/config/playbooks/tripleo-ci-periodic-base/containers-push-podman-login-push.yaml@master]13:51
* weshay|ruck gets coffee13:51
marios|ruckweshay|ruck: ack np13:51
chandankumarweshay|ruck, no idea13:51
chandankumarweshay|ruck, there is a config already which can be overridden13:52
rlandy2020-02-21 04:28:24.470984 | primary | + dlrnapi --url https://trunk.rdoproject.org/api-centos8-master-uc report-result --agg-hash d15d61afc27948b715fc611aa20636dd --job-id periodic-tripleo-centos-8-master-containers-build-push-test --info-url https://logserver.rdoproject.org/64/25064/8/check/periodic-tripleo-centos-8-master-containers-build-push-test/106d015 --timestamp 1582259304 --success True13:52
rlandydlrn report is correct13:52
rlandyugh - no idea what happens here :)13:54
rlandyno push logs13:54
weshay|ruckrlandy, rfolco build_id d15d61afc27948b715fc611aa20636dd13:56
weshay|ruckhttps://trunk.registry.rdoproject.org:8443/oapi/v1/namespaces/tripleomaster/imagestreamtags/13:56
*** Goneri has quit IRC13:57
*** rascasoft has joined #oooq13:57
weshay|ruckmarios|ruck++13:59
weshay|ruckthanks13:59
marios|ruckweshay|ruck: np but really can't take credit for it as usual ykarel|afk helped us out https://bugs.launchpad.net/tripleo/+bug/1864108/comments/814:00
openstackLaunchpad bug 1864108 in tripleo "centos-8 container builds fail in promotion on base container" [Critical,Triaged]14:00
rlandyhttps://logserver.rdoproject.org/64/25064/8/check/periodic-tripleo-centos-8-master-containers-build-push-test/106d015/logs/build.log14:00
rlandy^^ what is that hash??14:00
rlandya1bd02d30481c514e0a16c6d6ce5ea458f7d1001_750cce0214:00
*** dpawlik has quit IRC14:00
rlandywhere it pulled from?14:00
* chandankumar headed home14:03
marios|ruckykarel|afk: revote please i answered there https://review.opendev.org/#/c/709069/114:03
marios|ruckweshay|ruck: rlandy: rfolco: please votes on that one https://review.opendev.org/#/c/709069/1 for centos7 containers check https://bugs.launchpad.net/tripleo/+bug/1864018/comments/1014:03
openstackLaunchpad bug 1864018 in tripleo "periodic master container build fails ImportError: No module named queue" [Critical,Triaged]14:03
rlandymarios|ruck: ykarel|afk has a -1 on that14:04
marios|ruckrlandy: yah replied i think he is mistaken https://review.opendev.org/#/c/709069/1/zuul.d/build-containers.yaml14:04
marios|ruckrlandy: s/mistaken/misread14:05
rlandyrfolco: where does a1bd02d30481c514e0a16c6d6ce5ea458f7d1001_750cce02 come from?14:12
rlandytag=a1bd02d30481c514e0a16c6d6ce5ea458f7d1001_750cce0214:14
rlandyah - tag is wrong?14:14
marios|ruckpanda: hmmm... not sure if that means we're safe yet 2020-02-21 13:44:33,339 8823 INFO     promoter Promoting the container images for dlrn hash c71cab7fcceaa4d979a5104a31605e74689e64a1 on train to current-tripleo   (that merged14:19
marios|ruck1:48 PM14:19
marios|ruckpanda: that being https://review.rdoproject.org/r/#/c/24678/14:19
marios|ruckpanda: not sure of the timezones...14:19
marios|ruckpanda: i think it should be running ^^ already.. somethign we can check in logs?14:20
marios|ruckah there is 'using legacy' foo in the loggin right?14:20
pandamarios|ruck: this is a machine, please leave a message after the sound.14:20
marios|ruckpanda: 1 hawaian with extra fetta cheese please14:20
marios|ruckextra large stuffed crust14:20
pandamarios|ruck: yeah, but we are using the new workflow by default14:20
pandamarios|ruck: and I left the original logging as close as possible to the old one14:21
pandamarios|ruck: not easy to undestand if the promotion happened with the patch or not.14:21
marios|ruckpanda: ok14:21
marios|ruckpanda: what about my pizza tho14:21
pandamarios|ruck: your order is being process, thank you choosing panda pizza!14:23
marios|ruck\o/14:23
*** amoralej is now known as amoralej|lunch14:23
weshay|ruckrfolco, rlandy so who's on the contianer push fun?14:32
rlandyweshay|ruck: looks like https://logserver.rdoproject.org/64/25064/8/check/periodic-tripleo-centos-8-master-containers-build-push-test/106d015/job-output.txt14:33
rlandybut I the tag is off14:33
rlandyhttps://logserver.rdoproject.org/64/25064/8/check/periodic-tripleo-centos-8-master-containers-build-push-test/106d015/conf/kolla-build.conf14:34
rlandywould be really helpful if this logged ANYTHING14:35
rlandyweshay|ruck: rfolco: meet to sort this?14:35
rfolconeed a few to finish w/ zbr14:36
rfolconeed to reboot14:44
*** rfolco has quit IRC14:47
*** TrevorV has joined #oooq14:49
*** rfolco has joined #oooq14:54
rfolcozbr, it works :)14:57
rfolcozbr, thanks a TON14:57
zbrhuh, happy it was just that.14:57
zbrso the turn it off and on... rules14:57
rlandyha - ok - found where it's pulling the tag14:58
marios|ruckweshay|ruck: fyi did you see that one already https://review.opendev.org/#/c/708893/114:59
rfolcorlandy, did you want to sync / meet ?14:59
rlandyrfolco: yeah - thin k found the error14:59
rlandyrfolco: https://meet.google.com/mdy-vycw-mzx15:00
chandankumarweshay|ruck, rlandy https://logserver.rdoproject.org/97/24997/8/check/tripleo-centos-8-buildimage-overcloud-full/966d7d7/ one successful15:03
chandankumarneed to look into ironic python agent15:04
rlandyhttps://logserver.rdoproject.org/64/25064/8/check/periodic-tripleo-centos-8-master-containers-build-push-test/106d015/etc/yum.repos.d/delorean.repo15:11
rlandyhttps://logserver.rdoproject.org/64/25064/8/check/periodic-tripleo-centos-8-master-containers-build-push-test/106d015/job-output.txt15:12
*** amoralej|lunch is now known as amoralej15:19
rfolcorlandy, I think I know whats going on15:29
rfolcopodman_push is for rhel8 only15:29
rfolcohttps://github.com/rdo-infra/review.rdoproject.org-config/blob/8cf4df16ec123264ab827a645585247b0f3da415/playbooks/tripleo-ci-periodic-base/containers-push-podman-login-push.yaml#L715:29
rfolcowe need to add centos8 there ^15:29
rfolcosending patch rlandy weshay|ruck ^15:29
rlandyrfolco: more patches needed there than that15:30
rlandybut sure15:30
rfolcoof course, but this one will enable push for centos815:30
mjturekweshay|ruck rfolco: so it sounds like building on centos7 might not even be an option for the ppc64le issue? http://paste.openstack.org/show/789815/15:32
weshay|ruckmjturek, did you send me a summary of your fun world?15:33
mjturekweshay|ruck: yep yesterday I pinged you with an etherpad link15:33
mjtureklet me get it15:33
*** ykarel|afk is now known as ykarel15:34
mjturekweshay|ruck: https://etherpad.openstack.org/p/ppc64le-container-build-situation15:34
*** Goneri has joined #oooq15:36
weshay|ruckmjturek, passing to my peers now15:36
*** jpena is now known as jpena|off15:39
*** dpawlik has joined #oooq15:41
*** jpena|off is now known as jpena15:42
dpawlikchandankumar, can I send an email about ended migration of image host?15:42
chandankumardpawlik, yes15:42
chandankumarmarios|ruck, ^^ any issues with image server ?15:43
chandankumarrlandy, weshay|ruck https://review.opendev.org/#/q/topic:c8imagebuild+(status:open+OR+status:merged) please review when free15:43
marios|ruckchandankumar: not aware of any15:44
dpawlikchandankumar, yeasterday we merge one thing, so if all if fine, I will send15:44
chandankumarmarios|ruck, thanks, dpawlik we are good15:44
chandankumardpawlik, thanks :-)15:44
dpawlikchandankumar, marios|ruck cool. Nice to work with you guys15:45
marios|ruckdpawlik: o/15:45
marios|ruckdpawlik: i just did some code review ;) but thanks :)15:45
rfolcoweshay|ruck, centos8 need/must/have to/should to push w/ podman?15:48
rlandychandankumar: pls review ...  https://review.rdoproject.org/r/25232 ... we need these vars defined for centos 8 https://github.com/openstack/tripleo-ansible/blob/master/tripleo_ansible/roles/tripleo_podman/vars/redhat.yml15:48
rlandyrfolco: ^^15:49
rlandyif you put you patch in we can rebase15:50
weshay|ruckrfolco, it aint docker bro15:50
rfolcoweshay|ruck, ok so the change must go on rdo-infra/config15:50
rfolcojust like rhel815:50
weshay|ruckk.. put the patch up and we'll merge + test15:51
rfolcook ok just to confirm15:51
mjturekokay thanks weshay|ruck15:53
rfolcoweshay|ruck, rlandy https://review.rdoproject.org/r/25233 Enable container push to CentOS815:55
weshay|ruckrfolco, wait for merge.. and retrigger the test project15:56
rfolcok15:56
rfolcoweshay|ruck, what about namespace? need to change it15:56
rfolcobefore retriggering15:56
weshay|ruckrfolco, no.. not before retriggering15:56
weshay|ruckbut ya.. we need to create one first15:57
rfolcoif you run the job w/ the patch merged its gonna push to tripleomaster15:57
rfolconeed some logic here weshay|ruck https://github.com/openstack/tripleo-ci/blob/4fdcb893565029ec30c0e09b51bc908c01499c14/roles/build-containers/templates/kolla-build.conf.j2#L1515:58
rfolcoworking on it15:58
weshay|ruckrfolco, ya.. pushing to tripleomaster for right now is fine brotha15:59
weshay|ruckit's just a housekeeping issue.. not technical15:59
rlandyhttps://review.rdoproject.org/r/#/c/25232/15:59
rlandy^^ will need this as well15:59
rlandyweshay|ruck: rfolco: ^^15:59
rlandyand patch to fix tagging - coming up15:59
rlandyhold on job rerun please15:59
weshay|ruckactually rfolco continue to use tripleomaster16:00
weshay|ruckrlandy, panda ^16:00
weshay|ruckbecause centos-7 container builds will be shutdown asap16:01
rfolcorlandy, we should rebase our chain, as all changes in config16:01
rfolcorlandy, feel free to rebase yours on mine, or vice-versa16:01
rlandyI'll rebase on yours since weshay|ruck already w+'ed yours16:01
chandankumarweshay|ruck, https://review.rdoproject.org/r/#/c/25232/16:02
rfolcoah ok16:02
weshay|ruckrfolco, rlandy fyi.. we're turning off centos-7 container check jobs in ussuri upstream16:09
chandankumarrlandy, weshay|ruck I would like to present https://review.rdoproject.org/r/#/c/24997/ tripleo-centos-8-buildimage-overcloud-fullSUCCESS in 21m 20s tripleo-centos-8-buildimage-ironic-python-agentSUCCESS in 20m 55s (non-voting)16:10
rlandychandankumar+++++16:10
rlandyrock on!!16:10
chandankumarsince we donot care about tripleo-centos-8-buildimage-overcloud-hardened-full i am not going to look16:10
marios|ruckweshay|ruck: fyi scen7 failed last 2 runs but 2 different ways so not filing bug yet ... links on etherpad (and posted testproject to see which one of those reproduces.. maybe we get a third? )16:11
*** rfolco is now known as rfolco|eats16:11
chandankumarweshay|ruck, rlandy since we have containers and images, can we plug fs01 jobs?16:13
weshay|ruckchandankumar, we have centos-8 images?16:13
chandankumarweshay|ruck, yes16:13
weshay|ruckchandankumar, we don't have containers yet16:13
weshay|ruckwow16:13
weshay|ruckchandankumar++16:13
chandankumarweshay|ruck, https://review.rdoproject.org/r/#/c/24997/ it has pushed already16:13
weshay|ruckoh soooky sooky now16:14
weshay|ruckchandankumar, k.. hold for containers16:14
weshay|ruckor help review16:14
weshay|ruckchandankumar, nothing is pushed to the rdo registry yet16:14
rlandyhttps://review.opendev.org/709180 Add option to use the hash_info file to get container tags16:14
weshay|rucklook at that beautiful work16:14
weshay|ruckchandankumar++16:14
rlandyweshay|ruck: rfolco|eats: ^^ thoughts?16:14
weshay|ruckrlandy, I'm a little out of context tbh w/ regards to the hash info16:15
weshay|ruckwe DO need the aggregated hash.. as the tag id16:15
rlandyweshay|ruck: from meeting this morning ...16:15
*** chem has quit IRC16:16
rlandycurrent code read the delorean repo16:16
rlandywhich results in it using the last component full_hash16:16
rlandyI can make that safer by only using hash_info if it's centos816:16
weshay|ruckk.. rfolco|eats dang it16:17
weshay|ruckchandankumar, review for rlandy16:17
rlandysec - let me only do that for centos 8 so limit damage16:18
rfolco|eatsthats what we talked about this morning16:19
chandankumarweshay|ruck, which one?16:22
weshay|ruckhttps://review.opendev.org/70918016:23
weshay|ruckchandankumar, ^16:23
pandapromoter integration tests are passing \o/ single-legacy single-new and integration-new !16:24
chandankumarrlandy, do we have test project change to verify that?16:27
chandankumarhash info16:27
rlandyhttps://review.opendev.org/#/c/709180/1..2/roles/build-containers/tasks/main.yaml16:27
rlandy^^ ok - so I don;t know if that is less or more safe :)16:28
* marios|ruck had enough of logs for one week ...16:28
marios|ruckdon't talk to me about logs or jobs or containers until monday or i will hit you16:28
rlandychandankumar: I did a local test16:28
rlandybut I will put in a testproject change16:29
pandamarios|ruck: would you like to log a complaint ?16:29
rlandysec16:29
rlandyha ha16:29
marios|ruckpanda: where's my pizza?!16:29
pandamarios|ruck: it's on its way16:29
* marios|ruck holds breath16:29
pandaliterally, she escaped with my birger16:29
pandac'est l'amour.16:29
pandaburgher16:29
weshay|ruckrlandy, will depends-on work w/ your patch and the test project?16:30
rlandyweshay|ruck: it should for non-periodic16:30
rlandybecause no push/ no dlrn/no protected repos16:30
ykarelrfolco|eats, typo https://review.rdoproject.org/r/#/c/25233/16:32
ykareli posted -W16:32
ykarelto avoid merge16:32
rlandyhttp://zuul.openstack.org/status runs container build tests16:32
ykareldeug16:32
ykarelinstead of debug16:32
rlandytripleo-build-containers-centos-8-master16:33
rlandy(non-voting)16:33
rfolco|eatssorry16:33
rlandyso short of that ... not sure what testproject will do16:33
*** rfolco|eats is now known as rfolco16:33
pandaykarel: because it's a small bug, so it's a ug.16:33
marios|ruckykarel: to be fair to rfolco it was only 4 times, easy mistake to make16:33
ykarel:)16:33
rfolcomarios|ruck, thank you for defending me16:33
marios|ruckrfolco: anytime mate16:34
rfolcoI fell so ...16:34
marios|ruck:)16:34
* rlandy tries16:35
rfolcorlandy, weshay|ruck ykarel marios|ruck panda and everyone patch updated https://review.rdoproject.org/r/#/c/25233/16:36
ykarelack16:37
rfolcorlandy, you retriggering testproject ?16:37
weshay|ruckrfolco, has to merge16:38
weshay|ruckhold16:38
rlandysec- fixing patch16:38
rfolcoI know... just want to know who is doing it after the patches get merged16:38
rlandyrfolco: if https://review.opendev.org/#/c/709180/3/roles/build-containers/tasks/main.yaml works in check, we can try testproject it - with push set to false16:45
rlandyrfolco: ^^ pls review in the mean time16:46
weshay|ruckmarios|ruck, http://tripleo-cockpit.usersys.redhat.com/d/9DmvErfZz/cockpit?orgId=116:49
marios|ruckweshay|ruck: which bit?16:50
weshay|ruckmarios|ruck, upstream cockpit is dying16:50
weshay|ruckwill be resurrected in vexhost16:50
weshay|ruckmarios|ruck, not sure how to proceed w/ https://review.opendev.org/#/c/708264/16:50
marios|ruckweshay|ruck: yeah i am lost there i looked earlier but it seems same... i can spend some time digging at it on monday ^16:51
marios|ruckweshay|ruck: i mean the new depends-on didn't help16:51
weshay|ruckchanged the failing module16:51
weshay|ruckI think16:51
marios|ruckweshay|ruck: no same i think 2020-02-20 20:23:53.079667 | primary | ImportError: cannot import name suppress16:52
marios|ruckhttps://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_a45/708264/9/check/tripleo-ci-centos-7-containers-multinode-queens/a45d17e/job-output.txt16:52
pandazbr: on monday I'll try to move py-36 to cento8, I need it now. Also I don't understand why I'm unable to make tox-py27 use ansible2.8 and a play is failing on an unsupported docker_image parameter16:52
rlandyhttps://review.rdoproject.org/zuul/stream/d14a021a0e194924920053a236fdce51?logfile=console.log16:55
rlandyugh - even the check jobs need the md516:55
rlandythey will tag with the wrong hash16:55
rlandyand there is no hash_info file in check16:55
rlandyrfolco: weshay|ruck: ^^ - we need another switch case16:57
rlandythe check jobs on centos 08 are also wrong16:58
rlandyhttps://logserver.rdoproject.org/95/708295/1/openstack-check/tripleo-build-containers-centos-8-master/e15090f/logs/build.log16:58
rlandysee ^^16:58
weshay|ruckrlandy, check job tagging is no big deal16:58
weshay|ruckthose containers don't go anywhere16:58
rlandyweshay|ruck: yeah - but it's still wrong16:58
weshay|ruckyes.. fix but lower priority is my point16:58
rlandyusing the hash_info is not an option in the case of check16:59
rlandywe can take the md5sum in the job16:59
* rlandy codes another patch16:59
marios|ruckweshay|ruck: need something? /me about to go17:00
*** tesseract has quit IRC17:00
weshay|ruckmarios|ruck, 0/ have a good weekend17:00
marios|ruckweshay|ruck: thanks17:01
marios|ruckweshay|ruck: you too17:01
marios|ruckhave a good weekend all17:01
*** bogdando has quit IRC17:01
chandankumarmarios|ruck, Have a good weekend, enjoy Mozito :-)17:01
pandahttps://review.rdoproject.org/r/24818 look at all the green on PS 31. Next station: docstrings, and fixing deps on tox-py jobs.17:01
marios|ruckchandankumar: thanks ;)17:02
*** marios|ruck is now known as marios|out17:02
chandankumarrfolco, weshay|ruck, rlandy do we want to discuss about release file tripleo include pkgs issue/question?17:08
rlandysure17:08
chandankumarwaiting for others17:09
*** rascasoft has quit IRC17:11
chandankumarykarel, I hope it answers your query on release file https://review.opendev.org/#/c/708968/17:12
ykarelchandankumar, reading17:12
ykarelchandankumar, then it's wrong as per my comment17:13
chandankumarykarel, please elaborate17:13
ykarelchandankumar, rdo_dlrn=`curl --silent ${NODEPOOL_RDO_PROXY}/{{ distro_ver }}-{{ release }}/{{ dlrn_hash_path_newest|default(dlrn_hash_tag_newest, true) }}/delorean.repo -S 2>>~/dlrn_repo_curl_errors.log | grep baseurl | cut -d= -f2`17:13
chandankumarrlandy, ^^17:14
*** marios|out has quit IRC17:15
weshay|rucksure.. chandankumar is there a patch up?17:15
weshay|rucksee it17:15
rlandysec17:15
ykarelchandankumar, run this will give hint, curl --silent https://trunk.rdoproject.org/centos8-master/promoted-components/delorean.repo -S | grep baseurl | cut -d= -f217:15
ykarelwhat /me trying to say there17:15
weshay|ruckheh17:15
weshay|ruckykarel++17:15
*** dtantsur is now known as dtantsur|afk17:16
chandankumarweshay|ruck, https://review.opendev.org/#/c/708968/6/config/release/tripleo-ci/CentOS-8/master.yml@6117:16
weshay|ruckya.. that needs to change17:16
weshay|ruckI think it is easier now actually17:16
chandankumarykarel, it is printing a list of hash with urls17:16
weshay|ruckchandankumar, ykarel rlandy I think we can just baseurl w/ https://trunk.rdoproject.org/centos8-master/current-tripleo/delorean.repo17:17
chandankumarykarel, may be we just need a repo file as it is17:17
ykarelchandankumar, yes multiple hashes17:17
weshay|ruckthat will ALWAYS be the current one17:17
ykarelweshay|ruck, yes u can use get_file17:17
ykareli gave an example there17:17
ykareland then replace trunk.rdo with proxy in repo cmd after17:18
weshay|ruckya.. we just need to account for the mirrors17:18
*** holser has quit IRC17:19
chandankumarykarel, you mean download the file and then rewrite the file with nodepool proxy?17:19
weshay|ruckw/  export RDO_DLRN_REPO=${rdo_dlrn/https:\/\/trunk.rdoproject.org/$NODEPOOL_RDO_PROXY}17:19
weshay|ruck  export TRIPLEO_DLRN_REPO=${tripleo_dlrn/https:\/\/trunk.rdoproject.org/$NODEPOOL_RDO_PROXY}17:19
ykarelchandankumar, yes17:19
weshay|ruckya.17:19
chandankumarykarel, may be constructing a new repo there then17:19
weshay|ruckor17:19
chandankumarwith nodepool proxy17:20
weshay|ruckwe code in each component17:20
weshay|rucknot saying that's better17:20
ykarelchandankumar, can use down_url: "https://trunk.rdoproject.org/centos8-master/current-tripleo/delorean.repo" with repo type: file17:20
chandankumarneed to try down_url17:20
weshay|ruckwe need to make the base url a var17:20
chandankumarykarel, will make the change there17:20
*** holser has joined #oooq17:20
weshay|ruckand sub appropriately17:21
ykarelchandankumar, ack17:21
ykareli commented in the review earlier, can use that as ref17:21
chandankumarweshay|ruck, rlandy coming to next question17:21
chandankumarhttps://review.opendev.org/#/c/708968/6/config/release/tripleo-ci/CentOS-8/master.yml@61 here we are using include pkgs for all tripleo projects17:22
chandankumarwe have tripleo component also17:22
chandankumarcan we use that repo there?17:22
chandankumarit will contains the fresh content build by dlrn17:22
weshay|ruckrfolco, rlandy everything merged?17:22
weshay|ruckminus tripleo-ci?17:23
rlandyweshay|ruck: there are two options for tripleo-ci patches ...17:23
rlandyhttps://review.opendev.org/709193 Use md5sum if baseurl shows up more than once in delorean.repo file17:23
*** tosky has quit IRC17:23
rlandychecking that in check now17:23
weshay|ruckchandankumar, ooh.. that will break17:23
weshay|ruckyes we need also those packages included17:24
weshay|ruckbut /me wonders how the create repo role will handle this17:24
rlandyack to chandankumar question17:24
rlandywe need to sort that for the triple component17:24
weshay|ruckchandankumar, ykarel perhaps we start to consider tripleo-repos17:24
chandankumarweshay|ruck, we need tripleo component also as well as packages17:24
rlandyit will break there17:24
chandankumarweshay|ruck, what about other components if they want to use the same17:24
weshay|rucklet's chat17:25
ykarelweshay|ruck, yeap, but possibly can be a seperate effort17:25
ykarelmay be exercise for new members17:25
rlandyover-complicated!!!!17:25
weshay|ruckhttps://meet.google.com/suc-gooj-ipn?authuser=117:25
zbrpanda: good luck with your need. tox-py36 can be done right away, but NOT with docker.17:27
pandaoh shoot17:27
pandathat's true17:27
zbror you can start using ansible zuul instance, where it works.17:27
pandawell we'll have the same problem for every aspect of promoter in centos817:28
chandankumarweshay|ruck, https://github.com/openstack/tripleo-quickstart/blob/master/roles/repo-setup/templates/repo_setup.sh.j2#L7217:28
pandazbr: well, then don't upstrede to centos8, I'll find another way17:29
rfolcozbr, panda: are you talking about fixing rdo-tox-molecule job ?17:29
pandarfolco: no, fixing tox jobs on my promoter patch17:29
rfolcooh I see17:29
pandaanyway, my job is done for today17:29
pandaintegration jobs are passing, taht means that all in all, if  nothin g else works, we have a stage at thich the promoter works with aggregate17:30
pandaI'll rest and start with docs on monday.17:30
chandankumarweshay|ruck, https://github.com/rdo-infra/rdo-jobs/blob/master/playbooks/podman/install-podman-rpm.yaml#L3317:30
* panda six, going dark.17:31
*** Trevor_V has joined #oooq17:39
rlandyrfolco: https://review.opendev.org/#/c/709193/ is a better solution - pls review17:41
rfolcolooking rlandy17:41
weshay|ruckykarel, https://hackmd.io/FyI1eRz9SoCdgnsITR3PuQ17:42
*** TrevorV has quit IRC17:42
ykarelweshay|ruck, ack17:44
*** amoralej is now known as amoralej|off17:44
*** jpena is now known as jpena|off17:47
rfolcorlandy, logic seems good, we can optimize in follow-up changes17:50
rlandyweshay|ruck: rfolco: going to try testproject with depends on https://review.opendev.org/#/c/709193/17:51
rlandyit may or may not take17:51
rlandywe will push with the wrong hash if it doesn't17:51
weshay|ruckthat's fine17:51
weshay|ruckthanks17:51
rfolcoisn't that bad17:51
rlandyhere we go ....17:52
rlandy2020-02-21 17:51:31 | 127.0.0.1:8787/tripleomaster/centos-binary-neutron-base                    d15d61afc27948b715fc611aa20636dd   04fc2a77e6a3   3 minutes ago        997 MB17:57
*** derekh has quit IRC17:58
rlandywe can't get tripleo-ci container patch in due to centos07 conatinerbuild fails18:00
rlandyok - if we move centos-7 container build check jobs to non-voting??18:02
weshay|ruckrlandy, it's being removed from ussuri18:03
*** holser has quit IRC18:04
weshay|ruckrlandy, https://review.opendev.org/#/c/708893/18:04
rlandyhttps://review.opendev.org/#/c/709193/ will be blocked on ut18:04
rlandyit18:04
ykarelweshay|ruck, i just posted comment there, seems issue there18:04
weshay|ruckrlandy, rebase on top18:04
* weshay|ruck looks18:04
rlandyack: ^^ when is resolved18:05
ykarelwith that patch, branched jobs will stop running18:05
ykarelin tripleo-ci18:06
weshay|ruckykarel, why? they are running now18:06
weshay|ruckin zuul status 70919318:06
ykarelweshay|ruck, looking again18:06
* weshay|ruck thinks we should also land https://review.opendev.org/#/c/709069/118:07
weshay|ruckit's not an either or18:07
ykarelisn't the patch 708893?18:07
* ykarel looks other review too18:08
weshay|ruckah ya.. sorry18:08
weshay|rucktired18:08
weshay|ruckbut they are running there too w/ 70889318:08
zbrweshay|ruck: look at https://review.opendev.org/#/c/709184/4 and its parent, fixing docker installation on centos-8, what we need.18:19
zbri am already engaging with infra on it, hopefully we will get it in.18:19
weshay|ruck++18:20
*** ykarel is now known as ykarel|away18:24
weshay|ruckrlandy, heh.. tripleo-repos is also broken on current18:30
weshay|ruckbut I think it will be easier to fix there18:30
rlandycharming18:31
* rlandy gets minimal release file review in so we can test there ...18:31
rlandysec18:31
rlandycontainers build-push started18:32
rlandyat least the JOB started18:33
*** openstackstatus has quit IRC18:33
rlandy https://review.opendev.org/709215 DNM: Really DNM ... Testing centos 8 release files and repos18:52
rlandyweshay|ruck: ^^ we can start to add tripleo-repos on to there18:52
rlandyI used the tripleo-ci-testing file because we hope to have those containers18:53
rlandyforce_periodic: true18:53
weshay|ruckk19:05
weshay|ruckfine.. what ever unblocks at this point :)19:05
rlandyweshay|ruck: rfolco: ping ping19:13
rfolcoo/19:13
rlandyhttps://review.rdoproject.org/zuul/stream/7463b6277bc04da79b5717fe607d20d0?logfile=console.log19:13
rlandyworking!!!19:14
rfolcolooking good19:14
rlandywe have containers tagged with md519:14
weshay|ruckhey19:15
weshay|ruckOH19:15
weshay|rucksorry out put was set to headphones19:15
rlandyhttps://console.registry.rdoproject.org/registry#/images/tripleomaster/centos-binary-aodh-api:d15d61afc27948b715fc611aa20636dd19:15
rlandyweshay|ruck: ^^19:15
rlandywoohoo19:15
rlandythat's it19:16
weshay|rucksee it !!!19:16
weshay|ruckhttps://trunk.registry.rdoproject.org:8443/oapi/v1/namespaces/tripleomaster/imagestreamtags/19:16
weshay|ruckNICE19:16
weshay|ruckNICE19:16
weshay|ruckNICE19:16
weshay|ruckrlandy, rfolco well done19:16
rlandyOMG19:16
rlandyfinally19:16
weshay|ruckrlandy, just hold tight.. I'm pretty sure we're just on to the next .. BOOM19:17
rlandyweshay|ruck: I know19:18
rlandycalled release files19:18
rlandytrying to run standalone19:18
weshay|ruckrfolco, paste me a link to the molecule work19:18
rlandywait - are all chandankumar's patches merged?19:18
weshay|ruckrlandy, no19:18
rlandyfor image builds19:18
weshay|ruckwe're building overcloud images off a testproject19:19
rlandylet's get in what we can19:19
rfolcoweshay|ruck, I am making it modular in several patches as rlandy's changes broke it completely19:19
weshay|ruckrlandy, ya.. let's try a standalone19:19
rlandyhttps://review.opendev.org/#/q/topic:c8imagebuild+(status:open+OR+status:merged)19:19
chandankumarrlandy, /me thinks, c8 image build needs love19:19
rlandyweshay|ruck: regular or component?19:19
weshay|ruckand then maybe promote tripleo-ci-testing to current-tripleo if it works19:19
rlandychandankumar: on it19:19
rfolcooriginal start was https://review.rdoproject.org/r/#/c/25026/19:19
weshay|ruckrlandy, the baby step19:19
chandankumarweshay|ruck, rlandy we have images available on dpawlik server19:20
rfolcobut I am breaking into smaller changes and adding more tests19:20
chandankumardpawlik, sorry19:20
rlandyweshay|ruck: sorry - which standalone - regular periodic - using tripleo-ci-testing?19:20
chandankumarrfolco, rlandy thanks, if needs any stuff, please let me know in your evening19:20
rlandychandankumar: yep - will look after I start standalone19:21
rlandyit's late for you :)19:21
weshay|ruckrlandy, doesn't matter19:22
rlandyk -19:22
rlandyhttps://review.opendev.org/#/c/709193/ - pls vote19:25
rlandyworked on periodic19:25
chandankumarpanda, weshay|ruck rfolco rlandy weshay|ruck zbr see ya on monday, Have a nice weekend ahead :-)19:26
*** chandankumar is now known as raukadah19:27
weshay|ruckraukadah, 0/ thanks brotha.. take care19:27
rlandyraukadah: have a good weekend19:31
rlandyoops19:46
rlandyERROR! Could not find specified file in role: vars/vars/redhat.yml19:47
rlandyfixing19:47
rlandyhttps://review.rdoproject.org/r/2524119:50
rlandyrfolco: ^^ pls review ... fixing error above19:50
rfolco+119:51
*** jfrancoa has quit IRC19:52
rlandythanks19:52
*** TrevorV has joined #oooq20:23
rlandyrfolco: question ...20:25
rlandyhttps://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/roles/get_hash/tasks/get_hash.yaml#L5620:26
rlandyfor any periodic jobs running ...20:26
rlandypromote_source would have to be defined20:26
rlandymaybe this goes into the release file??20:26
rfolcoI remember this is default somewhere... if not, we can define here https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/roles/get_hash/defaults/main.yml20:27
*** Trevor_V has quit IRC20:27
rfolcobut this is being set for each case20:29
rfolcohttps://github.com/rdo-infra/review.rdoproject.org-config/search?q=promote_source&unscoped_q=promote_source20:29
rfolcothere is no "default"... in legacy jobs, default was tripleo-ci-testing if not specified, I guess20:30
rlandyrfolco: true but for promotion jobs21:00
rlandybut regular periodic jobs are not set21:01
* rlandy think release file21:01
weshay|ruckrlandy, sorry had to change a tire.. and go to the tire store :(21:25
rlandyweshay|ruck: no problem21:27
rlandyweshay|ruck: you didn't miss much21:27
weshay|ruckk21:27
rlandyjust trying to get repo-setup to work21:27
rlandyand then install-packages21:27
weshay|ruckaye21:27
rlandyso our next mountain is release files21:27
rlandyhttps://review.opendev.org/#/c/70919321:28
rlandyok - w+ ^^?21:28
weshay|ruck2020-02-21 20:59:27 | Error: Unable to find a match: python3-tripleoclient ceph-ansible21:28
rlandyweshay|ruck: yeah - well I pulled out all other repos21:28
rlandyI curled the delorean repo21:29
weshay|ruckoh.. ya.. you still need deps and ceph etc21:29
weshay|rucksorry21:29
weshay|ruckthat's my bad21:29
rlandybut I think it got overwritten by tripleo-ansible21:29
rlandyso I changed that name to start21:29
* rlandy looks21:29
weshay|ruckhttps://logserver.rdoproject.org/95/24995/16/check/periodic-tripleo-ci-centos-8-standalone-master/3356a96/logs/undercloud/etc/yum.repos.d/21:30
rlandymaster-build-deps21:30
rlandyand what else?21:30
* weshay|ruck looks at another job to be sure21:30
rlandysee how delorean.repo is triple-ansible?21:30
weshay|ruckhttps://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_313/709210/2/check/tripleo-ci-centos-7-scenario001-standalone/31335a6/logs/undercloud/etc/yum.repos.d/quickstart-centos-ceph-nautilus.repo21:31
weshay|ruckrlandy, that's odd21:31
weshay|ruckyou just did a curl I thought21:31
rlandywhere does that come from?21:31
rlandyI did :)21:31
rlandyhttps://review.rdoproject.org/zuul/stream/5b0dca80753443abb879919923785848?logfile=console.log21:32
rlandywill see if the name change helps21:32
weshay|ruck  - type: generic21:32
weshay|ruck    reponame: quickstart-centos-qemu21:32
weshay|ruck    filename: quickstart-centos-qemu.repo21:32
weshay|ruck    baseurl: ${NODEPOOL_RDO_PROXY}/centos8-master/deps/advvirt/8.1/21:32
weshay|ruck  - type: generic21:32
weshay|ruck    reponame: quickstart-centos-ceph-nautilus21:32
weshay|ruck    filename: quickstart-centos-ceph-nautilus.repo21:32
weshay|ruck    baseurl: ${NODEPOOL_RDO_PROXY}/centos8-master/deps/storage/storage8-ceph-nautilus/x86_64/21:32
rlandyadding back21:33
weshay|ruckrlandy, maybe it's best to leave line 59 and lower21:33
weshay|rucksorry..21:33
rlandylet's try21:34
rlandyadding back21:34
rlandyincluding the repo_cmd_after?21:34
rlandyhttps://logserver.rdoproject.org/95/24995/16/check/periodic-tripleo-ci-centos-8-standalone-master/5b0dca8/logs/undercloud/etc/yum.repos.d/21:37
rlandyhmmm ... my curl doesn't work21:38
rlandyOH21:38
rlandyI lie ... it does21:38
rlandyhttps://logserver.rdoproject.org/95/24995/16/check/periodic-tripleo-ci-centos-8-standalone-master/5b0dca8/logs/undercloud/etc/yum.repos.d/delorean-components.repo.txt.gz21:38
rlandyok - adding repos back21:38
*** TrevorV has quit IRC21:47
*** rfolco has quit IRC21:52
rlandyweshay|ruck: I'm out - back on sunday ...22:08
rlandypls w+ https://review.opendev.org/#/c/709193/if you agree22:09
weshay|ruckrlandy, don't work sunday :)22:09
* weshay|ruck looks22:09
weshay|ruckdone22:09
weshay|ruckgood shabbos22:09
rlandyweshay|ruck: thanks good shabbos ... get some rest :)22:10
*** rlandy has quit IRC22:10
*** dpawlik has quit IRC22:22
*** rfolco has joined #oooq23:03
*** jbadiapa has quit IRC23:48
*** holser has joined #oooq23:54
« Thursday, 2020-02-20 Index Saturday, 2020-02-22 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!