Thursday, 2019-04-25

*** dsneddon has quit IRC00:20
*** Goneri has quit IRC00:30
*** dsneddon has joined #oooq00:47
*** rlandy|bbl is now known as rlandy00:48
*** dsneddon has quit IRC00:52
*** vinaykns has quit IRC01:06
*** dsneddon has joined #oooq01:25
*** dsneddon has quit IRC01:30
*** ykarel|away has joined #oooq01:37
*** apetrich has quit IRC01:57
*** dsneddon has joined #oooq02:01
*** dsneddon has quit IRC02:06
*** ykarel|away has quit IRC02:06
*** ykarel|away has joined #oooq02:34
*** dsneddon has joined #oooq02:40
*** dsneddon has quit IRC02:45
*** dsneddon has joined #oooq03:20
*** dsneddon has quit IRC03:25
*** ykarel|away is now known as ykarel03:30
*** dsneddon has joined #oooq03:55
*** udesale has joined #oooq03:55
*** aakarsh is now known as aakarsh|off03:59
*** dsneddon has quit IRC04:00
*** ykarel is now known as ykarel|afk04:14
*** ykarel|afk has quit IRC04:19
*** ykarel|afk has joined #oooq04:35
*** dsneddon has joined #oooq04:35
*** ykarel|afk is now known as ykarel04:36
*** dsneddon has quit IRC04:40
*** quiquell|off is now known as quiquell05:09
*** dsneddon has joined #oooq05:12
*** dsneddon has quit IRC05:23
*** marios has joined #oooq05:25
*** dsneddon has joined #oooq05:58
*** dsneddon has quit IRC06:02
*** jfrancoa has joined #oooq06:25
*** dsneddon has joined #oooq06:38
*** dsneddon has quit IRC06:43
*** quiquell is now known as quiquell|brb06:48
*** apetrich has joined #oooq06:52
*** amoralej|off is now known as amoralej07:04
*** jfrancoa has quit IRC07:07
*** ccamacho has joined #oooq07:08
*** dsneddon has joined #oooq07:09
*** ccamacho has quit IRC07:09
*** ccamacho has joined #oooq07:10
*** dsneddon has quit IRC07:18
*** zbr has joined #oooq07:24
zbrhi! I am finally back :)07:24
mariosWE07:26
mariosR07:26
pandazbr: sorry, who are you ?07:26
mariosSOREN07:26
marioshttps://review.opendev.org/#/c/653951/2 docs review please thanks about emit-release http://logs.openstack.org/51/653951/2/check/openstack-tox-docs/90eca1b/html/ci/emit_releases_file.html and also comments in https://review.opendev.org/65209507:30
mariosnice easy morning review07:30
mariosplease07:30
mariosthank you07:30
* marios bows07:30
*** bogdando has joined #oooq07:39
*** jpena|off is now known as jpena07:43
*** kopecmartin|off is now known as kopecmartin07:46
*** dsneddon has joined #oooq07:47
*** ykarel is now known as ykarel|lunch07:49
*** dalvarez has joined #oooq07:51
*** dsneddon has quit IRC07:53
*** dsneddon has joined #oooq08:00
*** quiquell|brb is now known as quiquell08:07
quiquellmarios: Who are the new re?08:08
quiquellRuck rover I mean08:08
marios quiquell: hm i think sorin is the lucky guy? /me checks etherpad08:09
marioszbr: and rfolco|ruck08:10
marioszbr: good news! ^08:10
mariosquiquell: ^08:10
zbrhaha :) always happens to me when I am out....08:10
marioszbr: yeah he got me like that before too :) if you can't complain about it you're it08:10
zbron the other hand, there is one serious issue: my yearly PTO starts on May 8 so I will be able to rover until 7th only.08:11
zbri kinda hate etherpad due to its inability to remember pads, is all about knowing the right url...08:12
zbrnot to mention its stupidity of using star icon for "save revision" instead of a standard icon like a floppy-disk. start is defacto stantdard for bookmarking/following.08:14
marioszbr: weshay added something in grafana to point to current etherpad08:14
zbrbut is opensource... which means if I care enough, I could make a OPR.08:14
mariosmore good news !08:14
marioszbr: thats your first task ^08:14
*** derekh has joined #oooq08:14
marioszbr: i mean in http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=1&from=now-7d&to=now you can see "current ruck rover etherpad foo"08:15
marioszbr: if that's what you're on about08:15
*** zbr is now known as zbr|rover08:18
zbr|roveri wonder if it would not be better if we would have a static ruckrover document, and just move old data to a google doc.08:20
quiquellzbr|rover: new etherpad https://etherpad.openstack.org/p/ruckroversprint908:24
*** ykarel|lunch is now known as ykarel08:39
*** dsneddon has quit IRC09:03
*** dtantsur|afk is now known as dtantsur09:33
*** dsneddon has joined #oooq09:35
*** dsneddon has quit IRC09:39
*** zbr|rover has quit IRC09:42
*** zbr has joined #oooq09:43
*** vinaykns has joined #oooq09:56
*** dsneddon has joined #oooq10:06
*** ccamacho has quit IRC10:11
*** dsneddon has quit IRC10:11
*** dsneddon has joined #oooq10:46
*** dsneddon has quit IRC10:52
*** jpena is now known as jpena|lunch10:55
*** jfrancoa has joined #oooq10:58
*** dsneddon has joined #oooq11:00
*** vinaykns has quit IRC11:04
*** rfolco|ruck is now known as rfolco|ruck|doct11:12
*** gkadam has joined #oooq11:16
*** zbr has quit IRC11:31
*** zbr has joined #oooq11:32
*** udesale has quit IRC11:33
marioshttps://review.opendev.org/#/c/653951/2 docs review please thanks rfolco weshay|rover sshnaidm about emit-release http://logs.openstack.org/51/653951/2/check/openstack-tox-docs/90eca1b/html/ci/emit_releases_file.html and also comments in https://review.opendev.org/65209511:53
weshay|roverrfolco|ruck|doct don't delete anything from the ruckrover etherpad please11:55
weshay|roverjust push items down lower and strike through11:55
quiquellweshay|rover: I did cleanup the dups11:56
weshay|roverok... rfolco|ruck|doct quiquell && all don't delete anything from the ruckrover etherpad please11:57
weshay|rover:)11:57
quiquellnoted11:57
weshay|rovermatbu you around?11:57
weshay|rovermatbu /me wants you to say hey to old friend11:57
weshay|roverto an old11:58
*** panda is now known as panda|lunch11:59
weshay|roverquiquell so we're money w/ ovb jobs :) it looks like.. thanks for the help on htat11:59
quiquell\o/12:00
quiquellafter the two review merged12:00
quiquellweshay|rover: also f28 is working there too diskimage-builder upgrade did work12:01
weshay|roverNICE12:01
quiquellas preventing action maybe they can put some infra to sync opendev version and rdo version12:01
*** vinaykns has joined #oooq12:03
ykarelyeah master is promoting12:03
*** jfrancoa has quit IRC12:03
chandankumararxcruz: Hello12:03
arxcruzchandankumar: hello12:03
*** dsneddon has quit IRC12:04
ykarelweshay|rover, can you check comment on https://review.rdoproject.org/r/#/c/20147/12:04
chandankumararxcruz: can we start working on this https://tree.taiga.io/project/tripleo-ci-board/us/867?milestone=229187 ? so that we have few upstream jobs ported to os_tempest before ptg12:04
arxcruzchandankumar: can we? you're the UA now :)12:05
chandankumararxcruz: we as a team :-)12:05
arxcruzchandankumar: is that your plan for next sprint ?12:05
arxcruzif so, I have no problems :)12:06
chandankumararxcruz: https://tree.taiga.io/project/tripleo-ci-board/task/1033 has a long list, this will take time as those are on puppet-projects12:06
chandankumararxcruz: yes it is next sprint12:06
matbuweshay|rover: hey12:09
matbuweshay|rover: yep around12:09
weshay|rovermatbu hey join my blue in 5min if you can.. old friend is on12:11
matbuweshay|rover: ack12:13
fultonjfyi: 3rd party jobs green going to start implementing http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005599.html12:17
zbrweshay|rover: let me know when you have few minutes to sync12:19
weshay|roveraye.. few min12:19
*** rlandy has joined #oooq12:20
*** derekh has quit IRC12:23
*** tosky has joined #oooq12:25
quiquellweshay|rover: upgrading reproducer containers https://review.rdoproject.org/r/#/c/20381/12:29
*** dsneddon has joined #oooq12:33
*** altlogbot_3 has quit IRC12:33
weshay|roverEVERYONE12:36
weshay|roverplease read https://etherpad.openstack.org/p/ruckroversprint9 lines 10-1412:36
weshay|roverzbr quiquell rlandy ykarel panda|lunch ^12:38
*** dsneddon has quit IRC12:38
*** altlogbot_0 has joined #oooq12:38
* ykarel looks12:38
weshay|roverany reason we should not let fultonj proceed?12:38
weshay|roverw/ https://review.opendev.org/#/c/655220/112:39
ykarel+1 from me, considering jobs are greeen both in check and promotion pipeline12:39
weshay|roveragree12:39
weshay|roverrlandy quiquell as former ruck/rover...  can you guys please review12:39
mariosweshay|rover: ack you mean on the non voting12:40
mariosweshay|rover: i mean, as long as they have a clear path back to voting12:40
mariosweshay|rover: that's what we did last time fultonj12:40
rlandyweshay|rover: looks good- discussed with rfolco|ruck|doct yesterday12:40
mariosweshay|rover: we even posted the patches at the time with depends-on so they were ready12:40
rlandyadded note that the logging patch is w+12:41
weshay|roverthe patches required to merge are on https://etherpad.openstack.org/p/tripleo-integration-squad-status  lines 15 -1912:41
mariosweshay|rover: hmm not sure they have that yet12:41
weshay|rovermarios have what exactly?12:41
mariosweshay|rover: i mean the clear path back to making it voting12:41
weshay|roverfultonj ^12:42
mariosweshay|rover: like 'and then this patch will make them voting again and it depends-on: foo12:42
quiquellchecking12:42
weshay|roverfultonj marios the plan atm is to merge through line 19 and have a revert of https://review.opendev.org/#/c/655220/12:42
weshay|rovercores.. please review the tq/tqe/ci patches12:43
zbrweshay|rover: i see one problem: making non-voting risks breaking branches that do not receive nautilius. but i guess we have no easy way to make them non voting only for master.12:43
weshay|roverzbr ya.. I wish tripleo-ci was branched12:44
weshay|roverin this case12:44
quiquellweshay|rover, zbr: we can filter branches at zuul config12:44
*** derekh has joined #oooq12:44
quiquellweshay|rover: and creatre a pair of jobs the non voting for nautilus migration and voting for non migration12:45
zbrweshay|rover: i am not sure I share the same dream, as dealing with branches can easily become a nightmare.12:45
quiquellack12:45
chandankumarkopecmartin: can I add you as a qe on this user story https://tree.taiga.io/project/tripleo-ci-board/us/867 ?12:45
zbrquiquell: i know we can filter, but we would have to split jobs to do this.12:45
weshay|roverif you like I can open my blue to discuss12:45
ykarelzbr, standalone scenarios running other than master/steing also?12:45
quiquellzbr: that's what I  mean in case nautilus migration is going to be long process12:45
kopecmartinchandankumar, sure12:45
*** gkadam has quit IRC12:46
quiquellweshay|rover: let's just do non-voting if nautilus migration takes too long you can split jobs12:46
ykarelzbr, afaik they are running for master/stein only and nautilus transition going on only for master/stein12:46
quiquell^ this is the answer12:46
quiquell:-)12:46
fultonjsorry i got pinged12:46
weshay|roverquiquell that sounds like a good plan12:46
quiquellweshay|rover: going to +w https://review.opendev.org/#/c/65522012:46
zbrahh, ok. i missed to read the entire regex12:46
* fultonj reads backlog12:46
*** altlogbot_0 has quit IRC12:47
*** altlogbot_3 has joined #oooq12:48
weshay|roverthe gate queue is quite small  atm12:48
fultonjline 25 of https://etherpad.openstack.org/p/tripleo-integration-squad-status12:48
fultonj"finally we will revert 655220 to make 001/004 voting again"12:49
fultonj(fmount and gfidente are both out today and i'm getting lots of pings)12:49
weshay|roverfultonj so we need to merge https://review.opendev.org/#/q/topic:ceph-nautilus-stable/stein before bringing jobs back to voting too?12:49
mariosfultonj: why not post therevert with depends-on12:49
weshay|roverit would be ideal to have all the patches w/ a +1 from zuul and cores +2 lined up12:51
mariosfultonj: if the story is true, a revert with depends-on should show the jobs green12:51
mariosfultonj: and we have ci run there ready as soon as we can to make it voting again12:51
*** aakarsh|off is now known as aakarsh12:53
fultonjmarios weshay|rover ykarel what i'm hearing is that the plan is not complete so we can't proceed yet12:56
fultonji need to update the plan and i'll probably ask a few clarifying questions about the backlog ^12:56
*** udesale has joined #oooq12:56
mariosfultonj: if you recall last time you and i did this for some other ceph new thing, we had the revert patch ready to go with depends-on the thing that fixed it again12:56
mariosfultonj: if for some reason we can't do that then lets discuss but for me that would be ideal.12:57
mariosperhaps if we can't do it yet then lets not proceed with the making it non voting until we can? or that isn't possible? fultonj ? weshay|rover ?12:57
fultonjmarios: yes, that's fine. i'm glad to improve the plan. i just need to figure out what exactly you're talking about. i'll look for the review you did last time as an example12:58
fultonjmarios: i remember it, i just don't have the exact reviews in place12:58
fultonji'll need to find it, diff it against the current plan and see what' missing and get feedback from you + weshay|rover on the updated plan12:59
mariosfultonj: this https://review.opendev.org/63836412:59
weshay|roverstarting a planning meeting now  fultonj you are more than welcome to join.. this is at the top of my agenda13:00
mariosfultonj: joining a call will be slower to respond for a while13:00
weshay|roverhttps://bluejeans.com/587845809713:00
fultonjweshay|rover: thanks but i think i need to carefully understand the details and put the other reviews in place.13:01
fultonji'll join just in case actually :)13:01
chandankumarweshay|rover: i think we are waiting here https://bluejeans.com/707186672813:01
*** dsneddon has joined #oooq13:05
*** ykarel is now known as ykarel|mtg13:08
*** dsneddon has quit IRC13:09
*** rfolco|ruck|doct is now known as rfolco|ruck13:18
*** altlogbot_3 has quit IRC13:24
*** altlogbot_0 has joined #oooq13:26
*** altlogbot_0 has quit IRC13:28
*** altlogbot_1 has joined #oooq13:32
*** aakarsh has quit IRC13:35
*** dsneddon has joined #oooq13:41
*** altlogbot_1 has quit IRC13:42
*** dsneddon has quit IRC13:46
*** Vorrtex has joined #oooq13:46
*** altlogbot_3 has joined #oooq13:56
*** aakarsh has joined #oooq14:01
*** panda|lunch is now known as panda14:02
*** matbu has quit IRC14:05
*** mjturek has joined #oooq14:06
weshay|roverzbr did you drop?14:08
zbrweshay|rover: i am back14:09
weshay|roverzbr can't hear u14:09
*** dtrainor has quit IRC14:10
weshay|roverzbr what is your PTO again...? may 5 and may 8th?14:10
weshay|roveror 5-814:10
zbr5 and >=814:11
*** matbu has joined #oooq14:12
pandazbr: 8 to ?14:12
*** Goneri has joined #oooq14:14
zbrpanda: May 8-2214:16
*** dsneddon has joined #oooq14:17
mariosweshay|rover: reminder i'm out tomorrow14:20
*** dsneddon has quit IRC14:22
*** dtrainor has joined #oooq14:24
mariosweshay|rover: rfolco|ruck chandankumar updated https://tree.taiga.io/project/tripleo-ci-board/us/959 https://tree.taiga.io/project/tripleo-ci-board/task/989 https://tree.taiga.io/project/tripleo-ci-board/task/990 WRT make it about stable/stein14:32
*** jpena|lunch is now known as jpena14:33
weshay|roverpanda have a sec to join me in #rhos-dev?14:34
*** bogdando has quit IRC14:34
chandankumarmarios: looks good, thanks :-)14:35
chandankumarweshay|rover: panda https://review.opendev.org/#/c/653702/ please have a look when free!14:35
weshay|roverpanda join my blue14:37
marioshttps://review.opendev.org/#/c/653951/ docs review please thanks about emit-release http://logs.openstack.org/51/653951/2/check/openstack-tox-docs/90eca1b/html/ci/emit_releases_file.html and also comments in https://review.opendev.org/65209514:38
rfolco|ruckarxcruz, chandankumar this failed in gate, intermittent failure ? http://logs.openstack.org/41/653941/2/gate/tripleo-ci-centos-7-standalone/e004f37/logs/undercloud/home/zuul/tempest/tempest.html.gz14:41
arxcruzlet me check14:41
weshay|roverya.. basic ops has been failing on the standalone deployments from time to time14:42
weshay|roverarxcruz I think standalone was set to qemu virt14:42
rfolco|ruckok yeah another one for f2814:42
rfolco|ruckhttp://logs.openstack.org/41/653941/2/gate/tripleo-ci-fedora-28-standalone/a1b74a6/logs/undercloud/home/zuul/tempest/tempest.html.gz14:42
arxcruzrfolco|ruck: the first one is random, but we must investigate14:43
arxcruzthe second one on fedora, i need to check14:44
rfolco|ruckarxcruz, one more... I'm soreeen14:44
rfolco|ruckhttp://logs.openstack.org/69/648469/8/gate/tripleo-ci-centos-7-standalone/99fa027/logs/undercloud/home/zuul/tempest/tempest.html.gz14:44
arxcruzrfolco|ruck: but those are not the same test14:45
arxcruzit's the same network scenario14:45
rfolco|rucksame class14:45
rfolco|ruck.TestNetworkBasicOps14:45
arxcruzi believe it's random, but need to investigate14:45
arxcruzrfolco|ruck: yes, same class, diff scenario14:45
rfolco|ruckok14:45
rfolco|ruckif this doesn't disappear from cockpit I'll file a bug14:46
arxcruzok14:47
chandankumararxcruz: thanks for looking into that.14:50
arxcruzrfolco|ruck: weird in openstack-health isn't showing the failures for this standalone14:55
rfolco|ruckarxcruz, cockpit rocks!14:55
arxcruz:D14:55
*** dsneddon has joined #oooq14:55
*** Goneri has quit IRC15:05
*** ykarel|mtg is now known as ykarel15:06
ykarelrfolco|ruck, weshay|rover fedora container build is broken just in case you not knew already15:07
ykarelboth in check and promotion pipeline15:07
rfolco|ruckhmm will take a look asap15:08
rfolco|ruckykarel, any clues? kolla cherry picks ?15:09
ykarelrfolco|ruck, tripleo-repos pip installation is failing15:09
ykarelrfolco|ruck, example log: https://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-fedora-28-master-containers-build-push/2d6daf7/job-output.txt.gz#_2019-04-25_04_10_02_04499215:09
rfolco|ruckzbr, you probably know this better than anyone ^15:10
zbrlooking now...15:13
zbrpinning virtualenv, very bad idea from start.15:14
*** quiquell is now known as quiquell|off15:14
zbrwow, that pinning is not in code we manage, I wonder what would `pip check` return on such a system. going to start a container now.15:18
mariosshutdown sequence()15:18
*** Goneri has joined #oooq15:18
fultonjweshay|rover: the recheck of https://review.opendev.org/#/c/651239 is still not in the queue http://zuul.openstack.org/status15:24
fultonjykarel: ^15:24
fultonji think i need a rebase15:24
ykarelfultonj, there is some merge failed15:25
ykarelneed to check dependency chain15:25
ykarelhmm i think weshay rebased some patch just after rechecking it15:25
ykarelone of dependency15:25
fultonjwhich merge failed? we haven't yet sent anything to merge from the reviews in the plan15:26
fultonjdo you mean a merge to test the dependency chain?15:27
ykarelfultonj, https://review.opendev.org/#/c/655481/ was rebased just after putting recheck15:27
ykarelso job got cancelled15:27
zbrrfolco|ruck: i didn't even know we are installing tripleo-repos from URLs15:32
fultonjykarel: i think i need to work my way backwards up the chain rebasing15:33
zbrrfolco|ruck: ykarel  do you know the value of tripleo_repos_repository variable? is key for that failure. I am trying to reproduce that failing task.15:33
ykarelfultonj, rebasing why?15:33
ykarelzbr that must be commit from zuul cached repo like /home/zuul/src/opendev.org/openstack/tripleo-repos15:34
ykarelbut you can install from https://opendev.org/openstack/tripleo-repos to reproduce locally15:34
ykarelfultonj, rebase all chain?15:35
fultonjykarel: that's probably too heavy handed15:36
ykarelshould not be needed i think, zuul queue is already too long15:36
fultonjykarel: i basically want to get 651239 rechecked15:36
ykarelfultonj, i think the open stable/stein patches also will block master patches to merge because of Depends-On based on change id15:36
ykarelfultonj, can you change change-id based depends on on url ones? to avoid ^^ issue15:38
ykarelfor the tht patch15:38
ykarelmainly tripleo-common depends-on needs adjustment15:38
ykarelon 239 Depends-On: I133cd0cd632e0ba6b5a844a89bf74a063cf50f0715:39
fultonjykarel: you mean the depends on in https://review.opendev.org/#/c/655482 should depend on https://review.opendev.org/#/c/655481 by URL15:39
fultonj?15:39
fultonjwait...15:40
zbrykarel: i am not convinced, look at http://codesearch.openstack.org/?q=tripleo_repos_repository&i=nope&files=&repos= -- which tells me that it can also be an url, which AFAIK, is invalid if it does not have the #egg= part15:40
ykarelfultonj, i mean https://review.opendev.org/#/c/65123915:40
zbrwe need more verbosity to see what vars are passed to it.15:40
ykarelzbr, tripleo_repos_repository: "{{ ansible_user_dir }}/{{ zuul.projects['opendev.org/openstack/tripleo-repos'].src_dir }}"15:41
ykarelin ci ^^ is used15:41
ykarelfultonj, since stein patches will be merged after all master patches are merged no need to adjust those15:41
ykarelmainly need to adjust master patches if you keep stable/stein patches open15:42
fultonjykarel: ah, because the chage id is the same15:42
ykarelfultonj, yes15:42
fultonjwe need to differentiate the cherry pick by using a URL with a diff #15:43
fultonji get it15:43
fultonjthanks ykarel15:43
fultonjyes, i'll upate https://review.opendev.org/#/c/651239 right now15:43
ykarelfultonj, yes ^^ should be enough15:43
ykarelfultonj, so you will wait for result on ^^ before start merging15:43
fultonjykarel: yes i will15:44
ykarelfultonj, okk15:44
ykarelfultonj, the other patch can be workflowed i think15:44
ykarelthat should not harm15:44
fultonjwhich "other patch" ? :)15:44
fultonjso many...15:44
ykarel2nd in the list15:45
ykarelhttps://review.openstack.org/#/c/65295915:45
weshay|roverrfolco|ruck can you please review the tripleo promotion jobs for queens15:45
fultonjykarel: ok15:46
ykarelweshay|rover, you saw my comment in https://review.rdoproject.org/r/#/c/20147/?15:46
weshay|roverexploring that15:47
ykarelokk15:48
zbrykarel: i am unable to reproduce venv creation failure in my f28 pure image.15:50
ykarelzbr, you tried same task?15:50
ykarellike site package enabled and all?15:50
weshay|roverinteresting15:50
*** marios has quit IRC15:51
zbrykarel: mostly but no ansible, just cli. now trying with ansible15:51
ykarelzbr, ack try exact same task15:52
weshay|roverykarel thanks for pointing that out https://review.rdoproject.org/r/#/c/2046115:54
ykarelweshay|rover, +115:54
weshay|roverzbr rfolco|ruck you'll have a green board  later today15:54
fultonjykarel: ok i see 651239,1515:54
fultonjin the queue15:54
ykarelyup i saw15:55
*** ykarel is now known as ykarel|away15:57
weshay|roverzbr renick please15:59
*** weshay|rover is now known as weshay15:59
weshayzbr I have time now to sync if needed15:59
zbrok, i will be quick16:00
*** amoralej is now known as amoralej|off16:00
zbrweshay: i joined your bj16:04
*** ykarel|away has quit IRC16:08
weshayarxcruz you around?16:12
*** matbu has quit IRC16:13
rfolco|ruckweshay, whats up with queens ?16:14
weshayrfolco|ruck https://review.rdoproject.org/r/#/c/20461/16:14
weshayrfolco|ruck join my blue16:14
*** matbu has joined #oooq16:15
*** sshnaidm|off has quit IRC16:18
*** quiquell|off has quit IRC16:20
*** ykarel|away has joined #oooq16:32
weshayzbr rfolco|ruck https://bugs.launchpad.net/tripleo/+bug/182612916:37
openstackLaunchpad bug 1826129 in tripleo "Multiple jobs with IOError: [Errno 26] Text file busy: '/home/zuul/workspace/.quickstart/bin/python2'" [Critical,Triaged]16:37
*** zbr is now known as zbr|rover16:37
*** udesale has quit IRC16:50
*** jpena is now known as jpena|off16:50
*** Goneri has quit IRC16:52
*** kopecmartin is now known as kopecmartin|off16:54
*** saneax has quit IRC16:57
*** sshnaidm has joined #oooq17:03
*** sshnaidm has quit IRC17:03
*** derekh has quit IRC17:08
*** dtantsur is now known as dtantsur|afk17:14
weshayrfolco|ruck zbr|rover fyi https://bugs.launchpad.net/tripleo/+bug/1806720 might want to see if that's still an issue as qemu might be why tempest is failing from time to tome17:38
openstackLaunchpad bug 1806720 in tripleo "standalone tempest seems to just hang" [Critical,Fix released] - Assigned to Alex Schultz (alex-schultz)17:38
weshaytime17:38
rfolco|ruckweshay, k checking17:38
weshayrlandy this is funny http://codesearch.openstack.org/?q=quickstart-extras-standalone.yml&i=nope&files=&repos=17:53
weshayhowever quickstart-extras-standalone.yml playbook has been deleted and I can't find when17:53
chandankumarrlandy: weshay two more patches https://review.opendev.org/#/c/652919/ and https://review.opendev.org/#/c/652920/ please have a look thanks!17:57
weshaychandankumar fyi.. slowed down by ^ what I pasted to rlandy17:58
weshayI have to recreate / reconstruct a few things17:58
chandankumarweshay: /me takes a look at above bz17:59
weshaythanks18:00
rlandychandankumar: weshay: sorry lunch ... looking now18:08
rlandylol- no trace of that playbook18:10
rlandyweird something must be kicking in ci.centos fs05218:12
rlandyweshay: see https://ci.centos.org/view/rdo/view/tripleo-gate/job/tripleo-quickstart-extras-gate-master-delorean-full-featureset052/lastSuccessfulBuild/consoleFull18:14
rlandy7 weeks ago this passed18:15
weshayhuh18:17
weshayinteresting18:17
*** tosky has quit IRC18:20
rlandyweshay: getting there - change must have been after march 1st18:21
zbr|roverrfolco|ruck: still here? i have an idea about the Text file busy bug18:49
zbr|roveris related to another partial fix from december https://review.opendev.org/#/c/602347/13/quickstart.sh18:49
zbr|roverthere is something fishy there18:50
rfolco|ruckzbr|rover, hmm looking18:51
zbr|roverthat test is using wrong variable.... wrong fix.18:51
zbr|rovermaking a change now.18:51
rfolco|ruckzbr|rover, great18:53
rlandyweshay: beats me - I have no idea where playbook quickstart-extras-standalone.yml is19:18
weshayrlandy right19:18
weshayppooof19:18
* weshay recreating and testing now19:18
rlandythis is insane19:18
rlandyrenamed?19:18
weshayoh maybe19:18
rlandyweshay: how is this possible???19:19
weshayci.centos sucsk19:19
weshaysucks19:19
rlandyI have been through every commit from that date19:19
weshayoh man19:19
weshaysorry19:19
rlandyforget ci.centos - we lost a file in git19:19
rlandypretty impressive19:20
rlandywhich clearly existed on Build #2763 (Mar 1, 2019 1:31:14 PM)19:20
rfolco|ruckweshay, nothing related to qemu from what I could verify... looks more a transient network issue19:21
rfolco|ruckhttps://bugs.launchpad.net/tripleo/+bug/1806720/comments/719:21
openstackLaunchpad bug 1806720 in tripleo "standalone tempest seems to just hang" [Critical,Fix released] - Assigned to Alex Schultz (alex-schultz)19:21
weshayzbr|rover rfolco|ruck can you guys see if we can get ansible bumped https://review.opendev.org/#/c/650169/19:21
*** ykarel|away has quit IRC19:22
weshayzbr|rover /me might look at what version tripleo is using19:22
zbr|roverclearly we cannot bump to 2.8, but i am confident about 2.7 is safe.19:23
zbr|roveri have being using it for months locally19:23
zbr|roverrfolco|ruck: fix for the famous error 26: https://review.opendev.org/#/c/655772/119:27
weshaywoot.. keep picking items that we can fix from "alert" :)19:36
*** tcw has joined #oooq19:38
zbr|roverrlandy: please wf https://review.opendev.org/#/c/655767/1 -- not sure how we ended up with this bug, we are the only repo with that.19:46
rlandyweshay: last attempt ... https://review.opendev.org/#/c/591652/ I can find the review that supposedly added the playbook but was never merged19:49
rlandyif you you still want me to chase this, let me know19:49
weshayrlandy no no.. focus on your things.. I'll take care of this19:51
weshaythank you19:51
weshayrfolco|ruck do we have a bug on http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-fedora-28-master-containers-build-push/69d4144/job-output.txt.gz20:08
weshayI think folks were talking about it20:08
rfolco|ruckzbr|rover, you have a fix, right? I'll file a bug for it weshay20:09
weshayI don't have a fix no20:09
rfolco|ruckoh wait20:10
rfolco|ruckdifferent error20:10
rfolco|ruckthis is not about reusing virtualenv20:10
rfolco|ruckzbr|rover has fix for the text file busy when venv exists20:10
rlandyweshay: ok - so history seems to show that the file was submitted in two reviews that were abandoned and never committed20:12
weshayheh.. ok that's why it was hard to find in git I guess20:13
weshayI may have to announce we're dropping support for quickstart.sh20:15
weshaynowhere to ci it20:15
rfolco|ruckweshay, https://bugs.launchpad.net/tripleo/+bug/182645220:15
openstackLaunchpad bug 1826452 in tripleo "pip install tripleo-repos: DistributionNotFound" [Critical,Triaged]20:15
weshayzbr|rover don't you only want this to run when install-deps.sh is changed? https://review.opendev.org/#/c/652971/5/zuul.d/layout.yaml20:16
rlandyweshay: afaict, it did pass at times, but that could be cruft left in ci.centos workspace. ack if ci.centos is a non-starter then yes, drop it20:18
rlandycommit 973b89ca1db454f2ec9baa4a290fc81d1b9d09dd20:18
rlandy^^ abandoned commit20:18
rlandynever made it into the repo20:18
*** Vorrtex has quit IRC20:18
weshayrlandy doesn't work on the current level of ansible :(20:19
rlandystrike three, it's out20:19
* rlandy moves on20:20
rlandychandankumar: k - that should be all the patches w'ed20:23
*** aakarsh has quit IRC20:23
zbr|roverrfolco|ruck: i did not had time to find the root cause of distribution was not found error but I can say that i am almost sure is related to the image we are using on the node, because I did run the same role on clean images of f28.20:28
zbr|roveri will look tomorrow into that, now is too late.20:28
pandatrunk.registry.rdoproject.org/tripleorocky/rhel-binary-base               585e62e24481ce3e8c1d02a4d0b6f99b4f9df8c0_a43e9ff5   25988a3ca197        2 minutes ago        541 MB20:29
panda(only locally)20:30
pandagood night.20:30
*** panda is now known as panda|off20:30
weshaypanda|off NICE20:31
weshayhrm.. don't see it.. but I'll check tomorrow20:33
weshayonly locally20:33
weshayha20:33
weshayk k20:33
panda|offweshay: yes, it's more complicated, if I push a container after running rhos-release then the container will work only on that releaase. If I push before running rhos-release, then I'll need another rebuild to run rhos-release inside the container. So I'm gonna push the base base base rhel7 image, then modify the image locally and make kolla use the locally modified image. Most of this is in20:40
panda|offhttps://code.engineering.redhat.com/gerrit/16772920:40
*** Goneri has joined #oooq21:14
rlandyweshay: some advice pls ... according to https://bugzilla.redhat.com/show_bug.cgi?id=1693307, this big should be fixed in the 2019-04-24.2 puddle. afaict from https://sf.hosted.upshift.rdu2.redhat.com/logs/95/168695/7/check/tripleo-ci-rhel-7-standalone-rhos-14/8885f60/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-04-24_17_05_20 (checking rpms installed and yum.repos.d) it is not. The bug is ON_QA now. Should I comment or wait for21:15
rlandytheir QA in case it's a standalone config thing still?21:15
openstackbugzilla.redhat.com bug 1693307 in openstack-tripleo-heat-templates "[osp14] TripleO support for configuring Glance's cinder_store settings"" [High,On_qa] - Assigned to pdeore21:15
panda|offINFO:kolla.common.utils:===========================21:21
panda|offINFO:kolla.common.utils:Images that failed to build21:21
panda|offINFO:kolla.common.utils:===========================21:21
panda|offERROR:kolla.common.utils:collectd Failed with status: error21:21
panda|offERROR:kolla.common.utils:fluentd Failed with status: error21:21
panda|offERROR:kolla.common.utils:haproxy Failed with status: error21:21
panda|offERROR:kolla.common.utils:mistral-api Failed with status: matched21:21
panda|offERROR:kolla.common.utils:mistral-base Failed with status: error21:21
panda|offERROR:kolla.common.utils:mistral-engine Failed with status: matched21:21
panda|offERROR:kolla.common.utils:mistral-event-engine Failed with status: matched21:21
panda|offERROR:kolla.common.utils:mistral-executor Failed with status: matched21:21
panda|offERROR:kolla.common.utils:neutron-server Failed with status: error21:21
panda|offERROR:kolla.common.utils:neutron-server-opendaylight Failed with status: matched21:22
panda|offERROR:kolla.common.utils:neutron-server-ovn Failed with status: matched21:22
panda|offERROR:kolla.common.utils:sensu-base Failed with status: error21:22
panda|offERROR:kolla.common.utils:sensu-client Failed with status: matched21:22
panda|offERROR:kolla.common.utils:skydive-agent Failed with status: matched21:22
panda|offERROR:kolla.common.utils:skydive-analyzer Failed with status: matched21:22
panda|offERROR:kolla.common.utils:skydive-base Failed with status: error21:22
panda|off(I can't sleep now)21:22
panda|offall the other succeeded21:22
rlandywow - now we can all dream about kolla errors21:24
panda|offrlandy: dream about rhel containers that are built with only those exceptions21:30
*** Goneri has quit IRC21:33
rlandyI'll let you know on monday21:36
*** rlandy has quit IRC21:36
*** Goneri has joined #oooq21:37
*** Goneri has quit IRC21:46
*** Goneri has joined #oooq21:47
panda|offFINALLY images building in CI too21:47
panda|offI can finally sleep.21:47
*** Goneri has quit IRC21:57
*** ccamacho has joined #oooq21:59
*** vinaykns has quit IRC23:35

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!