Friday, 2019-01-25

*** rkukura has joined #openstack-infra00:02
*** slaweq has joined #openstack-infra00:11
*** slaweq has quit IRC00:15
*** rcernin has quit IRC00:18
*** rcernin has joined #openstack-infra00:19
*** markvoelker has quit IRC00:20
*** ddurst has quit IRC00:30
*** dklyle has quit IRC00:32
*** efried_2dmtg has quit IRC00:35
*** ddurst has joined #openstack-infra00:35
*** rh-jelabarre has joined #openstack-infra00:38
*** dklyle has joined #openstack-infra00:48
*** bgmccollum has quit IRC00:48
*** rlandy has quit IRC00:48
*** wolverineav has quit IRC00:50
*** wolverineav has joined #openstack-infra01:04
*** whoami-rajat has joined #openstack-infra01:06
*** gyee has quit IRC01:10
*** dklyle has quit IRC01:10
*** bobh has joined #openstack-infra01:16
*** bgmccollum has joined #openstack-infra01:17
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Add openstack/ansible-role-borgbackup  https://review.openstack.org/63309501:19
*** bobh has quit IRC01:24
*** bobh has joined #openstack-infra01:26
*** rkukura has quit IRC01:35
*** ijw has quit IRC01:37
*** rkukura has joined #openstack-infra01:43
*** bobh has quit IRC01:50
*** agopi has joined #openstack-infra01:51
*** ijw has joined #openstack-infra01:53
*** rkukura has quit IRC01:55
*** wolverineav has quit IRC01:56
*** ijw has quit IRC01:57
*** armax has quit IRC02:03
*** wolverineav has joined #openstack-infra02:05
*** wolverineav has quit IRC02:11
*** slaweq has joined #openstack-infra02:11
openstackgerritMerged openstack-infra/grafyaml master: Removed older version of python3.4  https://review.openstack.org/62875102:15
*** slaweq has quit IRC02:15
*** markvoelker has joined #openstack-infra02:25
*** hongbin has joined #openstack-infra02:25
*** wolverineav has joined #openstack-infra02:47
*** bhavikdbavishi has joined #openstack-infra02:47
*** eernst has joined #openstack-infra02:50
*** Qianbiao has joined #openstack-infra02:50
*** eernst has quit IRC02:50
Qianbiaogood morning02:50
QianbiaoI am building a CI and testing it on openstack-dev/ci-sandbox now02:51
Qianbiaogerrit review succecced but gerrit approve failed. any idea?02:51
*** bhavikdbavishi has quit IRC02:52
ianwQianbiao: do you mean having your 3rd party ci vote on changes?02:54
Qianbiaoyes02:54
Qianbiaohttps://review.openstack.org/#/c/630893/02:54
Qianbiaoi am testing this patch now02:54
*** markvoelker has quit IRC02:55
Qianbiaoianw02:56
Qianbiaoianw gerrit review works now, but gerrit approve failed02:57
QianbiaoNot sure why. It says that ci-sandbox project is public, not privileges required.02:57
*** chandankumar has quit IRC02:57
Qianbiaoanyone should have permission to gerrit approve02:58
*** chandankumar has joined #openstack-infra02:59
*** bhavikdbavishi has joined #openstack-infra02:59
ianwyes, it should.  it looks like Huawei Ironic CI left a +1 vote on PS602:59
*** agopi has quit IRC03:01
Qianbiaoianw it does not show in workflow label03:03
*** psachin has joined #openstack-infra03:03
ianwQianbiao: it hasn't put a +W ?  i can see just now it left a +1 vote that shows up03:04
Qianbiaoianw not sure why +W means.03:05
Qianbiaoi use ssh -i /var/lib/jenkins/.ssh/id_rsa -p 29418 huawei-ironic-ci@review.openstack.org gerrit review 630893,7 --message 'Build Successful' --verified 1 --code-review 003:05
Qianbiaoit works.03:05
Qianbiaossh -i /var/lib/jenkins/.ssh/id_rsa -p 29418 huawei-ironic-ci@review.openstack.org gerrit approve 630893,7 --message 'Build Successful' --verified 1 --code-review 003:06
Qianbiaoit failed.03:06
*** udesale has joined #openstack-infra03:06
ianwQianbiao: "approve" isn't a thing; we have the "workflow" label, which with sufficient verified +2's triggers zuul to merge it on our behalf03:10
Qianbiaoi see. do u mean if i change --verified 1 to --verified 2,03:12
Qianbiaoit then will appear in workflow label?03:12
*** bobh has joined #openstack-infra03:12
*** armax has joined #openstack-infra03:12
ianwQianbiao: no, you'd have to actually specify the workflow label ... but 3rd party ci's will not be adding workflow votes03:13
ianw3rd party ci would be in a group that can only add +/- 1 verified votes; the final workflow votes (commonly referred to as +W) are done by human core reviewers03:14
*** apetrich has quit IRC03:15
*** whoami-rajat has quit IRC03:15
Qianbiaoianw could u help me checking https://review.openstack.org/#/c/630893/03:16
QianbiaoIs huawei-ironic-ci vote right now?03:16
*** hongbin_ has joined #openstack-infra03:19
*** rh-jelabarre has quit IRC03:20
*** hongbin has quit IRC03:21
ianwQianbiao: sort of, it is correctly adding a +1 verified vote.  however the comment format is slightly wrong, so it will not match the regexes etc that put it in the box with the other ci reports03:23
ianw \* Sandbox-CI http://116.66.187.79/huawei-fc-dsvm-tempest/720/ : SUCCESS03:23
ianwyou've got a leading slash03:23
Qianbiaoi see03:23
Qianbiaoianw what about now03:25
ianwQianbiao: yep, that all looks correct to me03:26
Qianbiaook thanks ianw03:26
QianbiaoIt's very kind of you.03:26
ianwno worries :)03:27
*** armax has quit IRC03:35
*** agopi has joined #openstack-infra03:36
*** dklyle has joined #openstack-infra03:40
*** ramishra has joined #openstack-infra03:41
*** agopi has quit IRC03:43
*** whoami-rajat has joined #openstack-infra03:45
*** agopi has joined #openstack-infra03:46
*** bobh has quit IRC03:51
*** markvoelker has joined #openstack-infra03:52
*** ykarel|away has joined #openstack-infra03:52
*** dklyle has quit IRC04:02
*** bhavikdbavishi has quit IRC04:03
*** slaweq has joined #openstack-infra04:11
*** bhavikdbavishi has joined #openstack-infra04:12
*** slaweq has quit IRC04:16
*** ykarel|away is now known as ykarel04:25
*** markvoelker has quit IRC04:25
*** Bhujay has joined #openstack-infra04:36
*** wolverineav has quit IRC04:43
*** hongbin_ has quit IRC04:43
*** wolverineav has joined #openstack-infra04:46
*** wolverineav has quit IRC04:50
*** wolverineav has joined #openstack-infra04:54
*** udesale has quit IRC05:01
*** udesale has joined #openstack-infra05:04
*** slaweq has joined #openstack-infra05:11
*** slaweq has quit IRC05:16
*** markvoelker has joined #openstack-infra05:22
openstackgerritIan Wienand proposed openstack/diskimage-builder master: pip-and-virtualenv : only remove system files on centos  https://review.openstack.org/63314105:29
*** spsurya has joined #openstack-infra05:40
*** udesale has quit IRC05:45
*** udesale has joined #openstack-infra05:46
*** armax has joined #openstack-infra05:48
*** rkukura has joined #openstack-infra05:54
*** wolverineav has quit IRC05:54
*** markvoelker has quit IRC05:54
*** wolverineav has joined #openstack-infra05:55
*** wolverineav has quit IRC05:59
*** wolverineav has joined #openstack-infra06:01
*** ykarel has quit IRC06:02
*** wolverineav has quit IRC06:06
*** slaweq has joined #openstack-infra06:11
*** slaweq has quit IRC06:15
*** yamamoto has quit IRC06:17
*** yamamoto has joined #openstack-infra06:20
*** kjackal has joined #openstack-infra06:24
*** ykarel has joined #openstack-infra06:26
*** kopecmartin|off is now known as kopecmartin|devc06:27
*** wolverineav has joined #openstack-infra06:35
*** quiquell|off is now known as quiquell06:44
openstackgerritOpenStack Proposal Bot proposed openstack-infra/storyboard master: Imported Translations from Zanata  https://review.openstack.org/62539306:51
*** yamamoto has quit IRC07:08
*** ociuhandu has joined #openstack-infra07:20
*** ociuhandu_ has quit IRC07:20
*** yamamoto has joined #openstack-infra07:21
*** yboaron_ has joined #openstack-infra07:24
*** roman_g has joined #openstack-infra07:29
*** e0ne has joined #openstack-infra07:38
*** pgaxatte has joined #openstack-infra07:40
*** weshay has quit IRC07:47
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: config: add playbooks to job.toDict()  https://review.openstack.org/62134307:48
*** bhavikdbavishi has quit IRC07:50
*** wolverineav has quit IRC07:51
*** markvoelker has joined #openstack-infra07:52
*** soumplis has joined #openstack-infra07:52
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: config: add tenant.toDict() method and REST endpoint  https://review.openstack.org/62134407:55
*** slaweq has joined #openstack-infra08:00
*** quiquell is now known as quiquell|brb08:01
*** panda|off is now known as panda08:04
*** roman_g has quit IRC08:05
*** rpittau has joined #openstack-infra08:10
*** ccamacho has joined #openstack-infra08:13
*** ykarel is now known as ykarel|lunch08:14
*** tkajinam has quit IRC08:15
*** quiquell|brb is now known as quiquell08:19
*** markvoelker has quit IRC08:25
*** shardy has joined #openstack-infra08:25
*** bhavikdbavishi has joined #openstack-infra08:26
*** ginopc has joined #openstack-infra08:45
*** kashyap has joined #openstack-infra08:48
*** tosky has joined #openstack-infra08:49
*** dklyle has joined #openstack-infra08:50
*** shardy has quit IRC08:50
*** jpena|off is now known as jpena08:51
*** shardy has joined #openstack-infra08:51
*** Qianbiao has quit IRC08:54
*** jpich has joined #openstack-infra08:56
*** apetrich has joined #openstack-infra09:01
*** strigazi has quit IRC09:15
*** strigazi has joined #openstack-infra09:16
*** markvoelker has joined #openstack-infra09:22
*** rtjure has quit IRC09:24
*** rtjure has joined #openstack-infra09:26
*** udesale has quit IRC09:29
*** udesale has joined #openstack-infra09:30
*** derekh has joined #openstack-infra09:31
*** roman_g has joined #openstack-infra09:35
*** ssbarnea|bkp2 has joined #openstack-infra09:38
*** ssbarnea|rover has quit IRC09:39
*** ykarel|lunch is now known as ykarel09:40
*** rpittau has quit IRC09:51
*** rpittau has joined #openstack-infra09:53
*** markvoelker has quit IRC09:55
dulekandreaf: Only second +2 is needed on https://review.openstack.org/#/c/626885/ - this will reduce jobs flakiness a lot in Kuryr-Kubernetes gates.09:55
*** ssbarnea|bkp2 has quit IRC09:59
*** ssbarnea|rover has joined #openstack-infra10:00
*** ssbarnea|rover has quit IRC10:01
*** ssbarnea has joined #openstack-infra10:01
openstackgerritJean-Philippe Evrard proposed openstack-infra/zuul-jobs master: Allow different filenames for Dockerfiles  https://review.openstack.org/63297910:02
*** luizbag has joined #openstack-infra10:03
*** roman_g has quit IRC10:03
*** ssbarnea|bkp2 has joined #openstack-infra10:04
*** ssbarnea has quit IRC10:07
tobias-urdininfra-root: requesting a second core review on https://review.openstack.org/#/c/632699/ then we can retest release again10:12
tobias-urdin19:40 < fungi> sudo zuul enqueue-ref --tenant=openstack --trigger=gerrit --pipeline=release --project=openstack/puppet-aodh --ref=refs/tags/14.2.0 --newrev=617ffad84b633618490ca1023f8a31d9694b31a910:12
*** ramishra has quit IRC10:15
*** shardy has quit IRC10:19
*** shardy has joined #openstack-infra10:20
*** yamamoto has quit IRC10:23
openstackgerritMatthieu Huin proposed openstack-infra/zuul-jobs master: install-nodejs: add support for RPM-based OSes  https://review.openstack.org/63104910:38
*** ramishra has joined #openstack-infra10:39
*** markvoelker has joined #openstack-infra10:52
*** shardy has quit IRC10:55
*** jaosorior has quit IRC11:07
*** ociuhandu has quit IRC11:07
*** yamamoto has joined #openstack-infra11:09
*** shardy has joined #openstack-infra11:10
*** yamamoto has quit IRC11:14
*** shardy has quit IRC11:15
*** shardy has joined #openstack-infra11:17
*** shardy has quit IRC11:22
*** markvoelker has quit IRC11:25
*** bhavikdbavishi has quit IRC11:33
*** shardy has joined #openstack-infra11:35
*** shardy has quit IRC11:39
*** shardy has joined #openstack-infra11:53
*** e0ne has quit IRC11:59
*** yamamoto has joined #openstack-infra12:05
*** kjackal has quit IRC12:10
*** smarcet has joined #openstack-infra12:10
*** yamamoto has quit IRC12:12
*** wolverineav has joined #openstack-infra12:28
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Improve logging around ansible timeouts  https://review.openstack.org/63319112:31
*** wolverineav has quit IRC12:32
*** smarcet has quit IRC12:33
*** zaneb has quit IRC12:34
*** zbitter has joined #openstack-infra12:34
*** kjackal has joined #openstack-infra12:35
*** smarcet has joined #openstack-infra12:36
*** e0ne has joined #openstack-infra12:41
*** quiquell is now known as quiquell|food12:41
*** gcutrini has joined #openstack-infra12:42
*** jpena is now known as jpena|lunch12:44
*** smarcet has quit IRC12:45
*** kashyap has left #openstack-infra12:50
*** whoami-rajat has quit IRC12:55
*** bhavikdbavishi has joined #openstack-infra13:01
*** kjackal has quit IRC13:02
*** yamamoto has joined #openstack-infra13:03
*** gcutrini has quit IRC13:07
stephenfinclarkb, fungi: How does one cut a new release of git-review? I suspect one is long overdue13:08
*** rh-jelabarre has joined #openstack-infra13:11
*** yamamoto has quit IRC13:11
evrardjpcorvus: still in the docker publishing topic: https://review.openstack.org/#/c/632979/ :D13:12
*** mriedem has joined #openstack-infra13:12
evrardjpstephenfin: good question, and should we move this to being released by releases?13:13
stephenfinevrardjp: I'm not 100% sure what that would involve but it makes sense from a high level alright, yes13:13
evrardjpyes, I am just asking the questions here :)13:14
evrardjpI don't know either, but it's maybe worth having a look if we are forgetting these13:14
*** ccamacho has quit IRC13:14
*** armstrong has joined #openstack-infra13:17
*** smarcet has joined #openstack-infra13:21
*** smarcet has quit IRC13:26
*** quiquell|food is now known as quiquell13:27
*** ccamacho has joined #openstack-infra13:29
*** rcernin has quit IRC13:30
*** rlandy has joined #openstack-infra13:31
*** eharney has joined #openstack-infra13:44
*** jpena|lunch is now known as jpena13:49
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Make setup playbook timeout configurable  https://review.openstack.org/63320613:54
AJaegerpabelanger: could you look at https://review.openstack.org/632699 , please?13:55
*** bhavikdbavishi has quit IRC13:56
dmsimardGerrit 2.16.4 is out: https://groups.google.com/forum/#!topic/repo-discuss/JSJxPsd5fsc13:57
*** trown|outtypewww is now known as trown13:57
*** kgiusti has joined #openstack-infra14:00
*** weshay has joined #openstack-infra14:06
*** ramishra has quit IRC14:07
*** yamamoto has joined #openstack-infra14:08
Shrewsinfra-root, i plan to upgrade our nodepool builders (and possibly launchers) today, starting around 3 hours from now.14:11
mordreddmsimard: nice. we need to get ourselves upgraded to 2.15 so that we can upgrade to 2.1614:12
mordredShrews: ++14:12
dmsimarddat polymer UI though14:13
Shrewsinfra-root, that should give us an new dib build timeout and port cleanup14:13
mordredShrews: ah - we re-landed the port cleanup patch?14:15
mordredShrews: fingers crossed!14:15
Shrewsmordred: yeah14:15
*** kgiusti has left #openstack-infra14:20
openstackgerritMerged openstack-infra/project-config master: Update base-test jobs to publish artifacts too  https://review.openstack.org/63104514:25
*** kgiusti has joined #openstack-infra14:26
*** jcoufal has joined #openstack-infra14:28
*** Bhujay has quit IRC14:29
*** ekultails has joined #openstack-infra14:43
*** whoami-rajat has joined #openstack-infra14:46
*** smarcet has joined #openstack-infra14:46
*** dave-mccowan has joined #openstack-infra14:46
*** e0ne has quit IRC14:50
*** dave-mccowan has quit IRC14:51
*** psachin has quit IRC14:53
*** spsurya has quit IRC15:07
*** yamamoto has quit IRC15:12
*** dklyle has quit IRC15:22
*** ykarel is now known as ykarel|away15:22
tobiashyeah, that port-cleanup should work now ;)15:23
*** ykarel|away has quit IRC15:29
*** spsurya has joined #openstack-infra15:30
*** yamamoto has joined #openstack-infra15:35
*** betherly has joined #openstack-infra15:35
*** betherly has quit IRC15:36
*** betherly has joined #openstack-infra15:36
*** kmarc_ is now known as kmarc15:42
*** bnemec has quit IRC15:43
*** ramishra has joined #openstack-infra15:44
*** cmurphy is now known as cmorpheus15:51
*** ykarel|away has joined #openstack-infra15:52
*** e0ne has joined #openstack-infra15:55
*** quiquell is now known as quiquell|off15:56
*** udesale has quit IRC15:56
*** pgaxatte has quit IRC15:57
*** efried has joined #openstack-infra16:02
*** dklyle has joined #openstack-infra16:03
*** wolverineav has joined #openstack-infra16:04
*** efried is now known as efried_mtg16:05
*** ramishra has quit IRC16:08
*** wolverineav has quit IRC16:08
*** yamamoto has quit IRC16:08
*** yamamoto has joined #openstack-infra16:13
*** rkukura has quit IRC16:13
*** bhavikdbavishi has joined #openstack-infra16:14
*** yamamoto has quit IRC16:17
corvusevrardjp: nice!  i made a suggestion about the var name, otherwise lgtm16:18
*** bhavikdbavishi has quit IRC16:18
*** adriancz has quit IRC16:21
*** kjackal has joined #openstack-infra16:23
*** panda is now known as panda|off16:23
clarkbstephenfin: evrardjp there should be a release group for git-review that can push tags. We've avoided using the openatack release tooling because we often are a chicken and egg for them with our tools16:32
clarkbif infra breaks releases we need to get fix released independently16:32
*** wolverineav has joined #openstack-infra16:34
stephenfinclarkb: Gotcha. In that case, how does someone outside this group request a release? Just ask nicely?16:36
clarkbyup, and suggestion for version based on included changes is helpful.16:37
Shrewsclarkb: corvus: seems the new nodepool code has propogated to the builders, but i still see a version of openstacksdk in 'pbr freeze' that is less than the new requirements.txt version.16:37
clarkbShrews: is that for python3?16:38
stephenfinclarkb: I'd suggest 2.0, given commit 0376883 removes a feature (the auto branch naming based on 'Closes-Bug' commit message footers)16:38
clarkbI do python3 /usr/local/bin/pbr freeze to be extra sure16:38
Shrewsclarkb: oh, that's probably it. lemme check16:38
corvusor pip3 freeze might be good enough16:39
*** e0ne has quit IRC16:39
Shrewsyup, both of those report the correct thing16:39
Shrewscan't wait until py2 is no longer a thing16:40
Shrewsor we rewrite in c++  :)16:40
clarkbstephenfin: hrm I thought we were going to avoid big changes like that but will have to refresh memory on that one. iirc mordred was in favor16:41
Shrewsinfra-root, ok, gonna restart the builder on nb03 first since it seems idle16:41
stephenfinclarkb: Indeed. I think the main argument was this was functionality that went above and beyond what git-review was supposed to do (be a simple tool to work with Gerrit) and shouldn't have ever existed16:42
stephenfinIt also only supported LP. The alternative was extending it further to support StoryBoard16:42
clarkbstephenfin: and we'll probably cut releasemonday? I expect today (friday) is bad day for a release16:43
clarkbstephenfin: rgr16:43
stephenfinAye, good call16:43
corvusyeah, let's observe the friday moritorium on git-review releases16:43
corvusclarkb, stephenfin: given clarkb's surprise -- do we want to maybe circulate a changelog to the infra list and ask folks if we're okay with a release, or if anyone wants to panic about things like closes-bug?16:44
stephenfincorvus: Sure, I can do that16:44
corvus(the corollary to most of us not expecting big changes is that if any happen, most of us won't have been looking :)16:45
clarkbcorvus: probably a good idea. I recall the change to generally be well suplorted in review, but viewing it in context of release versioning may change opinions16:45
*** bhavikdbavishi has joined #openstack-infra16:46
corvuspersonally, i'm okay with it and buy the argument it's difficult to support well and is therefore a reasonable "big change" to make.16:46
*** jcoufal has quit IRC16:48
Shrewsclarkb: any issue if i manually kick off a build of debian-stretch-arm64 ?16:49
Shrewsjust want to watch nb03 do something16:49
*** redrobot has quit IRC16:51
clarkbshouldnt be16:51
*** betherly has quit IRC16:53
*** kjackal has quit IRC16:54
Shrewshrm, something is wrong. fun16:54
*** rpittau has quit IRC16:54
stephenfindone16:55
Shrewsand all i get in the build log is "Exit code: 1"16:55
*** ginopc has quit IRC16:55
*** jcoufal has joined #openstack-infra16:56
*** kjackal has joined #openstack-infra16:59
*** dklyle has quit IRC17:00
*** diablo_rojo has joined #openstack-infra17:01
Shrewsdiskimage_builder.element_dependencies.MissingElementException: Element 'openstack-repos' not found17:03
Shrews??17:03
*** derekh has quit IRC17:05
pabelangerShrews: that will be in project-config, path should be added for nodepool-builder in /etc/defaults/nodepool IIRC17:05
openstackgerritMerged openstack-infra/zuul master: Explicitly callout ZooKeeper as ext dependency  https://review.openstack.org/63273217:06
pabelangerShrews: sorry, in /etc/nodepool/elements17:07
Shrewspabelanger: thx17:07
*** dklyle has joined #openstack-infra17:07
pabelangerhttp://git.openstack.org/cgit/openstack-infra/puppet-nodepool/tree/manifests/init.pp#n163 is puppet code17:07
*** mriedem is now known as mriedem_afk17:10
openstackgerritJan Kundrát proposed openstack-infra/zuul master: Incoming connections over 19885/TCP are needed on nodes  https://review.openstack.org/63324217:11
*** gyee has joined #openstack-infra17:11
Shrewshrm, for some reason, disk-image-create isn't even writing out a log file17:12
*** mrhillsman is now known as mrhillsman_lunch17:13
pabelangerpath issue?17:14
Shrewspabelanger: i don't think so?17:18
andreafdulek: uhm, I have very little context to approve that, but if the kuryr gate shows mem utilisation is ok I guess it should be fine17:19
*** dklyle has quit IRC17:19
andreafdulek: since frickler had a comment on it, it would be good to get his ok17:19
Shrewspabelanger: running the command by hand and it works17:20
*** kjackal has quit IRC17:20
*** kjackal has joined #openstack-infra17:23
pabelangerShrews: can you see the diskimage-create being called in popen? does stdout=subprocess.PIPE some how imply a shell?17:25
pabelangerI'm not to familiar with subprocess sadly17:25
openstackgerritMerged openstack-infra/zuul master: Make setup playbook timeout configurable  https://review.openstack.org/63320617:26
openstackgerritMerged openstack-infra/zuul master: Improve logging around ansible timeouts  https://review.openstack.org/63319117:26
*** jcoufal has quit IRC17:26
Shrewspabelanger: maybe that has something to do with it. i can't verify that the process was even started. will look into that17:30
Shrewswe probably should revert that change while i look into this17:32
corvusShrews: i'll be interested to see what we missed in testing :)17:35
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Revert "Add a timeout for the image build"  https://review.openstack.org/63325217:36
*** jcoufal has joined #openstack-infra17:36
Shrewscorvus: yeah17:37
Shrewsit's not a *real* feature unless it's been reverted at least once17:37
*** diablo_rojo has quit IRC17:38
Shrewslet's get that revert in asap so i can restart nb0317:39
Shrewsinfra-root ^^^17:39
corvus+317:39
Shrewsdanke17:39
Shrewsgiven this issue, i don't want to restart launchers today17:41
*** jpena is now known as jpena|off17:44
* Shrews takes the merge wait time opportunity to find lunch17:45
corvusShrews will not go to space today17:46
openstackgerritMerged openstack-infra/zuul master: executor: properly format error exception  https://review.openstack.org/63092817:49
*** jpich has quit IRC17:53
*** Dobroslaw has quit IRC17:56
*** bhavikdbavishi has quit IRC17:57
*** dklyle has joined #openstack-infra18:00
*** ykarel|away has quit IRC18:02
*** ccamacho has quit IRC18:05
*** shardy has quit IRC18:06
*** wolverineav has quit IRC18:13
*** mrhillsman_lunch is now known as mrhillsman18:15
openstackgerritMerged openstack-infra/zuul master: Fix noop job toDict()  https://review.openstack.org/63040918:17
*** trown is now known as trown|lunch18:30
*** wolverineav has joined #openstack-infra18:33
*** whoami-rajat has quit IRC18:34
*** wolverineav has quit IRC18:35
*** wolverineav has joined #openstack-infra18:35
*** ijw has joined #openstack-infra18:40
*** ijw has joined #openstack-infra18:41
*** ijw has quit IRC18:41
*** mriedem_afk is now known as mriedem18:41
*** ijw has joined #openstack-infra18:42
*** ijw has quit IRC18:46
*** ijw has joined #openstack-infra18:56
*** yamamoto has joined #openstack-infra18:57
*** wolverineav has quit IRC19:00
*** yamamoto has quit IRC19:02
*** wolverineav has joined #openstack-infra19:02
*** bnemec has joined #openstack-infra19:02
openstackgerritPaul Vinciguerra proposed openstack-infra/zuul master: configloader.py: Not all jobs have updated attribute.  https://review.openstack.org/63325919:05
*** luizbag has quit IRC19:08
*** bnemec is now known as beekneemech19:11
*** dtroyer has quit IRC19:12
*** dtroyer has joined #openstack-infra19:13
*** e0ne has joined #openstack-infra19:25
*** wolverineav has quit IRC19:33
*** wolverineav has joined #openstack-infra19:34
*** rossella_s has quit IRC19:36
*** electrofelix has quit IRC19:37
*** wolverineav has quit IRC19:38
*** wolverineav has joined #openstack-infra19:39
*** rossella_s has joined #openstack-infra19:41
*** beekneemech has quit IRC19:42
*** smarcet has quit IRC19:43
*** smarcet has joined #openstack-infra19:43
*** betherly has joined #openstack-infra19:45
*** smarcet has quit IRC19:45
*** wolverineav has quit IRC19:45
*** smarcet has joined #openstack-infra19:46
openstackgerritJames E. Blair proposed openstack-infra/zuul master: WIP: support foreign required-projects  https://review.openstack.org/61314319:47
*** armstrong has quit IRC19:47
*** bobh has joined #openstack-infra19:48
openstackgerritPaul Vinciguerra proposed openstack-infra/zuul master: configloader.py: Not all jobs have an .updated attribute.  https://review.openstack.org/63325919:49
*** dklyle has quit IRC19:51
*** raissa has joined #openstack-infra19:54
*** bobh has quit IRC19:56
*** raissa has quit IRC19:57
*** smarcet has quit IRC20:01
*** rkukura has joined #openstack-infra20:01
*** trown|lunch is now known as trown20:01
*** smarcet has joined #openstack-infra20:02
*** smarcet has quit IRC20:03
*** jcoufal has quit IRC20:05
*** dklyle has joined #openstack-infra20:13
openstackgerritTom Barron proposed openstack-infra/project-config master: migrate manila-image-elements to in-tree job  https://review.openstack.org/63241220:15
*** smarcet has joined #openstack-infra20:15
openstackgerritTom Barron proposed openstack-infra/openstack-zuul-jobs master: Remove legacy manila artifacts job  https://review.openstack.org/63236220:16
*** e0ne has quit IRC20:18
*** wolverineav has joined #openstack-infra20:18
*** bobh has joined #openstack-infra20:18
*** wolverineav has quit IRC20:23
*** smarcet has quit IRC20:25
*** zul has quit IRC20:26
*** smarcet has joined #openstack-infra20:27
*** bdodd has quit IRC20:29
*** bdodd has joined #openstack-infra20:30
*** kei-ichi has quit IRC20:31
*** efried_mtg has quit IRC20:31
*** smarcet has quit IRC20:33
*** spsurya has quit IRC20:37
johnsomGerrit seems a bit slow today, both UI and git reviews.  Not sure if I am alone in that or not.20:39
*** smarcet has joined #openstack-infra20:46
*** slaweq has quit IRC20:49
corvusShrews: tell me about this error20:50
*** wolverineav has joined #openstack-infra20:50
corvus2019-01-25 19:36:19,031 INFO nodepool.builder.BuildWorker.0: Logging to /var/log/nodepool/builds/debian-stretch-arm64-0000000107.log20:51
corvus2019-01-25 19:36:20,441 INFO nodepool.builder.BuildWorker.0: DIB failed creating debian-stretch-arm64 (1) (timeout=False)20:51
corvusShrews: is that the error in question?20:51
corvus(cause that seems to have run dib -- there's a legit error in that log file)20:51
clarkbjohnsom: a quick glance at our java monitoring tools does t show any immediate red flags20:52
clarkbjohnsom: any specifics on what is slow?20:52
*** wolverineav has quit IRC20:55
*** dklyle has quit IRC20:55
johnsomJust moving patch to patch via the web site, and then doing a git review seems to take longer than normal to finish20:55
johnsomMaybe it's just some routing issue in my corner of the world.20:55
Shrewscorvus: that was the run i ran by hand20:55
Shrewscorvus: i just noticed a version of dib in /usr/local/lib/python2.7 and one in /usr/local/lib/python3.5. the one in /usr/local/bin seems to be python2 based20:56
Shrewswondering if that has something to do with this20:57
Shrewsthe py3 version is older, actually (2.16.0 vs 2.19.0 in py2)20:59
Shrewshow odd20:59
*** dklyle has joined #openstack-infra20:59
Shrewsyet the py3 version does not appear in PATH20:59
*** dklyle has quit IRC21:13
*** wolverineav has joined #openstack-infra21:14
*** slaweq has joined #openstack-infra21:16
*** slaweq has quit IRC21:21
*** wolverineav has quit IRC21:29
*** wolverineav has joined #openstack-infra21:30
*** wolverineav has quit IRC21:30
*** rlandy has quit IRC21:37
Shrewscorvus: i'm coming up on EOD. recommendations on how to handle nb03? land the revert and restart nodepool-builder?21:38
*** trown is now known as trown|outtypewww21:40
*** ekultails has quit IRC21:41
*** wolverineav has joined #openstack-infra21:52
*** eernst has joined #openstack-infra22:01
*** ijw_ has joined #openstack-infra22:02
corvusShrews: /win 2522:04
corvusgah22:04
corvusShrews: so what does the actual failure look like?22:05
*** ijw has quit IRC22:06
corvusoh i see22:06
Shrewscorvus: surprisingly lacking. what you pasted earlier is all there is22:06
corvusi started the builder and see that it's now attempting and failing to build quickly22:06
Shrewsyep22:06
Shrewsthe lack of info here is VERY frustrating22:06
corvusShrews: oh, maybe we shouldn't remove the stdout pipe22:09
Shrewscorvus: i thought of that, but it did work locally for me22:10
Shrewscorvus: the stdout pipe was the whole reason we needed to use the readline() call though, which blocked22:11
corvusShrews: we could use select/read instead22:11
corvusShrews: if i run "nodepool-builder -d" it appears to work22:11
Shrewsyes, that's how i ran it too22:12
corvuswhich makes me more inclined to think that this is dib refusing to run if there's no stdout22:12
Shrewshmm22:12
corvusthat would explain why testing didn't catch it (we don't really run dib)22:12
corvusthough... we do really run dib in the devstack job, rightL22:12
corvus?22:12
corvuserm... is there still a devstack job?22:14
corvusoh there it is http://logs.openstack.org/52/633252/1/check/nodepool-functional-py35-src/8bc0569/22:14
Shrewsyah, http://git.openstack.org/cgit/openstack-infra/nodepool/tree/devstack/plugin.sh#n464 is the config22:15
corvusoh!22:16
corvuswe don't run it daemonized22:16
corvusso it's the same as what i just did -- nodepool-builder -d22:16
corvusokay, i can't falsify the hypothesis that dib requires something hooked up to stdout, so we need to add that back into the call22:17
Shrewsalrighty. that's a better theory than anything i've had22:17
corvusShrews: if we're certain there won't be any output, we can wait until the process terminates and then do a read().  but, the safest thing would be a select/read(1) loop.22:17
Shrewscorvus: ack. let's revert, restart, and i'll re-look monday22:18
corvusShrews: sounds good.  i can take care of ensuring we restart with the revert if you want to get on with your EOD :)22:18
corvus(i left a comment on the revert and +3d)22:19
Shrewscorvus: that would be much appreciated.22:19
*** ijw_ has quit IRC22:19
*** bobh_ has joined #openstack-infra22:25
*** bobh has quit IRC22:29
openstackgerritMerged openstack-infra/nodepool master: Revert "Add a timeout for the image build"  https://review.openstack.org/63325222:37
*** slaweq has joined #openstack-infra22:38
*** slaweq has quit IRC22:43
*** bobh_ has quit IRC22:54
*** smarcet has quit IRC22:55
*** slaweq has joined #openstack-infra22:57
*** kgiusti has left #openstack-infra23:04
*** slaweq has quit IRC23:05
*** rh-jelabarre has quit IRC23:09
pabelangerdmsimard: ianw: mind a late friday / early saturday review: https://review.openstack.org/633095/ would like to see about working on this over weekend23:11
*** bnemec has joined #openstack-infra23:12
*** dklyle has joined #openstack-infra23:13
*** smarcet has joined #openstack-infra23:15
*** betherly has quit IRC23:16
*** betherly has joined #openstack-infra23:16
*** dklyle has quit IRC23:20
*** kopecmartin|devc is now known as kopecmartin|off23:20
*** wolverineav has quit IRC23:22
*** wolverineav has joined #openstack-infra23:23
*** efried has joined #openstack-infra23:23
*** wolverineav has quit IRC23:27
*** wolverineav has joined #openstack-infra23:27
*** xek has quit IRC23:30
*** betherly has quit IRC23:32
*** eernst has quit IRC23:38
*** smarcet has quit IRC23:39
*** bnemec has quit IRC23:43
*** bnemec has joined #openstack-infra23:44
corvusnb03 restarted with revert in place23:52
*** bnemec has quit IRC23:59
*** bnemec has joined #openstack-infra23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!