Wednesday, 2019-01-09

*** jamesmcarthur has joined #openstack-infra00:02
*** jamesmcarthur has quit IRC00:06
*** rcernin has quit IRC00:07
*** rcernin has joined #openstack-infra00:09
*** yamamoto has joined #openstack-infra00:09
*** rascasoft has quit IRC00:09
*** tosky has quit IRC00:09
*** armax has joined #openstack-infra00:15
openstackgerritClark Boylan proposed openstack-infra/system-config master: Nameservers are now managed with ansible  https://review.openstack.org/62925700:17
clarkbthat should pass tests now00:17
*** slaweq has quit IRC00:18
*** yamamoto has quit IRC00:19
clarkband once that is in we should be able to safely delete the servers as nothing will be trying to talk to it anymore (no more anisble/puppet)00:19
*** rascasoft has joined #openstack-infra00:21
clarkbif anyone is wondering the large spike in yum failures e-r is reporting appear related to kolla being very broken00:26
clarkbgrafana repo isn't installing00:26
dmsimardso golang has a bot called gerritbot which is totally different than our gerritbot: https://github.com/golang/go/wiki/GerritBot00:28
dmsimardit syncs github pull requests to gerrit reviews00:29
dmsimarda random pull request: https://github.com/golang/go/pull/29606 synced to gerrit: https://go-review.googlesource.com/c/go/+/156757 with the github PR link in the commit message00:30
clarkbdmsimard: ya they also had git-review once upon a time00:30
dmsimardonce upon a time ? they're using the ones from infra now ?00:30
clarkbno they renamed theirs to reduce confusion00:31
dmsimardeh00:31
dmsimardfrom a quick glance, it looks like the bot works well ? and it syncs in both directions00:33
*** owalsh_ has joined #openstack-infra00:33
dmsimardthat's kind of neat00:34
fungidepends on the use case00:34
fungiwe had a couple of reasons not to consider that approach00:34
*** rascasoft has quit IRC00:35
fungifirst, legally, the openstack project couldn't accept patches from anyone who hadn't already agreed to their icla00:35
*** slaweq has joined #openstack-infra00:35
dmsimardMakes sense00:36
*** owalsh has quit IRC00:36
clarkbdmsimard: the sha1s don't match up00:37
clarkbthey must be squashing the commits down00:37
dmsimardIf you look at that pull request I sent, there is a bot that applies a "cla:yes" tag so I guess it's a thing :p00:38
dmsimardclarkb: yes, actually I was wondering how they dealt with multiple commits in a PR. Good catch :)00:38
fungiright, they'd still need contributors to separately agree to a cla outside github00:38
clarkband ya the commit message has a pointer back to the github sha1 too, not sure I agree with that design but can see why you might choose it00:39
fungibut second, it looks like they can't sync review comments between gerrit changes and github prw00:39
fungiprs00:39
fungiso the benefit isn't clear to me if people who would rather only understand github still have to follow up in gerrit00:40
clarkbfungi: I think this was a compromise for when there was a big kerfluffle in their community where one side said you can't open source without github and the core reviewers/maintainers said they can't review code effectively without gerrit00:41
fungilooks like it echoes gerrit change comments one-way into github pr comments, but not back00:41
dmsimardI see Gerrit comments in GitHub ? I guess this is one: https://github.com/golang/go/pull/29606#issuecomment-45216897600:41
*** yamamoto has joined #openstack-infra00:41
fungiclarkb: yeah, i'd rather just pick a new term for free software if "open source" means you have to use github00:41
clarkbfungi: we can't use open core that one is already taken now :P00:42
fungiopen mind software00:42
clarkb(doesn't actually fit, but there is an open core conference or something now)00:42
fungidmsimard: yep, it's the other direction which is missing00:42
fungidmsimard: so the gerrit users can proclaim from on high into github, and the github-only users have no other choice but to respond with new commits00:43
fungi(not the end of the world, i agree)00:43
clarkbfungi: but also suggest you might be better of just using gerrit00:43
fungi"Please don’t reply on this GitHub thread. Visit ..."00:43
fungi(over and over and over...)00:44
fungialso apparently it's gerrit bot copying github prs into their gerrit, but gopherbot copying gerrit change comments back to them00:45
clarkbdansmith: some simple attempts to reproduce the weird behavior seem to show that the unified diff may be more reliable than side by side00:45
*** yamamoto has quit IRC00:45
fungiand it doesn't look like changes opened in their gerrit cause equivalent prs to be opened in github, so again github-only users aren't *really* going to be able to follow all their development activity anyway00:46
*** Swami has quit IRC00:46
*** armax has quit IRC00:46
clarkbdansmith: I also have managed to sort out what triggers it00:47
clarkbI've also now switched the render config option to slow to see if that helps at all (there have been reports both ways on that)00:47
*** slaweq has quit IRC00:47
*** rascasoft has joined #openstack-infra00:51
*** sthussey has quit IRC00:51
*** armax has joined #openstack-infra00:56
clarkbdansmith: I don't have much data yet but ^ seems to jump less on large side by side diffs00:59
clarkbI'll run with that for a bit and see if it helps00:59
dansmithclarkb: it's been killing me on specs00:59
clarkbdansmith: ya thats where I noticed it too. The side by side with one pane empty because its a giant new file01:03
*** rascasoft has quit IRC01:03
clarkbwith lots of comments (specifically zuuls distributed scheduelr spec)01:03
mrhillsmano/01:08
mrhillsmani was wondering if there is a general time for when things get reviewed or if i did this right - https://review.openstack.org/#/c/626084/01:09
*** armax has quit IRC01:09
mrhillsmannot sure if this type of change goes into a queue or just was missed with the holidays01:10
*** jesusaur has joined #openstack-infra01:12
clarkbmrhillsman: I think its mostly holidays and catching up on everything01:13
mrhillsman++01:13
*** rascasoft has joined #openstack-infra01:14
*** slaweq has joined #openstack-infra01:18
*** owalsh_ is now known as owalsh01:23
*** slaweq has quit IRC01:23
*** rascasoft has quit IRC01:28
*** rascasoft has joined #openstack-infra01:44
dmsimardfungi: the wiki mentions that it's because code reviews are done exclusively on gerrit01:46
dmsimardIt's another way to send a patch I guess. The kernel review process by email fascinates me :)01:48
*** hongbin_ has joined #openstack-infra01:48
*** xek has quit IRC01:51
dmsimardfungi, clarkb: so would it be as problematic in gitea, then ?01:53
dmsimardI realize that the spec mentions pull requests, wiki and issues would be disabled at first but it seems to leave a door opened01:54
clarkbno I dont think it does02:09
clarkbit says explicitly it doesnt :)02:09
clarkbfwiw I personally dont find the PR workflow to be effective or one I'dwant to support. It is lossy and ambiguous as to how you should curate commits02:11
clarkband syncing between systems is even worse02:11
*** xek has joined #openstack-infra02:12
*** xek has quit IRC02:12
*** xek has joined #openstack-infra02:13
clarkbI think the wiki functionality is the one most people have in mind for potentially adding later02:14
*** rascasoft has quit IRC02:17
dmsimardclarkb: I re-read again, the door is opened for the wiki only, you're correct :p02:21
*** rascasoft has joined #openstack-infra02:31
*** rascasoft has quit IRC02:36
*** rascasoft has joined #openstack-infra02:39
*** bobh has joined #openstack-infra02:41
*** bobh has quit IRC02:47
*** psachin has joined #openstack-infra02:48
*** rascasoft has quit IRC02:48
*** wolverineav has quit IRC02:48
*** wolverineav has joined #openstack-infra02:48
*** whoami-rajat has joined #openstack-infra02:52
*** rascasoft has joined #openstack-infra02:53
*** wolverineav has quit IRC02:54
*** wolverineav has joined #openstack-infra02:55
*** fuentess has quit IRC02:56
*** wolverineav has quit IRC02:57
*** wolverineav has joined #openstack-infra02:59
*** rascasoft has quit IRC03:03
*** wolverineav has quit IRC03:03
*** gyee has quit IRC03:10
*** markvoelker has joined #openstack-infra03:11
*** markvoelker has quit IRC03:12
*** rascasoft has joined #openstack-infra03:15
*** apetrich has quit IRC03:16
*** anteaya has quit IRC03:24
*** rascasoft has quit IRC03:27
*** ramishra has joined #openstack-infra03:29
*** ykarel|away has joined #openstack-infra03:32
openstackgerritMerged openstack/diskimage-builder master: package-installs: provide for skip from env var  https://review.openstack.org/61911903:34
*** rkukura has quit IRC03:36
openstackgerritMerged openstack/diskimage-builder master: simple-init: allow for NetworkManager support  https://review.openstack.org/61912003:38
*** rascasoft has joined #openstack-infra03:39
*** janki has joined #openstack-infra03:45
*** spsurya has joined #openstack-infra03:46
*** yamamoto has joined #openstack-infra03:52
*** rascasoft has quit IRC03:53
*** yamamoto has quit IRC03:54
*** rlandy|rover|bbl is now known as rlandy|rover03:54
*** yamamoto has joined #openstack-infra03:55
*** diablo_rojo has quit IRC04:00
*** rascasoft has joined #openstack-infra04:02
*** yamamoto has quit IRC04:04
*** yamamoto has joined #openstack-infra04:05
*** yamamoto has quit IRC04:12
*** rlandy|rover has quit IRC04:17
*** bhavikdbavishi has joined #openstack-infra04:22
*** hongbin_ has quit IRC04:24
*** udesale has joined #openstack-infra04:27
*** rascasoft has quit IRC04:28
*** markvoelker has joined #openstack-infra04:37
*** markvoelker has quit IRC04:42
*** rascasoft has joined #openstack-infra04:42
*** wolverineav has joined #openstack-infra04:45
*** rascasoft has quit IRC04:52
*** rf0lc0 has joined #openstack-infra04:53
*** hwoarang has joined #openstack-infra04:56
openstackgerritIan Wienand proposed openstack-infra/project-config master: Add Fedora 29 nodes  https://review.openstack.org/61867204:56
openstackgerritIan Wienand proposed openstack-infra/project-config master: Enable NetworkManager for Centos 7  https://review.openstack.org/61996004:56
*** lifeless_ has joined #openstack-infra04:57
*** jbadiapa has joined #openstack-infra04:57
*** rascasoft has joined #openstack-infra04:58
*** wolverineav has quit IRC05:00
*** psachin has quit IRC05:02
*** jesusaur has quit IRC05:02
*** rfolco has quit IRC05:02
*** sshnaidm has quit IRC05:02
*** hwoarang_ has quit IRC05:02
*** irclogbot_1 has quit IRC05:02
*** rcarrillocruz has quit IRC05:02
*** chason has quit IRC05:02
*** lifeless has quit IRC05:02
*** rascasoft has quit IRC05:02
*** irclogbot_1 has joined #openstack-infra05:04
*** auristor has quit IRC05:07
*** jesusaur has joined #openstack-infra05:09
*** sshnaidm has joined #openstack-infra05:09
*** ykarel|away has quit IRC05:10
*** zaro has joined #openstack-infra05:10
*** rascasoft has joined #openstack-infra05:15
*** wolverineav has joined #openstack-infra05:15
*** jpena|off has quit IRC05:22
*** jpena|off has joined #openstack-infra05:23
*** rascasoft has quit IRC05:25
*** ykarel|away has joined #openstack-infra05:26
*** bhavikdbavishi has quit IRC05:26
*** ykarel|away is now known as ykarel05:26
*** bhavikdbavishi has joined #openstack-infra05:27
*** armax has joined #openstack-infra05:27
openstackgerritMerged openstack-infra/irc-meetings master: Consolidate openstack-meeting channels  https://review.openstack.org/62618205:29
*** udesale has quit IRC05:32
*** roman_g has quit IRC05:32
*** auristor has joined #openstack-infra05:36
*** udesale has joined #openstack-infra05:40
*** rascasoft has joined #openstack-infra05:48
*** rascasoft has quit IRC05:53
openstackgerritMerged openstack-infra/nodepool master: Add Fedora 29 testing  https://review.openstack.org/61867105:54
*** markvoelker has joined #openstack-infra06:04
*** rkukura has joined #openstack-infra06:17
*** e0ne has joined #openstack-infra06:22
*** e0ne has quit IRC06:23
*** ramishra has quit IRC06:47
*** ramishra has joined #openstack-infra06:47
*** wolverineav has quit IRC07:04
*** wolverineav has joined #openstack-infra07:05
*** pcaruana has joined #openstack-infra07:06
*** wolverineav has quit IRC07:09
*** quiquell|off is now known as quiquelll07:09
*** quiquelll is now known as quiquell07:09
*** udesale has quit IRC07:17
*** e0ne has joined #openstack-infra07:18
*** pgaxatte has joined #openstack-infra07:22
*** e0ne has quit IRC07:22
*** udesale has joined #openstack-infra07:22
*** kjackal has joined #openstack-infra07:23
*** rcernin has quit IRC07:26
*** udesale has quit IRC07:27
*** udesale has joined #openstack-infra07:29
*** ykarel is now known as ykarel|lunch07:29
*** rascasoft has joined #openstack-infra07:36
*** bobh has joined #openstack-infra07:39
AJaegerianw: I gave https://review.openstack.org/618672 another +W now to merge...07:40
AJaegerianw: can we merge 619960 already or does it need some release/restarts?07:40
*** bobh has quit IRC07:43
*** e0ne has joined #openstack-infra07:47
*** armax has quit IRC07:49
openstackgerritMerged openstack-infra/project-config master: Add Fedora 29 nodes  https://review.openstack.org/61867207:51
amorinhey all07:53
*** rpittau has joined #openstack-infra07:54
*** lpetrut has joined #openstack-infra07:55
amorinclarkb: I saw your message about quota on BHS107:57
amorinI will check07:57
*** quiquell is now known as quiquell|brb08:00
openstackgerritTristan Cacqueray proposed openstack-infra/infra-specs master: Add anomaly detection in CI/CD jobs specification draft  https://review.openstack.org/58121408:04
*** gfidente has joined #openstack-infra08:06
*** slaweq has joined #openstack-infra08:13
*** bhavikdbavishi has quit IRC08:21
*** ginopc has joined #openstack-infra08:22
*** bhavikdbavishi has joined #openstack-infra08:22
*** quiquell|brb is now known as quiquell08:28
*** tosky has joined #openstack-infra08:37
*** eumel8 has joined #openstack-infra08:37
*** apetrich has joined #openstack-infra08:38
amorinclarkb: I upgraded the number of vcpu quota08:41
amorinshould be better08:41
amorinI also resync the quota08:43
*** dkehn has quit IRC08:46
*** e0ne has quit IRC08:47
*** jpena|off is now known as jpena08:52
*** quiquell has quit IRC08:53
*** quiquell has joined #openstack-infra08:54
*** quiquell has quit IRC08:54
*** quiquell has joined #openstack-infra08:56
*** yamamoto has joined #openstack-infra08:56
*** yamamoto has quit IRC08:57
*** jpich has joined #openstack-infra09:00
*** ccamacho has joined #openstack-infra09:16
*** kjackal has quit IRC09:22
*** kjackal has joined #openstack-infra09:22
openstackgerritNatal Ngétal proposed openstack/diskimage-builder master: [Configuration] Switch to stestr.  https://review.openstack.org/62941409:25
*** bobh has joined #openstack-infra09:25
*** yamamoto has joined #openstack-infra09:27
*** yamamoto_ has joined #openstack-infra09:27
*** hrubi has joined #openstack-infra09:28
*** yboaron has joined #openstack-infra09:29
*** yamamoto has quit IRC09:31
*** markvoelker has quit IRC09:33
*** yamamoto_ has quit IRC09:33
AJaegerssbarnea: regarding https://review.openstack.org/629205 - note that I'm not a core on devstack-gate09:36
ssbarneaAJaeger: these days it seems hard to find reviewers on that repo :p09:37
*** kjackal has quit IRC09:38
ssbarneai know is mostly in deprecation mode but is still used and i face issues on real jobs that originate from there, so we should fix them. any idea who could help?09:38
*** kjackal has joined #openstack-infra09:38
*** derekh has joined #openstack-infra09:38
ssbarneai asked on #openstack-qa but no answers yet09:39
*** ykarel|lunch is now known as ykarel09:39
*** zhangfei has joined #openstack-infra09:43
AJaegerssbarnea: sorry, no ideas who can help. Here and #openstack-qa seems best place indeed.09:43
openstackgerritTobias Henkel proposed openstack-infra/zuul master: WIP: Add spec for scale out scheduler  https://review.openstack.org/62147909:46
*** ramishra_ has joined #openstack-infra09:56
*** ramishra has quit IRC09:57
*** bhavikdbavishi has quit IRC10:00
* iurygregory be back in a few, going to other building =)10:05
*** dtantsur|afk is now known as dtantsur10:10
*** iurygregory has quit IRC10:12
*** panda|off is now known as panda10:18
*** bobh has quit IRC10:19
*** e0ne has joined #openstack-infra10:21
*** jaosorior has quit IRC10:22
*** zhangfei has quit IRC10:32
openstackgerritNatal Ngétal proposed openstack/diskimage-builder master: [Configuration] Switch to stestr.  https://review.openstack.org/62941410:38
openstackgerritNatal Ngétal proposed openstack/diskimage-builder master: [Core] Change openstack-dev to openstack-discuss.  https://review.openstack.org/62289510:40
openstackgerritNatal Ngétal proposed openstack/diskimage-builder master: [Configuration] Add missing py37 and corrected default envlist.  https://review.openstack.org/62467010:41
*** ldnunes has joined #openstack-infra10:43
*** electrofelix has joined #openstack-infra10:58
*** jaosorior has joined #openstack-infra10:59
openstackgerritWill Szumski proposed openstack-dev/pbr master: Fix white space handling in file names  https://review.openstack.org/62916111:03
*** pbourke has quit IRC11:10
*** pbourke has joined #openstack-infra11:12
openstackgerritThierry Carrez proposed openstack-infra/system-config master: Remove meeting-5 and meeting-cp from meetbot  https://review.openstack.org/62943911:16
*** jamesmcarthur has joined #openstack-infra11:17
*** jamesmcarthur has quit IRC11:21
*** yamamoto has joined #openstack-infra11:22
*** yamamoto has quit IRC11:23
*** yamamoto has joined #openstack-infra11:23
openstackgerritMerged openstack-infra/irc-meetings master: Update count_slot_usage for new recurrences  https://review.openstack.org/62565611:24
openstackgerritMerged openstack-infra/irc-meetings master: Add count_slot_usage argument for sensitivity  https://review.openstack.org/62565711:24
*** udesale has quit IRC11:24
openstackgerritWill Szumski proposed openstack-dev/pbr master: Do not globally replace path prefix  https://review.openstack.org/62900611:32
*** chandankumar is now known as chkumar|out11:39
*** rtjure has quit IRC11:40
openstackgerritWill Szumski proposed openstack-dev/pbr master: Fix white space handling in file names  https://review.openstack.org/62916111:47
*** dpawlik has joined #openstack-infra11:48
*** ykarel is now known as ykarel|afk11:58
*** rtjure has joined #openstack-infra12:00
*** jpena is now known as jpena|lunch12:08
*** jangutter has joined #openstack-infra12:12
*** wolverineav has joined #openstack-infra12:14
jangutterWeird question: I submitted a review, and openstack-infra updated the link to the review with a broken link on a blueprint. Did I do something stupid?12:14
jangutterThe review was  https://review.openstack.org/57132512:15
jangutterIt added: Gerrit topic: https://review.openstack.org/#q,topic:bp/vrouter-os-vif-conversion,n,z where it should have added https://review.openstack.org/#/q/topic:bp/vrouter-os-vif-conversion,n,z12:15
jangutterIt picked the correct topic, but it generated a URL where "#q," should be "#/q/"12:15
*** iurygregory has joined #openstack-infra12:18
*** bobh has joined #openstack-infra12:18
jangutterI think it's this: https://github.com/openstack-infra/jeepyb/blob/master/jeepyb/cmd/update_blueprint.py#L9212:19
*** wolverineav has quit IRC12:19
*** rpittau is now known as rpittau|lunch12:20
*** bobh has quit IRC12:24
*** udesale has joined #openstack-infra12:28
*** yamamoto_ has joined #openstack-infra12:29
*** e0ne has quit IRC12:30
*** yamamoto has quit IRC12:33
*** AJaeger has quit IRC12:43
*** ykarel|afk is now known as ykarel12:46
*** yfried has joined #openstack-infra12:47
yfriedHi, is this the right place for pbr questions?12:47
*** rf0lc0 is now known as rfolco12:49
*** AJaeger has joined #openstack-infra12:51
fungiyfried: #openstack-oslo is probably slightly more correct, but people here might know the answer too sure12:53
*** yamamoto_ has quit IRC12:54
*** rlandy has joined #openstack-infra12:55
*** yamamoto has joined #openstack-infra12:55
*** rlandy is now known as rlandy|rover12:56
*** yamamoto has quit IRC12:57
fungissbarnea: devstack-gate is an infra project, core reviewers on it are basically https://review.openstack.org/#/admin/groups/infra-core (plus sdague but he's not been around for a while)12:58
*** fuentess has joined #openstack-infra13:00
yfriedfungi: tnx. I need to package all *.j2 files recursively. Do you know if pbr "data-files" can handle this? Do I need a MANIFEST.in file?13:00
fungijangutter: where on that review are you seeing the broken link? in the topic field of the detail box or somewhere else?13:03
jangutterfungi: I'm seeing it on the blueprint whiteboard: https://blueprints.launchpad.net/nova/+spec/vrouter-os-vif-conversion (last line)13:04
*** jpena|lunch is now known as jpena13:04
jangutterfungi: I'm trying to find out where the '#/q/' url bit comes from and the best I could come up with is https://review.openstack.org/Documentation/config-gerrit.html#commentlink13:04
jangutterfungi: pretty hard to google arbitrary weird junk.13:05
*** ykarel is now known as ykarel|afk13:06
fungiyfried: i do not know whether the data files configuration can take directories rather than individual files, though i expect that's up to setuptools not pbr itself13:06
openstackgerritNatal Ngétal proposed openstack/diskimage-builder master: [Configuration] Switch to stestr.  https://review.openstack.org/62941413:07
*** ykarel|afk is now known as ykarel13:07
*** smarcet has joined #openstack-infra13:08
yfriedfungi: in MANIFEST there's a "recursive-include" directive. I didn't see an equivalent in data-files13:08
*** trown|outtypewww is now known as trown13:09
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: Default host_keys to empty list in static driver  https://review.openstack.org/62947813:11
*** yfried has quit IRC13:12
fungijangutter: it looks like https://git.openstack.org/cgit/openstack-infra/jeepyb/tree/jeepyb/cmd/update_blueprint.py#n92 was never updated after the gerrit upgrade which invalidated the old url format13:14
jangutterfungi: That line lasted 6 years!13:15
fungiindeed13:16
jangutterfungi: I've been trying to find where the url format is documented....13:16
*** jcoufal has joined #openstack-infra13:16
fungithere was a non-backwards-compatible change in gerrit's url format around 2.11 or 2.1213:16
fungiunfortunately they never treated things like urls as a stable api13:17
*** dave-mccowan has joined #openstack-infra13:19
jangutterfungi: one would have thought a permalink would be a nice feature, sadly.13:19
*** rpittau|lunch is now known as rpittau13:21
*** e0ne has joined #openstack-infra13:24
*** yamamoto has joined #openstack-infra13:28
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Fixes from php7.2 migration  https://review.openstack.org/62948013:28
openstackgerritMerged openstack-infra/openstackid-resources master: Fixes from php7.2 migration  https://review.openstack.org/62948013:30
ssbarneafungi: thanks for pointing me to the gerrit group. probably i should print the membership to have it handy ;)13:38
fungissbarnea: when in doubt, if you click the gear to the right of the project name when looking at a review in the gerrit webui (or go to projects->list at the top of the gerrit webui and search for the project), then switch to the access tab at the top you should hopefully see a group set for "Label Code-Review" with a -2 to +2 range; click that group name and you'll be on the membership list view for13:41
fungiit13:42
fungithat's how i usually go about finding out who to bug for reviews on something ;)13:42
*** dkehn has joined #openstack-infra13:49
*** jamesmcarthur has joined #openstack-infra13:50
*** ramishra has joined #openstack-infra13:50
*** whoami-rajat has quit IRC13:52
*** ramishra_ has quit IRC13:52
*** yamamoto has quit IRC13:54
mordredAJaeger: https://review.openstack.org/#/c/629193/1/ -- very broken patch - but grenade-py3 is green - I think this means grenade-py3 isn't actually testing openstacksdk :)13:56
openstackgerritDavid Moreau Simard proposed openstack-infra/project-config master: Remove noop job for openstack/ara-web  https://review.openstack.org/62948913:59
openstackgerritsebastian marcet proposed openstack-infra/openstack-zuul-jobs master: Updated laravel legacy jobs  https://review.openstack.org/62949514:08
fungimordred: seeing as how you're around, what's the best way to get the bindep role running after the ppa addition task in https://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/playbooks/legacy/laravel-openstackid-unittests/run.yaml ? move it from the roles list to an include_role task in the appropriate part of the tasks list?14:11
mordredfungi: ah - the ever popular problem of "what's the best way to deal with bindep+ppa" - which honestly doesnt' have a good answer14:21
*** kgiusti has joined #openstack-infra14:21
mordredfungi: my current take, which is more of giving up than a real answer is that things that need a PPA or other alternate repo to be installed shouldn't be in bindep.txt14:22
mordred(thus the "install_js_tools.sh" in the zuul repo)14:22
mordredfungi: but what I *REALLY14:22
fricklermordred: seems we would need something like a grenade-tips job that includes some proper LIBS_FROM_GIT list? but certainly you have just proven that we can drop grenade-py3, as that doesn't stop broken patches at all14:23
mordredfungi: but what I *REALLY* want is to add ppa support to bindep so that the whole thing could be self-contained14:23
*** boden has joined #openstack-infra14:24
smarcetfungi: mordred: morning guys please review https://review.openstack.org/#/c/629495/, thx14:25
mordredfrickler: yah. and grenade-py3 is still a legacy job, so I can't just pop a sensible override into the pipeline definition or aything14:25
*** jamesmcarthur has quit IRC14:26
*** janki has quit IRC14:26
*** janki has joined #openstack-infra14:27
*** jamesmcarthur has joined #openstack-infra14:27
*** yamamoto has joined #openstack-infra14:27
mordredfungi: oh - but yeah, to answer your actual question, move it to include_role:14:28
fungimordred: cool, so what smarcet did in 629495 i guess?14:29
mordredya14:29
fungismarcet: you're aware you can use "Depends-On: https://review.openstack.org/629495" in an openstackid change commit message to test whether that works, right?14:30
smarcetfungi: yes sire14:30
smarceti will14:30
*** whoami-rajat has joined #openstack-infra14:31
* fungi hasn't been called "sire" since his reactor days ;)14:31
fricklermordred: oh, indeed. so someone needs to take up andreaf's wip here https://review.openstack.org/548936 . tosky: do you still intend to continue with that? I could try to help a bit14:31
fungier, re-enactor14:31
*** yamamoto has quit IRC14:33
openstackgerritMerged openstack-infra/nodepool master: Default host_keys to empty list in static driver  https://review.openstack.org/62947814:34
openstackgerritsebastian marcet proposed openstack-infra/openstackid master: Migration to PHP 7.x  https://review.openstack.org/61193614:35
*** eernst has quit IRC14:40
openstackgerritMerged openstack-infra/zuul master: Clean up command sockets on stop  https://review.openstack.org/62899014:40
*** ykarel is now known as ykarel|away14:42
openstackgerritMerged openstack-infra/zuul master: Ensure command_socket is last thing to close  https://review.openstack.org/62899514:43
*** janki has quit IRC14:47
*** janki has joined #openstack-infra14:47
*** janki has quit IRC14:51
*** janki has joined #openstack-infra14:52
*** janki has quit IRC14:53
*** janki has joined #openstack-infra14:53
*** janki has quit IRC14:54
*** anteaya has joined #openstack-infra14:55
*** janki has joined #openstack-infra14:55
*** janki has quit IRC14:57
*** janki has joined #openstack-infra14:57
*** shardy has joined #openstack-infra14:58
*** roman_g has joined #openstack-infra14:59
*** janki has quit IRC14:59
dtantsurfolks, anyone knows elastic-search here? how do I link a query to a story?15:00
openstackgerritDmitry Tantsur proposed openstack-infra/elastic-recheck master: Create a query for story 2004744  https://review.openstack.org/62953315:00
dtantsurcontext ^^^ /cc TheJulia15:00
*** yboaron_ has joined #openstack-infra15:02
*** yamamoto has joined #openstack-infra15:03
*** arxcruz is now known as arxcruz|ruck15:03
*** rlandy|rover is now known as rlandy15:04
*** yboaron has quit IRC15:05
*** ykarel|away is now known as ykarel15:06
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Create base-ozj base job  https://review.openstack.org/58716515:07
mordredAJaeger: ^^ starting to work through the base-minimal-jobs stack15:07
*** yamamoto has quit IRC15:08
TheJuliadtantsur: thanks!15:08
*** chason_ has joined #openstack-infra15:09
AJaegermordred: ok.15:10
*** ssbarnea has quit IRC15:10
*** ssbarnea|rover has joined #openstack-infra15:10
openstackgerritMerged openstack-infra/project-config master: Remove noop job for openstack/ara-web  https://review.openstack.org/62948915:11
odyssey4mehi folks - on dec 6 I ran https://review.openstack.org/#/c/623253/ and it worked just fine (it's a cross-repo test), but now it's failing with http://logs.openstack.org/53/623253/1/check/openstack-ansible-deploy-aio_ceph-ubuntu-bionic/c5b5b9a/job-output.txt.gz#_2019-01-08_14_55_35_077278 (a very descriptive MODULE FAILURE) :/15:15
odyssey4meany ideas what may be causing that?15:15
odyssey4meit's failing in this play: https://github.com/openstack/openstack-ansible/blob/master/zuul.d/playbooks/run.yml15:16
pabelangerodyssey4me: http://logs.openstack.org/53/623253/1/check/openstack-ansible-deploy-aio_ceph-ubuntu-bionic/c5b5b9a/ara-report/result/b42c17fc-2cce-46ce-b036-7dfaface5611/15:20
*** ykarel is now known as ykarel|away15:21
odyssey4mepabelanger lol, well - that'd do it15:21
pabelanger/home/zuul/src/git.openstack.org/openstack/openstack-ansible-ceph_client was only created15:21
pabelangermissing required-project?15:21
odyssey4mepabelanger that may be - I'll add it to https://github.com/openstack/openstack-ansible/blob/master/zuul.d/jobs.yaml#L17 and see if that works15:22
odyssey4methanks!15:22
*** sara_nierodz has joined #openstack-infra15:23
fricklerdtantsur: I think that this is a feature that hasn't been implemented yet15:23
dtantsursigh15:23
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Delete current base-minimal job  https://review.openstack.org/58716915:24
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Reset base-test playbooks  https://review.openstack.org/58710415:24
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Add base-minimal / base-minimal-test playbooks  https://review.openstack.org/58710515:24
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Parent base-test to base-minimal-test  https://review.openstack.org/58710615:24
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Parent base to base-minimal  https://review.openstack.org/58710715:24
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Remove base-test now defined in openstack-zuul-jobs  https://review.openstack.org/58711315:24
*** weshay|ruck is now known as weshay15:24
*** sara_nierodz is now known as snierodz_15:24
fricklerdtantsur: you could start by creating a story for that at https://storyboard.openstack.org/#!/project/713 ;)15:25
*** ykarel|away has quit IRC15:26
openstackgerritMonty Taylor proposed openstack-infra/openstack-zuul-jobs master: Parent to base-ozj for testing  https://review.openstack.org/58716815:27
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Fixed issue with LV 5.6 upgrade  https://review.openstack.org/62954315:28
openstackgerritMerged openstack-infra/openstackid-resources master: Fixed issue with LV 5.6 upgrade  https://review.openstack.org/62954315:28
mordredfrickler, fungi: feel like +Aing https://review.openstack.org/#/c/587165 ?15:28
*** shardy has quit IRC15:29
*** shardy has joined #openstack-infra15:29
mordred(it should be a very safe change as it doesn't do anything yet - main thing is that base-ozj should be identical to the current base-minimal)15:29
dtantsurfrickler: https://storyboard.openstack.org/#!/story/2004745 :)15:29
*** udesale has quit IRC15:34
*** dpawlik has quit IRC15:35
*** yamamoto has joined #openstack-infra15:36
*** lpetrut has quit IRC15:37
openstackgerritMerged openstack-infra/zuul master: Document missing executor stats  https://review.openstack.org/62674915:41
*** armax has joined #openstack-infra15:41
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Enforce max-job-timeout for jobs without configured timeout  https://review.openstack.org/62955215:42
*** yamamoto has quit IRC15:42
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Updated laravel legacy jobs  https://review.openstack.org/62949515:43
openstackgerritDavid Moreau Simard proposed openstack-infra/zuul-jobs master: Add support for CentOS and RHEL for the install-nodejs role  https://review.openstack.org/62955415:45
*** quiquell is now known as quiquell|off15:45
openstackgerritNatal Ngétal proposed openstack/diskimage-builder master: [Configuration] Switch to stestr.  https://review.openstack.org/62941415:47
openstackgerritMerged openstack-infra/project-config master: Create base-ozj base job  https://review.openstack.org/58716515:47
smcginnisI'm experiencing the slow loading etherpad issue again. Anyone else seeing that?15:48
*** ykarel|away has joined #openstack-infra15:49
openstackgerritMerged openstack-infra/zuul master: Fix skipped job counted as failed  https://review.openstack.org/62591015:49
*** wolverineav has joined #openstack-infra15:51
*** ykarel|away is now known as ykarel15:52
fricklerdtantsur: thanks. I guess in the meantime the best workaround would seem to be creating a bug in https://bugs.launchpad.net/openstack-gate and crosslinking that to your story and your query15:54
*** bhavikdbavishi has joined #openstack-infra15:54
*** wolverineav has quit IRC15:55
clarkbwe lost yfried but pbr should package all files that are managed by git.15:56
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Pull fetch-output back out of base  https://review.openstack.org/62955615:56
mordredcorvus, AJaeger: ^^15:56
mordredclarkb: putting them in the source repo and having them included in teh wheel as data-files are different15:57
mordreds/repo/tarball/15:57
mordredif he wants data-files so that they get bundled in pkg-resources, then yeah, I think there would need to be a pbr patch to support recursing on directories or something15:58
AJaegerfrickler: btw. did you cleanup the 20 test nodes in ovh? Are they still needed?15:58
*** rcarrillocruz has joined #openstack-infra15:59
mordredcorvus: so, I think the new patch to fetch-output wants to put docs and artifacts into logs if zuul.change is defined and otherwise put them into zuul.executor.work_root/{docs,artifacts} - yeah?16:00
clarkbfrickler: dtantsur: you should be able to paste kibana urls into stories as comments at the veryleast16:01
clarkbbut generally we've used elastic-recheck to drive that coordination of work16:01
corvusmordred: i don't want to flip-flop on this.  do we have the whole system written up somewhere?  if not, can you do that?16:02
fricklerAJaeger: they were gone when I looked this morning, not sure who did the cleanup, but should be fine now anyway16:03
*** pcaruana has quit IRC16:03
*** bhavikdbavishi has quit IRC16:04
*** bhavikdbavishi has joined #openstack-infra16:04
* frickler heads home now, bbl16:04
mordredcorvus: http://lists.openstack.org/pipermail/openstack-infra/2017-October/005606.html16:06
mordredcorvus: I could put that into a spec though16:07
corvusmordred: oh no, i don't need more words, i need less :)16:07
corvusmordred: gimme a minute to digest the ml post16:07
mordredcorvus: scroll to "Artifacts and Docs" under "For OpenStack Today"16:08
*** yboaron_ has quit IRC16:09
*** yamamoto has joined #openstack-infra16:10
*** yboaron_ has joined #openstack-infra16:11
corvusmordred: yes.  :)16:12
*** snierodz_ has quit IRC16:12
mordredcorvus: actually - as I ponder that - I think the logic we're discussing wants to be a second role that moves things on the executor after fetch-output - beause uploading artifacts to the log server is an approach we're taking as openstack, but may not be how other deployers want to handle that16:13
corvusmordred: (to your question at 16:00)16:13
corvusmordred: that's fair16:13
mordredcorvus: so I think we want a second role which does the zuul.change based logic to put things into logs16:13
mordredwhich in our case we'll put into the base job right after fetch-output16:13
corvusmordred: yeah. i think *most* people will want it, but it will not hurt to have a second role and the system is nicer that way i think.16:14
toskyfrickler: if you have some time, be my guest! I'm a bit swamped with the M2 deadline for sahara, and I suppose it will continue a bit after M2 to clean up the fallout of two big changes landing in sahara16:14
mordredcorvus: ++16:14
*** yamamoto has quit IRC16:14
*** kjackal has quit IRC16:14
mordredcorvus: hard question: what should that role be called?16:16
corvusmordred: store-docs-and-artifacts-with-logs?  log-output?  merge-output?16:17
openstackgerritMerged openstack-infra/project-config master: Pull fetch-output back out of base  https://review.openstack.org/62955616:17
*** xarses_ has quit IRC16:17
mordredcorvus: I like the word merge ... merge-output-to-logs ?16:18
*** gyee has joined #openstack-infra16:18
corvuswfm16:18
mordredcool. I'll get that up in just a sec16:18
openstackgerritsebastian marcet proposed openstack-infra/openstackid master: Migration to PHP 7.x  https://review.openstack.org/61193616:21
*** shardy has quit IRC16:23
*** armax has quit IRC16:25
*** diablo_rojo has joined #openstack-infra16:26
*** chkumar|out is now known as chandankumar16:27
*** jamesmcarthur has quit IRC16:33
*** jpena_ has joined #openstack-infra16:36
*** nhicher has quit IRC16:39
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Add validate-dco-license role  https://review.openstack.org/62956516:39
*** pabelanger has quit IRC16:39
*** jpena has quit IRC16:39
*** weshay has quit IRC16:40
*** pabelanger has joined #openstack-infra16:40
*** kjackal has joined #openstack-infra16:44
*** ykarel is now known as ykarel|away16:44
clarkblogan-: as a datapoint I don't see anymoer of the tripleo subnode ssh failures since you restarted the neutron agents (we can keep watching that but so far looking good)16:45
clarkblogan-: thank you for digging into that16:45
openstackgerritMerged openstack-infra/subunit2sql master: Fix inconsistent treatment of test status  https://review.openstack.org/62719616:45
*** jamesmcarthur has joined #openstack-infra16:45
clarkbamorin: http://grafana.openstack.org/d/BhcSH5Iiz/nodepool-ovh?orgId=1 shows we are using more of bhs1 now. Thank you16:46
*** rlandy is now known as rlandy|brb16:46
*** eernst has joined #openstack-infra16:48
*** ykarel|away has quit IRC16:49
*** ykarel|away has joined #openstack-infra16:49
*** lucasagomes has joined #openstack-infra16:50
*** imacdonn has quit IRC16:51
*** imacdonn has joined #openstack-infra16:51
openstackgerritMerged openstack-infra/subunit2sql master: Replace DbMigrationError with DBMigrationError  https://review.openstack.org/50085816:59
*** e0ne has quit IRC17:03
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Add role to move docs and artifacts to log root  https://review.openstack.org/62957117:04
*** pgaxatte has quit IRC17:04
*** rpittau has quit IRC17:08
*** rlandy|brb is now known as rlandy17:09
logan-clarkb: awesome! no problem. catching up on backscroll I see your message yesterday afternoon -- will take a look at those hosts later this afternoon.17:09
*** dtantsur is now known as dtantsur|afk17:11
openstackgerritsebastian marcet proposed openstack-infra/project-config master: Updated Tox Docs job to run in Bionic  https://review.openstack.org/62957917:12
AJaegerfrickler: great, thanks!17:12
*** eumel8 has quit IRC17:12
openstackgerritsebastian marcet proposed openstack-infra/openstackid master: Migration to PHP 7.x  https://review.openstack.org/61193617:15
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Add role to move docs and artifacts to log root  https://review.openstack.org/62957117:16
*** armax has joined #openstack-infra17:17
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Add role to move docs and artifacts to log root  https://review.openstack.org/62957117:17
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Add merge-output-to-logs to base-test  https://review.openstack.org/62958417:22
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Add fetch-output and merge-output-to-logs to base  https://review.openstack.org/62958517:22
*** Dobroslaw has quit IRC17:22
*** yamamoto has joined #openstack-infra17:24
*** irdr has quit IRC17:25
*** yamamoto has quit IRC17:29
*** smarcet has quit IRC17:29
openstackgerritMonty Taylor proposed openstack-infra/project-config master: Upload artifacts in base-test job  https://review.openstack.org/51186017:31
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Add role to move docs and artifacts to log root  https://review.openstack.org/62957117:33
*** chandankumar is now known as chkumar|out17:42
*** jpich has quit IRC17:43
*** gfidente has quit IRC17:44
*** wolverineav has joined #openstack-infra17:44
*** aojea has quit IRC17:48
clarkbcorvus: care to rereview https://review.openstack.org/#/c/629257/ I cleaned up some additional stuff that was original missed nad updated cacti17:48
clarkbonce that is in I can delete the old nameservers unless someone else would like to17:49
*** wolverineav has quit IRC17:49
corvusclarkb: ++17:49
openstackgerritMonty Taylor proposed openstack-infra/zuul-base-jobs master: Add fetch-output to base jobs  https://review.openstack.org/62897517:51
openstackgerritMonty Taylor proposed openstack-infra/zuul-base-jobs master: Ignore errors on ssh key removal  https://review.openstack.org/62897617:51
openstackgerritMonty Taylor proposed openstack-infra/zuul-base-jobs master: Add merge-output-to-logs to base job  https://review.openstack.org/62960417:51
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Add role to move docs and artifacts to log root  https://review.openstack.org/62957117:52
*** trown is now known as trown|lunch17:55
*** derekh has quit IRC17:56
*** yamamoto has joined #openstack-infra17:56
*** ginopc has quit IRC17:59
*** sshnaidm is now known as sshnaidm|afk18:01
*** yamamoto has quit IRC18:01
openstackgerritMonty Taylor proposed openstack-infra/zuul-base-jobs master: Add merge-output-to-logs to base job  https://review.openstack.org/62960418:07
openstackgerritMark Goddard proposed openstack-infra/irc-meetings master: Add kayobe team meeting  https://review.openstack.org/62961218:10
*** jpena_ is now known as jpena|off18:12
openstackgerritMerged openstack-infra/nodepool master: Extract out common config parsing for ConfigPool  https://review.openstack.org/62164218:14
*** bhavikdbavishi1 has joined #openstack-infra18:15
*** armax has quit IRC18:16
*** bhavikdbavishi has quit IRC18:16
*** bhavikdbavishi1 is now known as bhavikdbavishi18:16
*** armax has joined #openstack-infra18:16
*** dpawlik has joined #openstack-infra18:16
*** bhavikdbavishi has quit IRC18:20
*** wolverineav has joined #openstack-infra18:27
*** wolverineav has quit IRC18:30
*** wolverineav has joined #openstack-infra18:31
*** lpetrut has joined #openstack-infra18:33
*** ykarel|away has quit IRC18:34
*** e0ne has joined #openstack-infra18:35
*** jamesmcarthur has quit IRC18:36
*** lpetrut has quit IRC18:37
*** irdr has joined #openstack-infra18:47
*** electrofelix has quit IRC18:55
*** yboaron_ has quit IRC18:57
*** e0ne has quit IRC19:02
*** jamesmcarthur has joined #openstack-infra19:04
*** e0ne has joined #openstack-infra19:05
*** trown|lunch is now known as trown19:08
*** harlowja has joined #openstack-infra19:10
*** yamamoto has joined #openstack-infra19:16
amorinclarkb: no pb19:17
*** armax has quit IRC19:17
*** panda has quit IRC19:19
*** yamamoto has quit IRC19:21
*** panda has joined #openstack-infra19:21
*** rkukura has quit IRC19:25
fungiokay, so i'm just now getting to trying to launch a new storyboard-dev01.opendev.org server and it's not applying any puppet that i can tell. the hostname matches the pattern at https://git.openstack.org/cgit/openstack-infra/system-config/tree/manifests/site.pp#n592 so what else am i missing? does it need to be added to our static manifest first?19:28
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Enforce max-job-timeout for jobs without configured timeout  https://review.openstack.org/62955219:29
*** lathiat has quit IRC19:29
clarkbfungi: the way it works now is that all it does is apply our base ansible server stuff19:31
clarkbfungi: then you add it to the inventory and kick.sh or wait for the next run to puppet it19:32
fungiaha, thanks. so working as expected?19:32
openstackgerritMerged openstack-infra/zuul master: Add cgroup support to ram sensor  https://review.openstack.org/54950619:32
clarkbyes. Also keep in mind you may want to reboot after puppeting (I think this is the biggest gotcha, that was automagic before)19:32
fungiprobably warrants expanding what we've got in https://git.openstack.org/cgit/openstack-infra/system-config/tree/launch/README19:32
clarkb++19:32
fungii'll see what i can work out19:32
pabelangerclarkb: corvus: ianw: So, I've been hacking away on https://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3-3rd-party-ci.html over the last few weeks, I'm basically at the point now in my local environment to start setting up pipelines in zuul and run a job in check.  One thing I'd actually like to experiment with, is seeing if we can load the pipelines for 3pci directly from git.o.o.  I19:34
pabelangerwas thinking we could use project-config-example or sandbox to test, and if works figure out the right location (or we can decided now). The main thing I want to confirm, is if we merge a pipeline stanza into untrusted job, is it won't affect zuul.o.o. I guess we could also update main.yaml to ignore pipelines from a specific project too19:34
clarkbpabelanger: we might also want to double check that the check pipeline is common enough among what 3pci are doing19:36
clarkbpabelanger: many of them due custom triggers so that you can recheck only their jobs19:36
clarkb(and that won't work in a centrally hosted config repo)19:36
pabelangerclarkb: yah, that is the trade off I think, custom triggers.19:36
corvusclarkb: well, we can host a config that we recommend, and since we don't recommend custom triggers, job done.19:37
clarkbcorvus: ya but if no one uses it...19:37
clarkbthen maybe we are better off making it easier to load a config locally19:37
corvuswell, it is easy :)19:37
corvusjust fork the repo and you're done19:38
corvuseither way, i think pabelanger's idea has merit19:38
corvus(you can use the repo directly and get our recommended config, you can fork the repo and modify it to suit if you want to go off-script)19:38
clarkbya19:38
clarkband I want to say that zuul will refuse to load pipelines out of untrusted repos but its been a while since I reviewed the config loading code19:39
pabelangeryah, I cannot remember myself19:39
corvusi believe it's an error, so we'd want a repo with pipelines excluded from the config objects for that repo19:40
clarkbssbarnea|rover: I've reviewed some of your d-g changes. Left some comments. Thinking beyond those changes is tripleo planning to stop using devstack-gate? I don't think the infra team intends to support it long term (and eg update the ansible version in use there)19:41
corvuspabelanger: (if you just propose a change to sandbox with a pipeline, i expect zuul to return an error)19:41
clarkbcorvus: ah19:41
pabelangercorvus: okay, let me try first, and then prepare a patch to main.yaml19:41
clarkbssbarnea|rover: if we can help with sketching out what a conversion to zuulv3 native jobs looks like I think we'd be happy to do that.19:42
ssbarnea|roverclarkb: in fact is not tripleo that is using the devstack-gate, i found it on some other projects (need to check which one)19:43
clarkbssbarnea|rover: ok whichever projects they are I think the ideal would be to work with them to update to zuulv3 native jobs rather than continue to fix or improve devstack-gate19:44
clarkb(in the short term we may need to do both)19:44
*** rkukura has joined #openstack-infra19:44
fungigrenade is still the elephant in the room19:44
ssbarnea|roverclarkb: lets try to do a grep to find out who are the users19:44
clarkbfungi: ya I think frickler intends to target that one specifically19:45
fungitosky had picked up the effort to get grenade jobs onto zuul v3 native configuration, but didn't have time to finish19:45
corvushttps://review.openstack.org/#/c/548936/ looks to be the effort in question19:49
openstackgerritSorin Sbarnea proposed openstack-infra/zuul master: docs: Added missing -i on docker-compose up command  https://review.openstack.org/62965219:52
fungiyep19:55
fungii asked him about it back before the holidays and he said he was hoping to find time to resume work on it, but i guess he hasn't been able to yet19:56
fungiif memory serves, he wanted it for sahara so that he could more easily extend the job for their devstack plugin or something19:56
clarkba quick look at that shows the framework is in place though which is good19:57
clarkbits mostly a matter of adding the run grenade steps to https://review.openstack.org/#/c/548936/24/playbooks/grenade.yaml ?19:57
openstackgerritMerged openstack-infra/system-config master: Nameservers are now managed with ansible  https://review.openstack.org/62925719:57
clarkboh hrm no its breaking grenade apart in the job19:57
clarkbit might be easier to cmd: ./grenade.sh20:02
clarkbbut I don't have enough fo that context switched in to know for sure20:02
fungicmurphy: clarkb: the futureparser list also has incorrect globs for storyboard and storyboard-dev servers, i'm noticing, but since i'm getting ready to replace the servers maybe that's okay?20:04
clarkbfungi: do the globs match only the new or the old servers?20:06
fungiclarkb: only the new servers20:06
clarkbif they only match the new then I think its fine. We should be able to quickly fix any puppet issues you run into on the new servers20:06
*** efried has joined #openstack-infra20:06
fungisame error which was made with the mailman hostname globs20:06
fungi[0-9]* means a digit followed by anything, not zero or more digits20:06
clarkbya20:07
clarkbthe main issue I've run into is apache vhosts not templating properly due to variable scoping errors. TO fix those you switch to the scope lookupvar function in the erb templates20:08
*** e0ne has quit IRC20:08
*** dpawlik has quit IRC20:08
fungiyeah, i'll keep an eye out20:09
fungiout of curiosity, is there a reason we need the private_v4 addresses in our static inventory?20:10
clarkbfungi: I am not aware of one20:13
fungimordred: ^ you may know why we included it?20:13
fungiseems like tracking a useless piece of data, but odds are i'm just not thinking of the reason we needed it20:14
openstackgerritMonty Taylor proposed openstack-infra/system-config master: Prevent gitea pods from being co-scheduled  https://review.openstack.org/62965520:14
clarkbdansmith: do you know the context of https://review.openstack.org/#/c/487664/4 I think mriedem is out this week and want to double check that the jobs that need this debugging run legacy devstack-gate instead of the new devstack stuff20:14
dansmithclarkb: I don't20:14
mordredfungi: I think it's just that we had been tracking it before - I do not believe we've ever used that for anything20:15
mordredfungi: I would be perfectly fine excluding that data20:16
*** ldnunes has quit IRC20:16
fungithanks. i guess we should consider that if nobody comes up with a good reason not to20:16
openstackgerritJeremy Stanley proposed openstack-infra/system-config master: Add storyboard-dev01.opendev.org to inventory  https://review.openstack.org/62965620:19
clarkbgoing to grab lunch now. I'll double check puppet stopped running on the old nameservers, delete them and clean up dns records for them this afternoon. Then I'll try to catch up on any fedora29 stuff that needs to ahppen and then hopefully have time to do administrivia20:20
openstackgerritJeremy Stanley proposed openstack-infra/zone-opendev.org master: Add storyboard-dev01.opendev.org  https://review.openstack.org/62965720:24
fungii think the error i'm getting from rackdns when trying to set the reverse ptr has to do with the address records not resolving20:25
fungioh, nope, it has to do with me being stupid20:26
fungii was pasting the commands to add the address records via rackdns :/20:26
toskyfungi, clarkb : frickler said earlier on this channel that he may have timeto work on it20:27
clarkbfungi new domain :)20:27
fungitosky: yep, fingers crossed!20:27
toskyI'm a bit swamped with some structural sahara changes20:27
fungihappens to all of us20:27
fungiclarkb: mordred: storyboard-dev01.opendev.org server is booted and has reverse dns, once 629656 and 629657 get reviewed i should be able to proceed with testing whether it gets puppeted20:28
*** smarcet has joined #openstack-infra20:35
openstackgerritMonty Taylor proposed openstack-infra/puppet-gerrit master: Add support for specifying which refs to replicate  https://review.openstack.org/62965820:35
*** slaweq has quit IRC20:37
*** efried has quit IRC20:40
openstackgerritMonty Taylor proposed openstack-infra/system-config master: Only replicate branches and tags to gitea  https://review.openstack.org/62966020:41
*** kjackal has quit IRC20:41
clarkbfungi: +2 on both. For the first one you might want ot just approve with one vote? its an update for a dev server. The other affects productio ndns so is a little more important but also not used for much yet :)20:41
openstackgerritMerged openstack-infra/zuul master: docs: Added missing -i on docker-compose up command  https://review.openstack.org/62965220:43
*** markvoelker has joined #openstack-infra20:45
fungithanks!20:48
*** slaweq has joined #openstack-infra20:53
*** yboaron_ has joined #openstack-infra21:02
*** yamamoto has joined #openstack-infra21:04
*** yamamoto has quit IRC21:09
clarkbI've stopped nsd on ns1.openstack.org and ns2.openstack.org21:12
*** ociuhandu has joined #openstack-infra21:13
clarkblet me know if you notice any dns issues. I'll remove A and AAAA records for them next21:14
fungiawesome21:15
clarkbfollowing up on fedora-29 it seems that the step we are at is waiting for nodepool to build images and boot them21:19
clarkbthe change to make this happen, https://review.openstack.org/#/c/618672/3/nodepool/nodepool.yaml is merged and applied but nodepool isn't doing it yet21:20
openstackgerritMonty Taylor proposed openstack-infra/puppet-gerrit master: Add support for specifying which refs to replicate  https://review.openstack.org/62965821:20
clarkbI'm initially stumped as to why that would be21:22
clarkbdo we need to restart builders to pick up new config? that may actually be it21:23
clarkbI'm going to restart the builders on nb01 and nb02 to check21:25
mordredclarkb: I thought nodepool would notice new config21:27
mordredShrews: ^^ ?21:27
Shrewsclarkb: i don't *think* a restart is necessary21:27
*** jamesmcarthur has quit IRC21:27
Shrewsbut i cannot think of any reason why they aren't picked up yet. been a while since i've thought about any builder stuff21:27
*** jamesmcarthur has joined #openstack-infra21:29
openstackgerritColleen Murphy proposed openstack-infra/system-config master: Turn on future parser for afs servers  https://review.openstack.org/62966221:29
openstackgerritColleen Murphy proposed openstack-infra/system-config master: Turn on future parser for kerberos servers  https://review.openstack.org/62966321:29
clarkbmordred: Shrews well both builders were out to lunch. nb01 since the 4th and nb02 since 2018 somtime21:30
clarkbboth were hungup on dib builds that were trying t oclone git repos21:30
clarkbI've cleaned that up and restarted the builders and now image builds are happening again21:30
clarkbmy guess is fedora-29 builds are queued up behind the backlog?21:31
clarkbShrews: we might want to add a timeout on the dib fork?21:31
clarkblike say 8 hours21:31
* clarkb context switches to cleaning up dns21:31
clarkbbut ya I think we may have queued things up but stuff got stuck behind the backlog. We should know in a few hours21:34
fungiso it probably did pick up the config update automatically, but was basically stuck so didn't act on it?21:34
clarkbthat is my theory21:36
mordredsounds right21:36
clarkbit should be unstuck now and once it clears the backlog we should see fedora-2921:36
clarkbinfra-root I see an ns3.openstack.org A record (not sure about AAAA yet, can I remove that too?)21:36
clarkbI'm removing A and AAAA records for adns1.openstack.org ns1.openstack.org and ns2.openstack.org now21:37
fungimordred: is the ref replication filter in service of not replicating change refs to gitea? if so, curious why it's a problem21:37
openstackgerritMerged openstack-infra/system-config master: Add storyboard-dev01.opendev.org to inventory  https://review.openstack.org/62965621:39
fungiclarkb: those ip addresses for ns3 respond to ping, and the ssh banner claims it's an ubuntu server21:39
*** kgiusti has left #openstack-infra21:39
fungimay be timing out trying to perform reverse dns lookups on my client though, ssh is hanging for me so far21:39
*** ociuhandu has quit IRC21:39
mordredfungi: yes, in fact it is, and is consumed here: https://review.openstack.org/62966021:40
mordredthere's two things - which I think are likely both solvable in time ...21:40
openstackgerritMerged openstack-infra/nodepool master: Extract common config parsing for ProviderConfig  https://review.openstack.org/62509421:41
fungii see, the commit message is fairly enlightening21:41
mordred\o/ commit messages21:41
clarkbcorvus: ^ do you know what ns3.openstack.org is?21:42
openstackgerritColleen Murphy proposed openstack-infra/system-config master: Update review-dev.o.o to puppet 4  https://review.openstack.org/62966721:42
clarkbat this point I think the last step for adns1.openstack.org ns1.openstack.org and ns2.openstack.org is to delete them21:42
fungithe ip addresses for ns3 are in ord, not dfw21:45
fungii wonder if that was an attempt at going cross-region with those nameservers but then later deleted and the dns entries for it never cleaned up?21:47
toskyI may have asked it already: is it still possible to use the finger client to access the logs of running zuul jobs? I remember that I could do it when the old zuul interface was used21:48
clarkbfungi: ya I don't see it in our ord server listing21:48
fungiyou can, yes21:48
clarkbadns1 is there though21:48
mordredtosky: finger d59ac94eb0374f3ba8b0cc8be2341c61@zuul.openstack.org21:50
fungitosky: it's `finger xxxxxxx@zuul.openstack.org` where the xxxxxxx is the build id21:50
mordredtosky: is the pattern21:50
fungiand you can grab the build id from the web console stream url21:51
toskyoh, right21:51
toskythanks21:51
toskymuch faster than the web interface21:52
fungiSo for http://zuul.openstack.org/stream/81ab3f251c32426380d5dfa83f17cc33?logfile=console.log you can `finger 81ab3f251c32426380d5dfa83f17cc33@zuul.openstack.org`21:52
clarkbdo we think we need any snapshots of these servers? I'm guessing no since the data they hosted is/was all in zone files and hiera?21:53
fungiclarkb: there should be nothing of importance on them now21:53
fungiunless someone put something they cared about in their homedir21:53
mordredfungi: it's where I keep my warez21:54
fungigreat idea21:54
clarkbI will be deleting f8c9e5d6-818f-4168-a53e-414c7e3ccb34 adns1.openstack.org in ORD, af56cafc-6a3d-4ffb-b443-932ece962673 ns1.openstack.org in DFW, and c4f203c0-5315-45ac-ab11-dbce5bd33d67 ns2.openstack.org in ca-ymq-121:55
clarkbinfra-root ^ last call21:57
*** fuentess has quit IRC21:58
clarkb#status log deleted f8c9e5d6-818f-4168-a53e-414c7e3ccb34 adns1.openstack.org in ORD, af56cafc-6a3d-4ffb-b443-932ece962673 ns1.openstack.org in DFW, and c4f203c0-5315-45ac-ab11-dbce5bd33d67 ns2.openstack.org in ca-ymq-1. DNS now hosted by opendev nameservers22:07
openstackstatusclarkb: finished logging22:07
clarkbcorvus: ^ the last remaining thing is whether or not A and AAAA records for ns3.openstack.org can be removed22:07
*** ccamacho has quit IRC22:08
clarkb| fedora-29-0000000001            | fedora-29            | nb02    | qcow2,raw,vhd | building | 00:00:00:24 |22:10
clarkbthat must've been it. stuck in the queue behind stuck dib builds22:10
*** trown is now known as trown|outtypewww22:10
corvusclarkb: i think so22:10
*** smarcet has left #openstack-infra22:12
clarkbI'll get those cleaned up too then22:13
clarkbfungi: corvus reverse dns lookup on the ipv4 addr says its artfixdaily.com22:20
clarkbso ya hsould be safe to remove :)22:21
clarkb#status log removed ns3.openstack.org A 166.78.116.117 and ns3.openstack.org AAAA 2001:4801:7825:103:be76:4eff:fe10:4f7a as these IPs don't seem to belong to us anymore.22:23
*** lujinluo has joined #openstack-infra22:23
openstackstatusclarkb: finished logging22:23
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Fixed cloud assets bugs * missing lib * missing real path  https://review.openstack.org/62967222:23
fungiclarkb: yeah, i saw that ptr record too, but don't trust that it wasn't something stale prior to our use of the address either22:25
*** ccamacho has joined #openstack-infra22:27
clarkbfungi: well it resolve sforward and their website is up at that addr22:28
fungii've seen rackspace fail to scrub ptr records in the past, but yeah22:28
*** ccamacho has quit IRC22:28
fungiseems to not be ours at any rate22:28
openstackgerritMerged openstack-infra/openstackid-resources master: Fixed cloud assets bugs * missing lib * missing real path  https://review.openstack.org/62967222:35
*** boden has quit IRC22:38
fungiclarkb: do you happen to know whether we still need the group variables in the global site manifest, or whether those are vestigial now?22:46
clarkbfungi: we use those for hiera lookups22:47
fungiokay, so mapping in inventory/groups.yaml alone is still not sufficient22:47
clarkbcorrect, that mapping is for the ansible side of thing s(tells ansible which hiera files to copy for example)22:47
clarkbthen on the puppet side the $group var tells puppet how to lookup those values22:47
openstackgerritJeremy Stanley proposed openstack-infra/system-config master: Create a storyboard-dev hiera lookup group  https://review.openstack.org/62967422:49
fungiclarkb: ^ so like that?22:49
fungii've copied host_vars/storyboard-dev.openstack.org.yaml to group_vars/storyboard-dev.yaml on bridge.o.o as well22:51
clarkbyes22:51
fungiand i guess the old host_vars file can go away once that merges22:51
*** smarcet has joined #openstack-infra22:51
*** rcernin has joined #openstack-infra22:52
*** smarcet has quit IRC22:52
*** yamamoto has joined #openstack-infra22:52
clarkbyup. At one point I cleaned up a fair bit of redundant files like that22:52
clarkbI imagine there is more we could do though22:52
*** diablo_rojo has quit IRC22:53
*** yamamoto has quit IRC22:56
*** markvoelker has quit IRC22:57
*** markvoelker has joined #openstack-infra22:57
*** markvoelker has quit IRC23:02
*** jcoufal has quit IRC23:03
*** whoami-rajat has quit IRC23:04
clarkb| fedora-29-0000000001            | fedora-29            | nb02    | qcow2,raw,vhd | ready    | 00:00:00:06 |23:04
clarkband now we wait for the image to boot23:05
ianwclarkb: yeah, that was weird, thanks just checking on that23:05
*** jamesmcarthur has quit IRC23:07
fungithis seems like not the only time in recent history all our nodepool builders were stuck23:08
openstackgerritIan Wienand proposed openstack-infra/project-config master: Add CentOS NetworkManager testing node  https://review.openstack.org/62968023:08
openstackgerritIan Wienand proposed openstack-infra/project-config master: Revert "Add CentOS NetworkManager testing node"  https://review.openstack.org/62968123:08
fungididn't the same thing happen a month or two back?23:08
ianwfungi: hrm, don't remember that.  i used to have monitoring scripts, but when we split into multiple builders it became harder23:09
clarkbya they were broken when we got back from summit in berlin23:09
fungiand here i was about to chalk that memory up to a feverish, eggnog-induced dream23:10
*** jamesmcarthur has joined #openstack-infra23:10
*** jamesmcarthur has quit IRC23:11
clarkbI'm trying to remember those circumstances, they had both broken on the same image build iirc23:11
clarkband I want to say theory was something particular to that build at that time was the cause?23:11
*** diablo_rojo has joined #openstack-infra23:12
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Run k8s-on-openstack to manage k8s control plane  https://review.openstack.org/62696523:16
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add resources for deploying rook and xtradb to kuberenets  https://review.openstack.org/62605423:16
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add gitea k8s resource definitions and playbook  https://review.openstack.org/62675923:16
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Prevent gitea pods from being co-scheduled  https://review.openstack.org/62965523:16
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Use project-config when syncing gitea repos  https://review.openstack.org/62968323:16
corvusmordred: ^ er, most of that was rebasing i think.23:17
corvusi'm running the contents of the last change now to create gitea repos23:17
*** jamesmcarthur has joined #openstack-infra23:18
corvushttp://38.108.68.64/ is today's gitea url23:19
clarkbnow I want tea23:20
corvusme too23:20
corvusi think i'll make some23:20
corvus(it's up to openstack/deb-python-heatclient)23:20
fungiwithout heat, the tea takes too long to steep23:22
*** rlandy has quit IRC23:25
openstackgerritMerged openstack-infra/system-config master: Install openshift module on bridge  https://review.openstack.org/62674823:27
*** jamesmcarthur has quit IRC23:30
*** yamamoto has joined #openstack-infra23:30
ianw[root@fedora-29-vexxhost-vexxhost-sjc1-0001606322 ~]# nmcli c23:31
ianwNAME         UUID                                  TYPE      DEVICE23:31
ianwSystem ens3  21d47e65-8523-1a06-af22-6f121086f085  ethernet  ens323:31
ianwthat simple command took quite a lot of building blocks to get to!23:31
*** jamesmcarthur has joined #openstack-infra23:32
clarkbianw: re centos7 network manager, we could also potentially just make the switch with 8 and leave 7 as is?23:32
*** lathiat has joined #openstack-infra23:34
*** jamesmcarthur has quit IRC23:35
ianwmy thinking there was that it's supported on 7, so it's not a weird thing, and so reducing or differences to 8 helps, and also making it so we just support one path in infra helps us too23:35
clarkbya having a common toolset is nice23:36
*** jamesmcarthur has joined #openstack-infra23:36
ianwif it was something as big as systemd/sysv type change, i'd think running in parallel ... but in this case i think basically nothing should notice23:37
*** jamesmcarthur has quit IRC23:37
ianw(insert fingers crossed emoji)23:37
*** rkukura has quit IRC23:37
ianw🤞23:37
clarkbmy font unfortunately did not render that23:38
anteayaI also lack the fingers crossed emoji23:39
openstackgerritMerged openstack-infra/system-config master: Create a storyboard-dev hiera lookup group  https://review.openstack.org/62967423:42
*** rkukura has joined #openstack-infra23:46
clarkbianw: are we ready to bump fedora-latest to 29 now that image is built and bootable?23:50
ianwclarkb: yeah, that was the intention -> https://review.openstack.org/#/c/618673/23:51
ianwi guess my prior mail didn't really cover this ... i wonder what's using it23:52
ianwmostly us, ansible-* projects, couple of others23:54
openstackgerritJohn Studarus proposed openstack-infra/nodepool master: Work in Progress  https://review.openstack.org/62968823:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!