| *** gyee has quit IRC | 00:02 | |
| *** sthussey has quit IRC | 00:02 | |
| *** dingyichen has quit IRC | 00:07 | |
| *** dingyichen has joined #openstack-infra | 00:07 | |
| *** longkb has joined #openstack-infra | 00:11 | |
| *** slaweq has joined #openstack-infra | 00:11 | |
| *** dingyichen has quit IRC | 00:15 | |
| *** slaweq has quit IRC | 00:16 | |
| *** dingyichen has joined #openstack-infra | 00:16 | |
| ianw | ok, so as part of run_all.sh we're running playbooks/update_puppet_version.yaml against the 'puppet4' hosts anyway | 00:16 |
|---|---|---|
| ianw | this runs install_puppet.sh with PUPPET_VERSION=4 . ergo, we're running it as if it was idempotent against the puppet4 hosts anyway | 00:17 |
| ianw | since the puppet4 hosts haven't exploded, i guess that's true enough. still ... we would probably be better off having ansible manage it from the base playbook | 00:21 |
| clarkb | ianw: I don't know that there are any puppet 4 hosts yet | 00:24 |
| clarkb | just puppet 3 hosts using puppet4's parser via the futureparser config option | 00:24 |
| *** olivierb has joined #openstack-infra | 00:26 | |
| ianw | clarkb: oh, ok, well maybe more reason to do this | 00:26 |
| ianw | infra-root: re above, if someone has a sec to look in on the role integration testing, i'd like to base any roles on that https://review.openstack.org/#/c/596018/ | 00:27 |
| ianw | this seems like it's generic enough to run under zuul, and might be handy for other puppet testing | 00:28 |
| *** olivierb_ has quit IRC | 00:29 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Only run role integration tests on role changes https://review.openstack.org/596966 | 00:29 |
| *** olivierb_ has joined #openstack-infra | 00:48 | |
| *** olivierb has quit IRC | 00:51 | |
| *** dingyichen has quit IRC | 00:57 | |
| *** dingyichen has joined #openstack-infra | 00:57 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 00:58 |
| *** auristor has quit IRC | 00:58 | |
| *** rh-jelabarre has quit IRC | 01:06 | |
| *** auristor has joined #openstack-infra | 01:06 | |
| *** dingyichen has quit IRC | 01:07 | |
| *** dingyichen has joined #openstack-infra | 01:07 | |
| *** dingyichen has quit IRC | 01:11 | |
| *** dingyichen has joined #openstack-infra | 01:12 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 01:22 |
| *** felipemonteiro has quit IRC | 01:24 | |
| *** anteaya has quit IRC | 01:24 | |
| *** dingyichen has quit IRC | 01:25 | |
| *** dingyichen has joined #openstack-infra | 01:25 | |
| *** dingyichen has quit IRC | 01:28 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Add role integration jobs https://review.openstack.org/596018 | 01:29 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Only run role integration tests on role changes https://review.openstack.org/596966 | 01:29 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 01:29 |
| *** dingyichen has joined #openstack-infra | 01:29 | |
| *** rosmaita has quit IRC | 01:30 | |
| *** dingyichen has quit IRC | 01:33 | |
| *** dingyichen has joined #openstack-infra | 01:34 | |
| *** graphene has quit IRC | 01:35 | |
| *** graphene has joined #openstack-infra | 01:37 | |
| *** hongbin has joined #openstack-infra | 01:39 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 01:41 |
| *** dingyichen has quit IRC | 01:44 | |
| *** dingyichen has joined #openstack-infra | 01:44 | |
| *** dingyichen has quit IRC | 01:45 | |
| *** dingyichen has joined #openstack-infra | 01:45 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 01:50 |
| *** ykarel has joined #openstack-infra | 01:56 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 02:03 |
| *** slaweq has joined #openstack-infra | 02:11 | |
| *** slaweq has quit IRC | 02:16 | |
| *** apetrich has quit IRC | 02:19 | |
| *** kiennt26 has joined #openstack-infra | 02:19 | |
| *** zxiiro is now known as zxiiro-away | 02:24 | |
| *** ykarel has quit IRC | 02:28 | |
| *** ykarel has joined #openstack-infra | 02:28 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 02:36 |
| ianw | The job "system-config-zuul-role-integration-xenial" exceeds tenant max-nodes-per-job 5 | 02:37 |
| ianw | well i guess that answers my question of "i wonder how many nodes this job should work with" :) | 02:38 |
| *** yamamoto has joined #openstack-infra | 02:42 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 02:43 |
| corvus | ianw: ha! maybe we should ask the sysadmins for a bump? :) | 02:43 |
| ianw | corvus: i think i can lay it out better anyway dividing it up by distro and having a 'base' and 'puppet4' node in each, rather than trying to jam it all into one job | 02:44 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 02:45 |
| corvus | ianw: ok, well for the run-base job, i chose to do it as a single job with multi-node, because we'd be testing all the nodes every change, so bridge+4 nodes == 5 total, versus multi-job it would be a total of 8 nodes used. seemed worth it. | 02:48 |
| *** dpawlik has joined #openstack-infra | 02:48 | |
| ianw | yeah, it just fits until centos 8 :) | 02:51 |
| *** dpawlik has quit IRC | 02:52 | |
| *** psachin has joined #openstack-infra | 02:54 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 02:55 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 03:05 |
| *** rlandy has quit IRC | 03:37 | |
| *** hongbin has quit IRC | 03:43 | |
| *** yamamoto has quit IRC | 03:46 | |
| *** ykarel has quit IRC | 03:50 | |
| *** yamamoto has joined #openstack-infra | 03:51 | |
| *** graphene has quit IRC | 04:02 | |
| *** graphene has joined #openstack-infra | 04:03 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 04:08 |
| *** kiennt26 has quit IRC | 04:08 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 04:10 |
| *** rkukura has quit IRC | 04:11 | |
| *** slaweq has joined #openstack-infra | 04:11 | |
| *** rkukura has joined #openstack-infra | 04:12 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 04:12 |
| *** ramishra has joined #openstack-infra | 04:14 | |
| *** slaweq has quit IRC | 04:15 | |
| *** Bhujay has joined #openstack-infra | 04:18 | |
| *** Bhujay has quit IRC | 04:19 | |
| *** ykarel has joined #openstack-infra | 04:23 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 04:25 |
| *** tdasilva has quit IRC | 04:27 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 04:39 |
| *** ykarel has quit IRC | 04:42 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: write-inventory: add ansible_python_interpreter variable https://review.openstack.org/596961 | 04:46 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 04:50 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove obsolete legacy-dg-hooks-dsvm https://review.openstack.org/596984 | 04:51 |
| *** rascasoft has quit IRC | 04:52 | |
| AJaeger | config-core, https://review.openstack.org/#/c/596543/ and https://review.openstack.org/#/c/596538/ are small cleanups (job removals) ready to merge - followed by the one above ^. Please review | 04:52 |
| *** haleyb has quit IRC | 04:54 | |
| *** _ari_ has quit IRC | 04:55 | |
| *** weshay|rover has quit IRC | 04:55 | |
| *** pabelanger has quit IRC | 04:55 | |
| *** udesale has joined #openstack-infra | 04:56 | |
| *** udesale has quit IRC | 04:57 | |
| *** udesale has joined #openstack-infra | 04:58 | |
| *** armax has quit IRC | 04:59 | |
| *** udesale has quit IRC | 04:59 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 05:00 |
| *** ramishra has quit IRC | 05:02 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 05:08 |
| ianw | you know what i hate ... spending 45 mintues debugging what you think is an int->string conversion issue, which just turns out to be a typo in the node type you're selecting :/ | 05:08 |
| AJaeger | oh fun ;( | 05:09 |
| *** slaweq has joined #openstack-infra | 05:11 | |
| *** slaweq has quit IRC | 05:16 | |
| *** ramishra has joined #openstack-infra | 05:18 | |
| *** udesale has joined #openstack-infra | 05:22 | |
| *** ykarel has joined #openstack-infra | 05:24 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove astara jobs https://review.openstack.org/596538 | 05:24 |
| *** olivierb has joined #openstack-infra | 05:25 | |
| *** janki has joined #openstack-infra | 05:26 | |
| *** olivierb_ has quit IRC | 05:28 | |
| *** slaweq has joined #openstack-infra | 05:28 | |
| *** bnemec has quit IRC | 05:30 | |
| *** udesale has quit IRC | 05:31 | |
| *** bnemec has joined #openstack-infra | 05:31 | |
| *** slaweq has quit IRC | 05:33 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 05:40 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 05:45 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [wip] install puppet https://review.openstack.org/596968 | 05:48 |
| *** icey has quit IRC | 06:05 | |
| *** icey has joined #openstack-infra | 06:05 | |
| *** udesale has joined #openstack-infra | 06:06 | |
| *** icey has quit IRC | 06:08 | |
| *** shabeeb has joined #openstack-infra | 06:09 | |
| *** slaweq has joined #openstack-infra | 06:11 | |
| *** apetrich has joined #openstack-infra | 06:15 | |
| *** slaweq has quit IRC | 06:16 | |
| *** icey has joined #openstack-infra | 06:21 | |
| *** icey has quit IRC | 06:21 | |
| *** icey has joined #openstack-infra | 06:33 | |
| *** jchhatbar has joined #openstack-infra | 06:35 | |
| *** hwoarang has joined #openstack-infra | 06:35 | |
| *** olivierb has quit IRC | 06:38 | |
| *** janki has quit IRC | 06:38 | |
| *** pcaruana has joined #openstack-infra | 06:39 | |
| *** aojea has joined #openstack-infra | 06:40 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: write-inventory: Add groups to created inventory https://review.openstack.org/596994 | 06:41 |
| AJaeger | frickler: could you review https://review.openstack.org/#/c/596543/ and https://review.openstack.org/596984, please? | 06:42 |
| *** udesale has quit IRC | 06:48 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Create ansible roles to install puppet https://review.openstack.org/596968 | 06:49 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Add install-puppet to base playbook https://review.openstack.org/596997 | 06:49 |
| *** yamamoto has quit IRC | 06:54 | |
| *** udesale has joined #openstack-infra | 06:55 | |
| *** yamamoto has joined #openstack-infra | 06:58 | |
| *** hashar has joined #openstack-infra | 06:58 | |
| *** alexchadin has joined #openstack-infra | 07:00 | |
| *** rcernin has quit IRC | 07:02 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: write-inventory: Add groups to created inventory https://review.openstack.org/596994 | 07:03 |
| *** slaweq has joined #openstack-infra | 07:03 | |
| *** ginopc has joined #openstack-infra | 07:03 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy-dg-hooks-dsvm https://review.openstack.org/596543 | 07:07 |
| *** dims has quit IRC | 07:08 | |
| *** dims has joined #openstack-infra | 07:10 | |
| *** vivsoni has quit IRC | 07:11 | |
| *** vivsoni has joined #openstack-infra | 07:16 | |
| *** rascasoft has joined #openstack-infra | 07:19 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Add install-puppet to base playbook https://review.openstack.org/596997 | 07:28 |
| *** dpawlik has joined #openstack-infra | 07:29 | |
| *** graphene has quit IRC | 07:33 | |
| *** graphene has joined #openstack-infra | 07:35 | |
| *** andymccr has joined #openstack-infra | 07:36 | |
| *** ykarel is now known as ykarel|lunch | 07:43 | |
| *** jpena|off is now known as jpena | 07:49 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Add install-puppet to base playbook https://review.openstack.org/596997 | 07:54 |
| *** tosky has joined #openstack-infra | 07:58 | |
| *** jpich has joined #openstack-infra | 07:59 | |
| *** shardy has joined #openstack-infra | 08:04 | |
| *** yamamoto has quit IRC | 08:19 | |
| *** yamamoto has joined #openstack-infra | 08:20 | |
| *** vivsoni has quit IRC | 08:21 | |
| *** vivsoni has joined #openstack-infra | 08:22 | |
| openstackgerrit | Luke Hinds proposed openstack-infra/project-config master: Retiring project Anchor https://review.openstack.org/597018 | 08:29 |
| *** shabeeb has quit IRC | 08:30 | |
| *** shabeeb has joined #openstack-infra | 08:30 | |
| *** dingyichen has quit IRC | 08:35 | |
| *** jchhatbar is now known as janki | 08:37 | |
| *** ykarel|lunch is now known as ykarel | 08:40 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: rewrite interface in react https://review.openstack.org/591604 | 08:42 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add build page https://review.openstack.org/597024 | 08:42 |
| *** e0ne has joined #openstack-infra | 08:43 | |
| *** apetrich has quit IRC | 08:43 | |
| *** electrofelix has joined #openstack-infra | 08:46 | |
| *** pbourke has quit IRC | 08:47 | |
| *** pbourke has joined #openstack-infra | 08:49 | |
| *** dirk has joined #openstack-infra | 08:50 | |
| dirk | whats the process to create a branch in git these days ? | 08:52 |
| dirk | I used to be able to do that directly in gerrit, but it seems the permissions changed | 08:52 |
| dirk | the project in question is openstack/rpm-packaging, I would like to have master branched into stable/rocky | 08:52 |
| *** yamamoto_ has joined #openstack-infra | 08:58 | |
| AJaeger | dirk: ask the release team to do this for you - via the release repository. Same repo you use for tagging. | 08:59 |
| *** shabeeb has quit IRC | 08:59 | |
| *** rcernin has joined #openstack-infra | 08:59 | |
| *** yamamoto has quit IRC | 08:59 | |
| AJaeger | dirk: "releases" is correct name | 08:59 |
| *** shabeeb has joined #openstack-infra | 08:59 | |
| dirk | AJaeger: we're not doing tagging/releases | 09:00 |
| *** priteau has joined #openstack-infra | 09:00 | |
| dirk | I guess I can copy the devstack review | 09:02 |
| *** sambetts|afk has quit IRC | 09:03 | |
| dirk | done so via https://review.openstack.org/597030 | 09:04 |
| *** holser_ has joined #openstack-infra | 09:07 | |
| *** takamatsu has joined #openstack-infra | 09:15 | |
| priteau | Hello. When I try to sign into Gerrit, I see a page with the error message "Provider is not supported, or was incorrectly entered." | 09:16 |
| AJaeger | dirk: you can use the repo just for creation of branches. | 09:17 |
| AJaeger | dirk: please talk with release team | 09:17 |
| AJaeger | dirk: yeah, that might work. | 09:17 |
| *** alexchadin has quit IRC | 09:31 | |
| *** dtantsur|afk is now known as dtantsur | 09:44 | |
| *** rcernin has quit IRC | 09:47 | |
| *** Emine has joined #openstack-infra | 09:47 | |
| openstackgerrit | Dobroslaw Zybort proposed openstack-infra/project-config master: Remove job settings for monasca-agent https://review.openstack.org/597035 | 09:49 |
| mnaser | dirk: if you’re under governance, you need to go though the release process | 09:53 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: write-inventory: Add groups to created inventory https://review.openstack.org/596994 | 09:54 |
| frickler | priteau: is this the first time you try to login or was it working earlier for you and is now broken? | 09:56 |
| priteau | It's working now, not sure what happened. | 09:56 |
| frickler | priteau: o.k., good to hear that. might have been a hickup from ubuntu one, then | 09:57 |
| *** quite has quit IRC | 09:57 | |
| Dobroslaw | AJaeger: Hi, do you think this change is fine? https://review.openstack.org/597035 | 09:58 |
| Dobroslaw | there is problem with monasca-agent that releases older than stable/rocky will always fail because of dependency not working with Python3 | 09:58 |
| Dobroslaw | so we need to remove py35 from global first and then we can merge zuul changes for older branches (with removed py35) | 09:58 |
| Dobroslaw | and also I don't know why stable/pike is passing even that it have same errors like in stable/ocata and stable/queens https://review.openstack.org/#/q/project:openstack/monasca-agent+topic:python3-first | 09:58 |
| *** longkb has quit IRC | 10:00 | |
| *** dpawlik has quit IRC | 10:01 | |
| ttx | I have a git-review hang on "git remote update gerrit" -- anyone else seeing that? | 10:01 |
| ttx | ah, it unblocked after about 1 minute hanging | 10:02 |
| AJaeger | Dobroslaw: yes, that is one way of doing it - works for me. | 10:02 |
| Dobroslaw | great | 10:02 |
| frickler | ttx: I do see that sometimes when I have broken IPv6 connectivity | 10:02 |
| ttx | now blocked at "git push gerrit..." | 10:03 |
| ttx | frickler: interesting | 10:03 |
| *** vivsoni has quit IRC | 10:03 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/project-config master: Add openstack-map repository https://review.openstack.org/597037 | 10:04 |
| *** alexchadin has joined #openstack-infra | 10:04 | |
| *** vivsoni has joined #openstack-infra | 10:04 | |
| *** dpawlik has joined #openstack-infra | 10:04 | |
| *** Emine has quit IRC | 10:05 | |
| *** Emine has joined #openstack-infra | 10:10 | |
| *** Emine has quit IRC | 10:14 | |
| *** Emine has joined #openstack-infra | 10:14 | |
| *** zul has joined #openstack-infra | 10:19 | |
| *** nicolasbock has joined #openstack-infra | 10:20 | |
| *** vivsoni has quit IRC | 10:20 | |
| *** quite has joined #openstack-infra | 10:24 | |
| *** ccamacho|brb has quit IRC | 10:29 | |
| *** vivsoni has joined #openstack-infra | 10:30 | |
| *** stephenfin has quit IRC | 10:35 | |
| *** stephenfin has joined #openstack-infra | 10:36 | |
| *** ccamacho has joined #openstack-infra | 10:50 | |
| *** alexchadin has quit IRC | 10:56 | |
| *** alexchadin has joined #openstack-infra | 10:56 | |
| *** alexchadin has quit IRC | 10:56 | |
| *** alexchadin has joined #openstack-infra | 10:57 | |
| *** ginopc has quit IRC | 10:57 | |
| *** alexchadin has quit IRC | 10:57 | |
| *** alexchadin has joined #openstack-infra | 10:58 | |
| *** alexchadin has quit IRC | 10:58 | |
| *** alexchadin has joined #openstack-infra | 10:58 | |
| *** ginopc has joined #openstack-infra | 10:59 | |
| *** alexchadin has quit IRC | 10:59 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: rewrite interface in react https://review.openstack.org/591604 | 11:18 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add build page https://review.openstack.org/597024 | 11:18 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add job page https://review.openstack.org/597048 | 11:18 |
| AJaeger | dhellmann: did you see my comment on https://review.openstack.org/#/c/593660/5/zuul.d/projects.yaml ? | 11:19 |
| *** jpena is now known as jpena|lunch | 11:28 | |
| *** slagle has joined #openstack-infra | 11:30 | |
| *** vivsoni has quit IRC | 11:33 | |
| *** alexchadin has joined #openstack-infra | 11:35 | |
| *** tpsilva has joined #openstack-infra | 11:36 | |
| *** yamamoto_ has quit IRC | 11:39 | |
| *** yamamoto has joined #openstack-infra | 11:39 | |
| *** alexchadin has quit IRC | 11:41 | |
| *** rh-jelabarre has joined #openstack-infra | 11:41 | |
| *** vivsoni has joined #openstack-infra | 11:50 | |
| *** alexchadin has joined #openstack-infra | 11:51 | |
| *** trown|outtypewww is now known as trown | 11:58 | |
| *** rosmaita has joined #openstack-infra | 11:59 | |
| *** boden has joined #openstack-infra | 12:00 | |
| *** yamamoto has quit IRC | 12:09 | |
| *** Emine has quit IRC | 12:17 | |
| *** shabeeb has quit IRC | 12:19 | |
| *** Emine has joined #openstack-infra | 12:21 | |
| *** ansmith has joined #openstack-infra | 12:27 | |
| *** jpena|lunch is now known as jpena | 12:28 | |
| *** weshay has joined #openstack-infra | 12:29 | |
| *** rlandy has joined #openstack-infra | 12:34 | |
| *** mriedem has joined #openstack-infra | 12:35 | |
| *** gema has joined #openstack-infra | 12:35 | |
| *** yamamoto has joined #openstack-infra | 12:36 | |
| openstackgerrit | Markus Hosch proposed openstack-infra/nodepool master: Add list of metrics provided to statsd https://review.openstack.org/590233 | 12:40 |
| openstackgerrit | Markus Hosch proposed openstack-infra/nodepool master: Add list of metrics provided to statsd https://review.openstack.org/590233 | 12:42 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Update query for nova functional test bug 1788922 https://review.openstack.org/597113 | 12:45 |
| openstack | bug 1788922 in OpenStack Compute (nova) "SRIOVServersTest.test_create_server_with_VF intermittently fails due to "FileNotFoundError: [Errno 2] No such file or directory: '/home/zuul/src/git.openstack.org/openstack/nova/instances/.directio.test'"" [Medium,Fix released] https://launchpad.net/bugs/1788922 - Assigned to Stephen Finucane (stephenfinucane) | 12:45 |
| *** dhill_ has quit IRC | 12:52 | |
| *** dhill_ has joined #openstack-infra | 12:53 | |
| *** kgiusti has joined #openstack-infra | 12:56 | |
| *** eharney has quit IRC | 12:58 | |
| *** r-mibu has joined #openstack-infra | 13:03 | |
| AJaeger | config-core, https://review.openstack.org/596984 is a simple job removal - please review. | 13:03 |
| AJaeger | config-core, and two grafana changes for review, please: https://review.openstack.org/595573 and https://review.openstack.org/595763 | 13:04 |
| fungi | ttx: did your gerrit connectivity clear up? | 13:08 |
| AJaeger | fungi: he pushed something through at least ;) | 13:08 |
| mpeterson | https://review.openstack.org/#/c/546734/ hasn't merged even though it has +2R+2V+W, is there something I'm missing? | 13:09 |
| AJaeger | mpeterson: does it have a dependency? Stacked on another chagne or a depends-on which are not merged? | 13:10 |
| mpeterson | AJaeger: no dep and the stacked-on change is merged | 13:10 |
| AJaeger | mpeterson: it's stecked on an older revision and gerrit cannot merge it - see the orange sticker besides "parent" | 13:11 |
| AJaeger | you need to rebase it ;( | 13:11 |
| AJaeger | mpeterson: that gerrit UI is not clear for this case :( | 13:11 |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Update query for nova functional test bug 1788922 https://review.openstack.org/597113 | 13:12 |
| openstack | bug 1788922 in OpenStack Compute (nova) "SRIOVServersTest.test_create_server_with_VF intermittently fails due to "FileNotFoundError: [Errno 2] No such file or directory: '/home/zuul/src/git.openstack.org/openstack/nova/instances/.directio.test'"" [Medium,Fix released] https://launchpad.net/bugs/1788922 - Assigned to Stephen Finucane (stephenfinucane) | 13:12 |
| mpeterson | AJaeger: gotcha, I thought that gerrit would take the selected strategy as it usually does when you have several patches against master | 13:13 |
| *** agopi has quit IRC | 13:13 | |
| *** tdasilva has joined #openstack-infra | 13:13 | |
| mpeterson | AJaeger: I rebased, I'll ask stable-maints to +W again, thanks | 13:13 |
| fungi | mpeterson: in particular it won't if its parent was an unmerged change in gerrit which then underwent additional revisions | 13:13 |
| fungi | mpeterson: because its parent is an object which will never appear in the branch's history, so a merge isn't an option | 13:14 |
| *** agopi has joined #openstack-infra | 13:14 | |
| mpeterson | fungi: that makes sense, thanks for the insight | 13:14 |
| ttx | fungi: it did finally get through. Took about 2min. If it persists I'll investigate deeper | 13:14 |
| *** jchhatbar has joined #openstack-infra | 13:14 | |
| *** jcoufal has joined #openstack-infra | 13:14 | |
| fungi | ttx: thanks, i'll keep an ear to the ground for any similar reports today | 13:14 |
| fungi | hopefully internet weather isn't stormy | 13:15 |
| AJaeger | ttx, did you see my comment on your review? Please remove that .zuul.yaml file, we will never import those. | 13:16 |
| *** Emine has quit IRC | 13:16 | |
| *** janki has quit IRC | 13:17 | |
| ttx | AJaeger: ok... The instructions on https://docs.openstack.org/infra/manual/creators.html#add-jobs-for-your-project are rather unclear on that | 13:19 |
| *** jchhatbar has quit IRC | 13:19 | |
| AJaeger | ttx, want to update - or shall I and you review? ;) | 13:20 |
| *** janki has joined #openstack-infra | 13:20 | |
| ttx | Let me try, that way I'll check on my git-review health again | 13:20 |
| AJaeger | thanks, ttx | 13:21 |
| ttx | fungi: looks like my git-review timeout is still around. Could be what frickler pointed to, some IPv6 misconfig on my part or my ISPs part | 13:22 |
| dhellmann | AJaeger : I had seen the comment but not done anything about it yet. I'll talk to prometheanfire | 13:24 |
| fungi | ttx: can you compare `ssh -4 -p 29418 review gerrit.openstack.org ls-projects` behavior vs -6 ? | 13:24 |
| AJaeger | dhellmann: fine, thanks | 13:24 |
| fungi | er, `ssh -4 -p 29418 review.openstack.org gerrit ... | 13:25 |
| ttx | fungi: -4 is instant, -6 takes a while | 13:26 |
| openstackgerrit | Thierry Carrez proposed openstack-infra/project-config master: Add openstack-map repository https://review.openstack.org/597037 | 13:26 |
| ttx | holds on Connecting to review.openstack.org [2001:4800:7819:103:be76:4eff:fe04:9229] port 29418. | 13:26 |
| AJaeger | dhellmann: I helped the openstack-ansible team to push 30+ python3-first changes through ;) | 13:26 |
| ttx | maybe some reverse resolution timeout ? | 13:27 |
| *** apetrich has joined #openstack-infra | 13:27 | |
| ttx | connect to address 2001:4800:7819:103:be76:4eff:fe04:9229 port 29418: Connection timed out | 13:28 |
| fungi | yeah, the delay you're seeing then is ssh falling back to v4 after timing out on v6 | 13:29 |
| fungi | next thing to do would be to traceroute to it and see if you get past your local gateway | 13:29 |
| *** efried is now known as efried_doc | 13:29 | |
| *** graphene has quit IRC | 13:30 | |
| fungi | a temporary workaround would be to put the v4 address of review.o.o in your local /etc/hosts so it's found before dns is consulted and then git-review will just use v4 | 13:30 |
| *** graphene has joined #openstack-infra | 13:32 | |
| *** stephenfin has quit IRC | 13:33 | |
| *** rtjure has quit IRC | 13:34 | |
| *** stephenfin has joined #openstack-infra | 13:34 | |
| ttx | fungi: yeah, looks like the ipv6 config was broken, as part of my eventful upgrade to 18.04 LTS | 13:36 |
| *** rtjure has joined #openstack-infra | 13:37 | |
| *** psachin has quit IRC | 13:39 | |
| *** Emine has joined #openstack-infra | 13:40 | |
| EmilienM | good morning here :-) - could someone take a look at https://review.openstack.org/592695 ? and let me know if someone is missing. Thanks | 13:44 |
| ttx | fungi: looks like ICMP being filtered, I'll dig deeper | 13:45 |
| dhellmann | AJaeger : thank you! | 13:45 |
| *** psachin has joined #openstack-infra | 13:49 | |
| *** eharney has joined #openstack-infra | 13:50 | |
| *** sthussey has joined #openstack-infra | 13:52 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/infra-manual master: Add details around initial .zuul.yaml addition https://review.openstack.org/597131 | 13:58 |
| *** psachin has quit IRC | 13:58 | |
| *** hongbin has joined #openstack-infra | 14:00 | |
| *** haleyb has joined #openstack-infra | 14:06 | |
| *** pabelanger has joined #openstack-infra | 14:08 | |
| *** yamamoto has quit IRC | 14:10 | |
| *** graphene has quit IRC | 14:12 | |
| *** graphene has joined #openstack-infra | 14:14 | |
| *** hamzy has quit IRC | 14:19 | |
| *** hamzy has joined #openstack-infra | 14:20 | |
| *** dpawlik has quit IRC | 14:21 | |
| *** armax has joined #openstack-infra | 14:22 | |
| *** anteaya has joined #openstack-infra | 14:23 | |
| *** graphene has quit IRC | 14:25 | |
| AJaeger | ttx, your change for infra-manual ^ looks fine - thanks | 14:26 |
| *** graphene has joined #openstack-infra | 14:27 | |
| *** pcaruana has quit IRC | 14:29 | |
| *** pcaruana has joined #openstack-infra | 14:30 | |
| ttx | AJaeger: wow, that error is pretty silly: ERROR: project openstack/openstack-map, description 'Base data to generate the OpenStack map and the Software section of the openstack.org website.': contains wrong word 'openstack', it should be 'OpenStack' | 14:31 |
| ttx | I'd argue it definitely should be openstack.org and not OpenStack.org. | 14:31 |
| ttx | AJaeger: let me know how you'd rather see me fix it... | 14:33 |
| fungi | to be pedantic about it, technically https://openstack.org/ redirects to https://www.openstack.org/ | 14:33 |
| ttx | would www.openstack.org go better ? | 14:33 |
| *** Bhujay has joined #openstack-infra | 14:33 | |
| * ttx tries | 14:34 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add config-error notifications drawer https://review.openstack.org/597147 | 14:34 |
| *** Bhujay has quit IRC | 14:34 | |
| fungi | i generally refer to it as www but... i agree the check as written is a bit silly nonetheless | 14:34 |
| *** Bhujay has joined #openstack-infra | 14:35 | |
| *** eharney_ has joined #openstack-infra | 14:37 | |
| *** eharney has quit IRC | 14:37 | |
| ttx | yes looks like "www.openstack.org" will pass :) | 14:38 |
| mnaser | infra-root: just wondering a little bit on the status of things on mirror.sjc1.vexxhost.openstack.org ? | 14:39 |
| *** r-daneel has joined #openstack-infra | 14:39 | |
| mnaser | (sorry if there are other things being done, that's fine, i just wanted to know where things were) | 14:40 |
| fungi | mnaser: last i saw, clarkb was working on getting puppet applied on the mirror | 14:40 |
| *** efried_doc is now known as efried | 14:41 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Use ansible git task so we can set umask https://review.openstack.org/596874 | 14:41 |
| fungi | looks like mirror.sjc1.vexxhost.openstack.org isn't in dns yet either | 14:41 |
| *** yamamoto has joined #openstack-infra | 14:42 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/project-config master: Add openstack-map repository https://review.openstack.org/597037 | 14:42 |
| *** yamamoto has quit IRC | 14:45 | |
| *** markvoelker has joined #openstack-infra | 14:45 | |
| *** alexchadin has quit IRC | 14:48 | |
| *** markvoelker has quit IRC | 14:49 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add config-errors notifications drawer https://review.openstack.org/597147 | 14:49 |
| *** alexchadin has joined #openstack-infra | 14:49 | |
| clarkb | mnaser: fungi: based on ianw's email to the infra list we have some changes proposed for a method to install puppet on the node | 14:51 |
| clarkb | I haven't reviewed them yet, but this is likely the next step in spinning up that server (get those changes in, get puppet installed, run puppet, update dns) | 14:52 |
| *** graphene has quit IRC | 14:52 | |
| mnaser | cool, glad i'm exercising your infra too :) | 14:52 |
| *** graphene has joined #openstack-infra | 14:55 | |
| *** pbourke has quit IRC | 14:55 | |
| clarkb | looks like the general setup is to install puppet each time through the ansible cron on all "puppet" group nodes | 14:55 |
| *** srwilkers has joined #openstack-infra | 14:55 | |
| clarkb | comes with testing too | 14:55 |
| clarkb | I have a meeting in a few minutes but will start reviewing that afterwards | 14:56 |
| *** e0ne has quit IRC | 14:56 | |
| *** nicolasbock has quit IRC | 14:56 | |
| *** Bhujay has quit IRC | 14:58 | |
| *** nicolasbock has joined #openstack-infra | 14:59 | |
| *** pcaruana has quit IRC | 15:00 | |
| *** hongbin has quit IRC | 15:00 | |
| *** pbourke has joined #openstack-infra | 15:01 | |
| *** jroll has quit IRC | 15:05 | |
| *** jroll has joined #openstack-infra | 15:06 | |
| *** dkehn has joined #openstack-infra | 15:09 | |
| *** dpawlik has joined #openstack-infra | 15:12 | |
| *** yamamoto has joined #openstack-infra | 15:15 | |
| *** dpawlik has quit IRC | 15:17 | |
| *** ykarel is now known as ykarel|away | 15:19 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/system-config master: Enable unattended upgrades https://review.openstack.org/596849 | 15:26 |
| *** gyee has joined #openstack-infra | 15:27 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/system-config master: Enable unattended upgrades https://review.openstack.org/596849 | 15:28 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: remove job settings for requirements repositories https://review.openstack.org/593660 | 15:31 |
| AJaeger | dhellmann: https://review.openstack.org/#/c/593660/ needs rebasing - and if you want to run the docs only on master, you should remove them again from requirements, don't you? | 15:31 |
| *** graphene has quit IRC | 15:31 | |
| dhellmann | I just rebased, let me check if they were imported | 15:31 |
| AJaeger | dhellmann: they were imported | 15:31 |
| AJaeger | dhellmann: thanks for rebasing... | 15:31 |
| dhellmann | oh, yeah, I'll fix that up | 15:32 |
| *** janki has quit IRC | 15:32 | |
| *** graphene has joined #openstack-infra | 15:33 | |
| *** david-lyle has quit IRC | 15:33 | |
| AJaeger | dhellmann: we can merge your change already... | 15:33 |
| AJaeger | dhellmann: want to leave the - publish-openstack-sphinx-docs template in project-config? | 15:33 |
| *** dklyle has joined #openstack-infra | 15:34 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/system-config master: Enable unattended upgrades https://review.openstack.org/596849 | 15:35 |
| AJaeger | corvus: could you review this infra-manual change about integrated-gate again, please? https://review.openstack.org/#/c/596268/ | 15:35 |
| *** ykarel|away has quit IRC | 15:36 | |
| *** ykarel|away has joined #openstack-infra | 15:37 | |
| dhellmann | AJaeger : yeah, since it should only run on master I want to leave it there. I'll write a follow-up to switch to the PTI job | 15:37 |
| AJaeger | dhellmann: will then review both together... | 15:38 |
| clarkb | corvus: I think the iptables changes may have broken gearman connectivity to logstash.openstack.org (for handing out log indexing jobs) | 15:39 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: fix documentation job for requirements repo https://review.openstack.org/597166 | 15:39 |
| dhellmann | AJaeger : ^^ | 15:39 |
| clarkb | I don't know it is urgent to address, but making note of it here while I am noticing it | 15:40 |
| corvus | clarkb: nifty | 15:40 |
| clarkb | (there is no ipv4 rule allowing 4730 in from the workers at least) | 15:40 |
| *** jhesketh has quit IRC | 15:41 | |
| openstackgerrit | Merged openstack-infra/infra-manual master: Add details around initial .zuul.yaml addition https://review.openstack.org/597131 | 15:41 |
| *** agopi has quit IRC | 15:41 | |
| *** jhesketh has joined #openstack-infra | 15:41 | |
| clarkb | probably similar problems submitting the gearman jobs from our zuul jobs too | 15:41 |
| *** jistr is now known as jistr|call | 15:42 | |
| *** ykarel|away has quit IRC | 15:42 | |
| AJaeger | dhellmann: fine, let's merge 597166 and https://review.openstack.org/#/c/593660/ together | 15:42 |
| AJaeger | config-core, please review ^ | 15:42 |
| *** jistr|call is now known as jistr | 15:43 | |
| corvus | clarkb: i'll investigate | 15:43 |
| dhellmann | AJaeger : Ok, I've removed my WIP | 15:43 |
| AJaeger | dhellmann: great. let's finish that one... Btw. regarding oslo, I saw there was one missing change - would be great to get that finished... | 15:44 |
| *** dtantsur is now known as dtantsur|afk | 15:44 | |
| dhellmann | yeah, we're having some problems with the tests in that repo | 15:45 |
| dhellmann | AJaeger : oh, maybe you can help there, actually | 15:46 |
| dhellmann | it's a post failure, and ARA has no details: http://logs.openstack.org/80/588780/1/check/castellan-functional-devstack/5a27eb1/ara-report/ | 15:46 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Correct iptables var names https://review.openstack.org/597167 | 15:46 |
| corvus | clarkb: ^ | 15:46 |
| dhellmann | AJaeger : I can't tell if it was just killed, or died, or what | 15:46 |
| AJaeger | dhellmann: let me check... | 15:46 |
| openstackgerrit | Nate Johnston proposed openstack-infra/project-config master: Match older neutron dashboards to HEAD one, bump versions https://review.openstack.org/597168 | 15:46 |
| clarkb | corvus: thanks! | 15:47 |
| dhellmann | AJaeger : bbiab | 15:47 |
| openstackgerrit | Merged openstack-infra/infra-manual master: Document shared queue and policy for integrated-gate https://review.openstack.org/596268 | 15:48 |
| clarkb | ok breakfast now, then sending parents off to the airport, then puppet install via ansible reviews | 15:48 |
| AJaeger | dhellmann: let's recheck, might be temporary ;( | 15:48 |
| corvus | Shrews: can you review https://review.openstack.org/597167 ? | 15:50 |
| Shrews | corvus: in a meeting w/ my boss. will in a bit | 15:51 |
| *** ginopc has quit IRC | 15:51 | |
| Shrews | corvus: done | 15:54 |
| Shrews | corvus: can you re-enqueue the latest zuul change again so we can test container push? | 15:54 |
| * Shrews works up a similar nodepool job | 15:55 | |
| corvus | Shrews: will do! | 15:56 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove legacy pbr jobs https://review.openstack.org/597169 | 15:57 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Add post job to push pbrx container images https://review.openstack.org/597171 | 15:57 |
| Shrews | pabelanger: ^^ may interest you | 15:58 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Install limestone CA on openstack using hosts https://review.openstack.org/593029 | 15:59 |
| *** udesale has quit IRC | 15:59 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config master: pbrx upload: remove protected flag https://review.openstack.org/597172 | 15:59 |
| corvus | Shrews: ^ | 15:59 |
| corvus | Shrews: protected is basically like final, except that it allows inheritance only within the current project. so neither one is compatible with expecting projects to set their own variables. | 16:00 |
| Shrews | so many rulez | 16:01 |
| *** Emine has quit IRC | 16:02 | |
| *** alexchadin has quit IRC | 16:03 | |
| *** eharney_ is now known as eharney | 16:04 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/system-config master: Enable unattended upgrades https://review.openstack.org/596849 | 16:05 |
| *** e0ne has joined #openstack-infra | 16:06 | |
| *** snapiri has quit IRC | 16:08 | |
| *** agopi has joined #openstack-infra | 16:08 | |
| *** jpena is now known as jpena|off | 16:11 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add config-errors notifications drawer https://review.openstack.org/597147 | 16:12 |
| pabelanger | corvus: Shrews: AJaeger: should we be concerned about jobs now being able to parent to the pbrx jobs and try to view docker credentials? | 16:12 |
| corvus | pabelanger: allowed-projects should take care of that | 16:13 |
| corvus | pabelanger: i think we need to rework the job so we don't need that, but for the moment, i think this will at least let us exercise it with zuul/nodepool | 16:15 |
| Shrews | corvus: how could we rework it? | 16:16 |
| pabelanger | okay, yah. I'm a little confused why any job would parent to it. | 16:16 |
| *** Bhujay has joined #openstack-infra | 16:17 | |
| corvus | Shrews: i don't know yet :) | 16:17 |
| corvus | pabelanger: i'm confused by your question. | 16:17 |
| corvus | or rather, i'm confused by your confusion. :) | 16:17 |
| pabelanger | let me read docs again, 1 sec | 16:18 |
| weshay | need reviews please https://review.openstack.org/#/c/596799/ | 16:18 |
| corvus | pabelanger: are you wondering why we needed to remove protected/final? that's because the project-pipeline job variant in zuul modified a variable. | 16:19 |
| pabelanger | Oooh | 16:20 |
| pabelanger | protected / final isn't just for job to parent too | 16:20 |
| pabelanger | right, I forgot about that | 16:20 |
| AJaeger | config-core, could you review https://review.openstack.org/#/c/593660/ and https://review.openstack.org/#/c/597166/ to finish python3-first for requirements repo, please? | 16:21 |
| *** Bhujay has quit IRC | 16:22 | |
| pabelanger | corvus: okay, even if we did parent and try to attack docker client, the post-run job will happen after the parent. I think my brain is caught up to yours :) | 16:23 |
| corvus | pabelanger: well, allowed-projects won't allow any other project to run it anyway. | 16:23 |
| corvus | pabelanger: (even if they inherit from it, they can't run it) | 16:23 |
| corvus | when inherited, allowed-projects always shrinks or stays the same, never grows. | 16:24 |
| pabelanger | yah, was mostly thinking if we ever forgot in zuul / nodepool. But agree with everything you'd said | 16:24 |
| corvus | yeah, and it's still a post-review job. so we'd have to forget a lot. :) | 16:24 |
| pabelanger | ++ | 16:24 |
| *** Bhujay has joined #openstack-infra | 16:30 | |
| pabelanger | 663928 | 16:32 |
| pabelanger | sorry | 16:33 |
| pabelanger | kids trying to play on laptop | 16:33 |
| openstackgerrit | Merged openstack-infra/project-config master: pbrx upload: remove protected flag https://review.openstack.org/597172 | 16:33 |
| corvus | Shrews: ^ queued | 16:35 |
| corvus | Shrews: sadly, there are 84 items in the post pipeline. :/ biab. | 16:35 |
| clarkb | pabelanger: could be a change number from the future | 16:36 |
| corvus | more than half of all accidental 2fa token emissions correspond to change numbers at this point :) | 16:38 |
| *** jpich has quit IRC | 16:39 | |
| pabelanger | indeed | 16:39 |
| AJaeger | thanks, clarkb for the reviews | 16:40 |
| *** anteaya has quit IRC | 16:40 | |
| AJaeger | config-core, two simple openstack-zuul-job cleanups: https://review.openstack.org/597169 and https://review.openstack.org/596984 for review, please | 16:40 |
| clarkb | corvus: for https://review.openstack.org/#/c/596966/2/.zuul.yaml does the regex there need a ^ prefix or is it assumed to always be a regex? | 16:42 |
| *** anteaya has joined #openstack-infra | 16:42 | |
| *** Bhujay has quit IRC | 16:43 | |
| *** Bhujay has joined #openstack-infra | 16:44 | |
| clarkb | looks like the jobs run on the followup change so it must work as a regex as is | 16:44 |
| *** ramishra has quit IRC | 16:44 | |
| *** aojea has quit IRC | 16:45 | |
| clarkb | corvus: for https://review.openstack.org/#/c/596968/23 and https://review.openstack.org/#/c/596997/3 the system-config-run-base test runs our playbooks/base.yaml playbook right? then we have testinfra suite of test cases that check the resulting state of the node? | 16:46 |
| AJaeger | question to the zuul experts: https://review.openstack.org/588780 has a job with a post_failure, and I cannot figure out what is wrong - log is http://logs.openstack.org/80/588780/1/check/castellan-functional-devstack/0a151b9/ . Could anybody help dhellmann and me here, please? | 16:46 |
| clarkb | AJaeger: I think ansible may have crashed or stopped abruptly. We probably need to look at executor logs for that /me greps | 16:49 |
| AJaeger | thanks, clarkb | 16:49 |
| dhellmann | thanks, AJaeger & clarkb | 16:50 |
| clarkb | AJaeger: dhellmann http://paste.openstack.org/show/728987/ | 16:52 |
| dhellmann | so, a bad role name? | 16:52 |
| dhellmann | I wonder what the right name of that role really is | 16:52 |
| *** signed8bit has joined #openstack-infra | 16:53 | |
| openstackgerrit | Merged openstack-infra/project-config master: remove job settings for requirements repositories https://review.openstack.org/593660 | 16:54 |
| openstackgerrit | Merged openstack-infra/project-config master: fix documentation job for requirements repo https://review.openstack.org/597166 | 16:54 |
| clarkb | dhellmann: I wonder if the role was renamed externally | 16:56 |
| dhellmann | I'm not finding any other uses | 16:56 |
| dhellmann | in hound | 16:56 |
| dhellmann | where would that be likely to exist? | 16:56 |
| clarkb | in devstack probably | 16:56 |
| clarkb | maybe tempest | 16:56 |
| openstackgerrit | Merged openstack-infra/system-config master: Correct iptables var names https://review.openstack.org/597167 | 16:57 |
| clarkb | dhellmann: I00d448c4e6b98a1f504b048c74eff4e110c0b511 both roles were merged into fetch-subunit-output | 16:57 |
| AJaeger | dhellmann: we renamed that ages ago - and at that point changed all users. I wonder how that sneaked in ;( | 16:57 |
| AJaeger | clarkb: you're faster ;) | 16:57 |
| corvus | clarkb: yeah, it's always a regex. | 16:58 |
| corvus | clarkb: re run-base: yes | 16:58 |
| corvus | clarkb: http://git.openstack.org/cgit/openstack-infra/system-config/tree/playbooks/zuul/run-base.yaml#n44 lines 44 and after are the interesting bits | 16:59 |
| corvus | clarkb: http://git.openstack.org/cgit/openstack-infra/system-config/tree/testinfra/test_base.py is testinfra | 16:59 |
| dhellmann | AJaeger : this is the stable/queens branch, so I'm not sure how it slipped by | 16:59 |
| clarkb | corvus: thanks! | 16:59 |
| corvus | clarkb: i anticipate we will grow additional jobs for each hostgroup, and additional testinfra test files as well. | 16:59 |
| AJaeger | dhellmann: hound searches only on master, might be we overlooked it - or they backported only one change... | 17:00 |
| corvus | clarkb: so eventually, we should have a system-config-run-etherpad job which runs the etherpad playbook and the test_etherpad testinfra test | 17:00 |
| corvus | file matchers will be important :) | 17:00 |
| dhellmann | AJaeger : ah, ok | 17:00 |
| AJaeger | dhellmann: I pushed Ic1844896b1d79216c0cb04592fdbe700d6fc043f - seems I missed queens ;/ sorry | 17:00 |
| dhellmann | I included the rename in https://review.openstack.org/588780 | 17:00 |
| clarkb | corvus: yup | 17:00 |
| AJaeger | dhellmann: thanks | 17:01 |
| dhellmann | thanks for the help debugging that, I would have never found it otherwise | 17:01 |
| AJaeger | thanks, clarkb ^ | 17:02 |
| * clarkb pops out again. Busy morning here trying to get parents to the airport | 17:03 | |
| clarkb | AJaeger: dhellmann no problem! | 17:03 |
| corvus | clarkb: we had a 30 minute discussion at the infra meeting about testing: http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-08-21-19.02.log.html#l-214 if you want the full discussion, but the cliff's notes are at https://etherpad.openstack.org/p/CJUWmx62al | 17:03 |
| openstackgerrit | Merged openstack-infra/system-config master: Add role integration jobs https://review.openstack.org/596018 | 17:08 |
| dmsimard | btw I just tagged 0.16.0rc1 for ARA, changelog: https://github.com/openstack/ara/releases/tag/0.16.0.0rc1 | 17:11 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Generate junit.xml file for testinfra https://review.openstack.org/597185 | 17:12 |
| *** r-mibu has quit IRC | 17:12 | |
| *** trown is now known as trown|lunch | 17:13 | |
| pabelanger | corvus: any more thoughts to zuul console getting blocked by iptables in system-config base job? | 17:15 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Rework testinfra junit collection https://review.openstack.org/597188 | 17:16 |
| *** hashar is now known as hasharAway | 17:16 | |
| corvus | pabelanger: ^ what do you think of that? | 17:16 |
| openstackgerrit | Merged openstack-infra/system-config master: Only run role integration tests on role changes https://review.openstack.org/596966 | 17:16 |
| corvus | pabelanger: (also, i believe the original patch will fail because of the zuul.project.src_dir typo) | 17:17 |
| corvus | pabelanger: no i haven't thought about a solution to that yet. | 17:17 |
| pabelanger | when: item is file | 17:18 |
| pabelanger | that is new | 17:18 |
| corvus | yeah, i think it's revolutionary :) | 17:18 |
| pabelanger | yah, I kinda like it | 17:19 |
| corvus | pretty sure it works :) | 17:19 |
| corvus | pabelanger: if you like it, maybe just squash that into your change since it's going to need an update anyway | 17:20 |
| pabelanger | as for your patch, I was thinking of generating a report of junit with junit2html, but maybe that should be a zuul-job, like we do for subunit2html | 17:20 |
| pabelanger | but if works, I am fine with it | 17:20 |
| corvus | pabelanger: i think a junit role in zuul-jobs would be great | 17:21 |
| *** graphene has quit IRC | 17:21 | |
| *** graphene has joined #openstack-infra | 17:23 | |
| pabelanger | cool | 17:23 |
| *** Bhujay has quit IRC | 17:24 | |
| *** apetrich has quit IRC | 17:32 | |
| *** graphene has quit IRC | 17:32 | |
| AJaeger | corvus, pabelanger, I think we need https://review.openstack.org/#/c/596645/ for linting - and to approve https://review.openstack.org/#/c/590636/ . change is for openstack-zuul-jobs with title "Add zuul roles in system-config to linters" | 17:32 |
| notmyname | fungi: corvus: with patches like https://review.openstack.org/#/c/596206/1/.zuul.yaml, the gate queue is defined in-repo. what would happen if there were a typo there? | 17:33 |
| *** graphene has joined #openstack-infra | 17:34 | |
| Shrews | corvus: pabelanger: i *think* the unattended-upgrades review (https://review.openstack.org/596849) is ready now, duplicating the puppet-unattended_upgrades stuff | 17:34 |
| notmyname | eg if instead of "queue: integrated" I typo'd it as "queue: intgrated" (or "queue: swift-only"--the letters are so close to each other!) | 17:34 |
| AJaeger | notmyname: try it ;) | 17:34 |
| AJaeger | notmyname: don't add integrated queue - it should stay in project-config | 17:34 |
| notmyname | AJaeger: I'm pretty sure that patch is infallible as proposed. it was done by some automated script somewhere | 17:35 |
| notmyname | ;-) | 17:35 |
| AJaeger | notmyname: we fixed the scripts to not propose integrated queue anymore ;) | 17:35 |
| notmyname | but what would happen? I saw the line, and it makes me think about the super long integrated gate queue times and seeing as it's a variable proposed in our own repo and variables can be changed.... you know ;-) | 17:37 |
| *** signed8bit is now known as signed8bit_Zzz | 17:37 | |
| AJaeger | notmyname: if you change it, you create a new queue ... | 17:38 |
| AJaeger | you can opt-out of the queue. | 17:39 |
| AJaeger | But we don't want other projects to opt-in without review - and make it even larger. | 17:39 |
| corvus | notmyname, AJaeger: the appearance in project-config will take precedence. you can not opt out of the queue. | 17:39 |
| corvus | it almost seems like zuul might have been designed to prevent projects from unilaterally leaving a shared change queue without achieving wider consensus first. i don't recall if any specific examples came to mind during the design phase... ;) | 17:39 |
| AJaeger | corvus: he can opt-out if he removes in project-config | 17:39 |
| AJaeger | and that needs review | 17:39 |
| notmyname | I was just curious, not actually proposing anything | 17:40 |
| corvus | AJaeger: right, he can opt out if the project achieves consensus :) | 17:40 |
| AJaeger | notmyname: I'm removing it from the in-repo file now... | 17:41 |
| corvus | the integrated queue is the only one currently managed this way because it's special to the project | 17:41 |
| openstackgerrit | Dan Radez proposed openstack-infra/project-config master: adding readthedocs to networking-ansible https://review.openstack.org/597194 | 17:41 |
| notmyname | AJaeger: ok | 17:41 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Test puppet is installed on base https://review.openstack.org/597195 | 17:49 |
| *** signed8bit_Zzz is now known as signed8bit | 17:49 | |
| clarkb | ianw: corvus ^ we probably want to squash that chagne down into the others but I wanted to try my hand at writing testinfra tests | 17:49 |
| clarkb | now to review the depends on | 17:49 |
| *** markvoelker has joined #openstack-infra | 17:51 | |
| *** anteaya has quit IRC | 17:52 | |
| *** r-daneel_ has joined #openstack-infra | 17:53 | |
| *** r-daneel has quit IRC | 17:53 | |
| *** r-daneel_ is now known as r-daneel | 17:53 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Generate junit.xml file for testinfra https://review.openstack.org/597185 | 17:54 |
| AJaeger | dhellmann: now castellan fails completely - http://logs.openstack.org/80/588780/2/check/castellan-functional-devstack/07a5c32/ara-report/result/352e99cf-0357-4e8f-bcfc-35d4b5ff2513/ | 17:58 |
| corvus | clarkb: lgtm | 17:58 |
| corvus | clarkb: i +2d its parent | 17:59 |
| corvus | but not gparent | 17:59 |
| clarkb | corvus: thanks | 17:59 |
| AJaeger | clarkb, corvus, what about https://review.openstack.org/#/c/596645/ ? do we need this for linting? | 17:59 |
| corvus | AJaeger: i think so -- i think if a playbook in project-config wants to use a role in system-config that'll need to be there. | 18:03 |
| AJaeger | thanks, corvus | 18:04 |
| *** signed8bit is now known as signed8bit_Zzz | 18:06 | |
| *** shardy has quit IRC | 18:06 | |
| *** signed8bit_Zzz is now known as signed8bit | 18:07 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Generate junit.xml file for testinfra https://review.openstack.org/597185 | 18:08 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: write-inventory: Add groups to created inventory https://review.openstack.org/596994 | 18:10 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Add zuul roles in system-config to linters https://review.openstack.org/596645 | 18:10 |
| dhellmann | AJaeger : thanks. I'll see if the oslo team can help with that. | 18:13 |
| AJaeger | dhellmann: one optoin: Make it non-voting for now - merge, add new change that is voting ;) | 18:13 |
| dhellmann | yeah | 18:14 |
| AJaeger | dhellmann: that unblocks us... | 18:14 |
| * AJaeger likes to get the large project-config change in... | 18:14 | |
| AJaeger | infra-root, I'm approving pabelanger's change https://review.openstack.org/#/c/591457/ to "Promote test-emit-job-header change" in 5 mins... | 18:14 |
| AJaeger | infra-root, I just approved "Remove legacy-install-afs-with-puppet role" - https://review.openstack.org/590636 | 18:17 |
| *** electrofelix has quit IRC | 18:23 | |
| *** tosky has quit IRC | 18:23 | |
| openstackgerrit | Kendall Nelson proposed openstack-infra/project-config master: Add Freezer to StoryBoard https://review.openstack.org/596918 | 18:23 |
| *** trown|lunch is now known as trown | 18:24 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy-install-afs-with-puppet role https://review.openstack.org/590636 | 18:24 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Test puppet is installed on base https://review.openstack.org/597195 | 18:25 |
| openstackgerrit | Kendall Nelson proposed openstack-infra/project-config master: Add Freezer to StoryBoard https://review.openstack.org/596918 | 18:25 |
| clarkb | corvus: ^ hopefully that minor syntax issue was the only problem | 18:25 |
| openstackgerrit | Kendall Nelson proposed openstack-infra/project-config master: Enable Searchlight to Use StoryBoard https://review.openstack.org/596916 | 18:26 |
| *** r-daneel_ has joined #openstack-infra | 18:31 | |
| *** r-daneel has quit IRC | 18:32 | |
| *** r-daneel_ is now known as r-daneel | 18:32 | |
| *** markvoelker has quit IRC | 18:32 | |
| *** jpena|off has quit IRC | 18:33 | |
| *** markvoelker has joined #openstack-infra | 18:33 | |
| *** jpena|off has joined #openstack-infra | 18:33 | |
| AJaeger | dhellmann: wnat me to split up that castellan change? | 18:36 |
| *** markvoelker has quit IRC | 18:37 | |
| clarkb | fungi: did a new gpg key get generated while I was pulling crab pots out of the Nehalem Bay? | 18:41 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Promote test-emit-job-header change https://review.openstack.org/591457 | 18:42 |
| pabelanger | corvus: Do you have an exmaple playbook using when: item is file? Your first change didn't seem to work. | 18:42 |
| *** dave-mccowan has joined #openstack-infra | 18:43 | |
| logan- | the 'is file' test only works on localhost. you will still have to use the stat module if you are testing a remote file iirc. | 18:43 |
| pabelanger | AJaeger: emit-job-header change is live | 18:45 |
| pabelanger | just seems a job use it | 18:46 |
| pabelanger | seen* | 18:46 |
| pabelanger | confirming in rdoproject now | 18:46 |
| AJaeger | pabelanger: seen it *working*? ;) good! | 18:46 |
| corvus | logan-, pabelanger: aha. too good to be true i guess. | 18:47 |
| pabelanger | ah, good to know | 18:47 |
| pabelanger | logan-: thanks! | 18:47 |
| fungi | clarkb: i'm working on it now actually. trying to use bridge.o.o and accumulating notes on some minor adjustments we need to make to the process to accommodate the newer gnupg version in bionic | 18:47 |
| fungi | some changes to behavior, configuration and option names | 18:48 |
| fungi | nothing major, but enough it would probably trip up people not familiar with it | 18:48 |
| clarkb | fungi: somewhat expected given how launch node has changed too :) | 18:49 |
| pabelanger | AJaeger: thanks, rdoproject is also good | 18:49 |
| AJaeger | \o/ | 18:49 |
| pabelanger | yah for distributed testing | 18:50 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Test puppet is installed on base https://review.openstack.org/597195 | 18:50 |
| * fungi is busy exhausting bridge.o.o's entropy pool apparently | 18:53 | |
| clarkb | oh is haveged installed? | 18:53 |
| clarkb | (could be something we need to add to the base package list) | 18:53 |
| fungi | especially since the default keysize has increased from 2048 to 3072 bytes | 18:53 |
| fungi | master key generation went pretty quickly, but i expect i deleted the entropy doing that | 18:54 |
| fungi | now the signing subkey is taking a while to generate as a result | 18:54 |
| Shrews | pabelanger: can you review https://review.openstack.org/596849 again when you get a moment? | 18:56 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Generate junit.xml file for testinfra https://review.openstack.org/597185 | 18:57 |
| *** cdent has joined #openstack-infra | 18:57 | |
| pabelanger | Shrews: sure, let me grab coffee first | 18:57 |
| fungi | infra-root: as cdent noted in #openstack-dev, planet.openstack.org seems to have no firewall rules allowing http/https connections | 18:58 |
| fungi | i'll see if we've got a typo somewhere in the parameters we passed in for it with the switch to the new ansible role | 18:58 |
| clarkb | that was the case with logstash rules | 18:58 |
| cdent | all my social media flowers are dying | 18:58 |
| fungi | cdent: in fairness, they were a bit over-watered to begin with ;) | 18:58 |
| openstackgerrit | Dan Radez proposed openstack-infra/project-config master: adding readthedocs to networking-ansible https://review.openstack.org/597194 | 18:59 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Add stable/rocky to periodic-stable templates https://review.openstack.org/596281 | 18:59 |
| clarkb | Meeting time over in #openstack-meeting | 19:00 |
| *** kgiusti has quit IRC | 19:05 | |
| pabelanger | Shrews: +2, we likely want to schedule the approval of that | 19:06 |
| pabelanger | to make sure nothing breaks | 19:06 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Correct inventory pattern for planet.openstack.org https://review.openstack.org/597211 | 19:07 |
| fungi | that ^ was easier than i expected! | 19:07 |
| *** e0ne has quit IRC | 19:08 | |
| *** markvoelker has joined #openstack-infra | 19:19 | |
| *** dpawlik has joined #openstack-infra | 19:21 | |
| *** e0ne has joined #openstack-infra | 19:23 | |
| *** dpawlik has quit IRC | 19:26 | |
| *** dave-mccowan has quit IRC | 19:26 | |
| *** cdent has left #openstack-infra | 19:29 | |
| *** owalsh_ has joined #openstack-infra | 19:31 | |
| openstackgerrit | Merged openstack-infra/system-config master: Enable unattended upgrades https://review.openstack.org/596849 | 19:33 |
| *** owalsh has quit IRC | 19:34 | |
| *** owalsh_ has quit IRC | 19:35 | |
| *** owalsh has joined #openstack-infra | 19:36 | |
| *** dpawlik has joined #openstack-infra | 19:38 | |
| *** dpawlik has quit IRC | 19:42 | |
| AJaeger | config-core, two small job removal changes from openstack-zuul-jobs, please add to your review queue: https://review.openstack.org/#/c/596984/ and https://review.openstack.org/#/c/597169/ | 19:47 |
| ianw | corvus: LMN if my comment in https://review.openstack.org/596968 convinces you | 19:47 |
| openstackgerrit | Merged openstack-infra/system-config master: Correct inventory pattern for planet.openstack.org https://review.openstack.org/597211 | 19:48 |
| clarkb | ianw: feel free to squash my test change into your stack however you like | 19:48 |
| clarkb | ianw: I was mostly interested in learning how testinfra things work so wrote that but don't care if it remains a separate change | 19:49 |
| fungi | corvus: basically they were able to find us a dining room in the hotel because there's another conference going on through sunday which had all their larger conference rooms booked up. if anyone's around sunday afternoon and wants to join the tc discussion, we're happy to squeeze in some extra chairs as well as we can accommodate them :/ | 19:49 |
| corvus | ianw: i think it does, particularly the bit about using it in the spec jobs. that seems pretty legit. | 19:49 |
| ianw | my system is running on borrowed time here. occasionally i wake up to my external monitor being totally borked, and there's no coming back from it. the moment i switch to a VT or pull the laptop from the docking station i'm in hard crash town | 19:49 |
| corvus | ianw, clarkb: i think i'm +2 on all related changes | 19:49 |
| clarkb | hrm maybe I should review them before eating lunch then | 19:50 |
| ianw | clarkb / corvus : one thought i had was should we stage it, by using a "puppet-ansible" group or something? i think our testing is pretty good, but i'd hate for it to roll out and then we have some weird puppet problems on every host | 19:51 |
| *** dpawlik has joined #openstack-infra | 19:52 | |
| clarkb | ianw: probably a good idea to do something like that | 19:53 |
| clarkb | looks like my test change needs to be rebased? | 19:53 |
| corvus | ianw: i don't think it's necessary; problems aren't likely to be especially end-user visible. probably the worst is puppet just stops working for a few mins. | 19:54 |
| clarkb | thats a good point puppet does a good job of failing noopy | 19:54 |
| clarkb | it just won't run | 19:54 |
| *** ccamacho has quit IRC | 19:54 | |
| fungi | infra-root: stein cycle artifact signing key is starting to show up on keyservers now (if you get a not found error, refresh or wait a bit and try again): https://sks-keyservers.net/pks/lookup?op=vindex&search=0x27023b1ffccd8e3ae9a5ce95d943d5d270273ada&fingerprint=on | 19:55 |
| clarkb | corvus: I had to fix another issue in https://review.openstack.org/#/c/597195/3 fwiw, zuul +1's now but gerrit says it cannot merge, not sure what is going on there | 19:55 |
| clarkb | I'm going to review ianw's changes now though and probably not approve as I need to eat | 19:55 |
| fungi | i've also got the symmertic decryption passphrase for the master key recorded in the usual secrets list now | 19:55 |
| Shrews | clarkb: it probably can't merge b/c my change to test_base.py just merged, so it now conflicts | 19:56 |
| clarkb | Shrews: ah | 19:56 |
| *** dpawlik has quit IRC | 19:56 | |
| *** jcoufal_ has joined #openstack-infra | 20:00 | |
| *** jcoufal has quit IRC | 20:01 | |
| clarkb | ianw: corvus ok I apprvoed the addition of the role and +2'd the enablement but didn't approve that as I need to afk for a bit | 20:01 |
| AJaeger | dhellmann: I removed my -W from https://review.openstack.org/#/c/588842 - want to WIP yourself or is it ready? | 20:02 |
| clarkb | I'll followup with a rebase to my test addition to fix the conflict when I return | 20:02 |
| *** vivsoni has quit IRC | 20:03 | |
| ianw | clarkb: ok, i'll grab some breakfast too and better to do it when we can watch it | 20:03 |
| *** vivsoni has joined #openstack-infra | 20:03 | |
| dhellmann | AJaeger : that one is ready to go, let me send the official announcement email | 20:05 |
| dhellmann | AJaeger : http://lists.openstack.org/pipermail/openstack-infra/2018-August/006113.html | 20:07 |
| AJaeger | any other config-core to +2A https://review.openstack.org/#/c/588842, please? - moves most of oslo out of project-config. | 20:08 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Install python2.7 on xenial nodes https://review.openstack.org/596913 | 20:09 |
| AJaeger | dhellmann: great! | 20:09 |
| clarkb | Shrews: ^ rebased to add in the extra packages, can you double check I got all the potential conflicts handled there? | 20:09 |
| * AJaeger waves good night | 20:09 | |
| ianw | AJaeger: see you this afternoon ;) | 20:11 |
| *** owalsh_ has joined #openstack-infra | 20:14 | |
| *** owalsh has quit IRC | 20:17 | |
| *** owalsh- has joined #openstack-infra | 20:17 | |
| *** owalsh_ has quit IRC | 20:18 | |
| fungi | mmm, looks like the ansible runs are still taking longer than we'd hoped? | 20:25 |
| fungi | begin run @ 2018-08-28T19:45:01+00:00 | 20:25 |
| Shrews | clarkb: lgtm | 20:25 |
| fungi | still underway | 20:25 |
| *** rkukura has quit IRC | 20:25 | |
| openstackgerrit | Merged openstack-infra/system-config master: Create ansible roles to install puppet https://review.openstack.org/596968 | 20:26 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Add query for n-cpu startup MessagingTimeout bug 1789484 https://review.openstack.org/597231 | 20:27 |
| openstack | bug 1789484 in OpenStack Compute (nova) "n-cpu fails init on timeout calling n-cond-cell1" [Medium,Triaged] https://launchpad.net/bugs/1789484 | 20:27 |
| fungi | looks like the 19:00 pulse ran just over 44 minutes | 20:28 |
| fungi | 18:00 pulse ran 47 minutes | 20:29 |
| fungi | 17:00 pulse was 48 minutes | 20:29 |
| fungi | so relatively consistent at least | 20:29 |
| fungi | but means we're getting config updates out roughly every hour (sometimes every 45 minutes) | 20:29 |
| corvus | fungi: huh, it was 28 minutes when i did the iptables stuff | 20:30 |
| *** ansmith has quit IRC | 20:30 | |
| corvus | or rather, *before* i did the iptables stuff | 20:30 |
| corvus | so iptables and what comes after should be suspect | 20:30 |
| openstackgerrit | Merged openstack-infra/storyboard master: Add a test openid server https://review.openstack.org/397998 | 20:31 |
| corvus | yay, bridge is in cacti now! | 20:31 |
| fungi | sounds reasonable | 20:31 |
| corvus | maybe we can try increasing the forks? | 20:31 |
| pabelanger | fungi: do you know if we enabled pipelineing in ansible.cfg? that should help speed up things a little | 20:31 |
| fungi | did we merge the bump to -f20 yet? checking | 20:32 |
| fungi | yeah, we're going at f20 | 20:32 |
| fungi | i guess now that bridge is in cacti we can tune for system load fairly easily | 20:33 |
| corvus | pabelanger: yeah pipelining is on | 20:33 |
| *** rosmaita has quit IRC | 20:33 | |
| corvus | i think bridge may not have enough ram | 20:33 |
| pabelanger | corvus: do you think running ara on bridge.o.o is doable at this point? Or should we review secrets | 20:33 |
| corvus | considering puppetmaster ooms constantly, i'm not sure we should have made it the same size | 20:34 |
| pabelanger | corvus: ack | 20:34 |
| pabelanger | ah, yah, we should make it larger them puppetmaster for RAM | 20:34 |
| notmyname | is etcd3 failing in devstack something that is known? http://logs.openstack.org/97/591497/7/check/swift-dsvm-functional-ipv6/484544c/job-output.txt.gz#_2018-08-28_17_44_17_378451 | 20:35 |
| *** owalsh- is now known as owalsh | 20:37 | |
| openstackgerrit | Rafael Folco proposed openstack-infra/zuul-jobs master: Set zuul job start time fact https://review.openstack.org/597235 | 20:38 |
| *** timburke has joined #openstack-infra | 20:38 | |
| *** slaweq has quit IRC | 20:40 | |
| pabelanger | notmyname: looks like config issue | 20:40 |
| pabelanger | http://logs.openstack.org/97/591497/7/check/swift-dsvm-functional-ipv6/484544c/controller/logs/screen-etcd.txt.gz#_Aug_28_17_44_17_701051 | 20:40 |
| pabelanger | not sure if reported already | 20:40 |
| *** slaweq has joined #openstack-infra | 20:40 | |
| *** priteau has quit IRC | 20:43 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Add query for n-cpu startup MessagingTimeout bug 1789484 https://review.openstack.org/597231 | 20:47 |
| openstack | bug 1789484 in OpenStack Compute (nova) "n-cpu fails init on timeout calling n-cond-cell1" [Medium,Triaged] https://launchpad.net/bugs/1789484 | 20:47 |
| openstackgerrit | Rafael Folco proposed openstack-infra/project-config master: Switch to emit-zuul-job-header-test role https://review.openstack.org/597236 | 20:47 |
| fungi | doesn't currently look like bridge.o.o is under significant memory pressure. it's got a fairly steady cache/buffers and active used is only spiking up to about 50% | 20:48 |
| fungi | swap is virtually (pun intended) untouched | 20:48 |
| fungi | cpu looks to me like a bit more of a concern. i expect it's maxing out or coming close for less than the snmp polling granularity | 20:49 |
| fungi | but load average spiking up around 4 is probably reasonable on a 2-core vm doing ansible activity? | 20:50 |
| ianw | notmyname: i haven't been watching super closely but i'm not aware of that issue. if it is an ipv6 thing it might be intermittent | 20:50 |
| fungi | though i do agree that if we double to -f40 or more we need more of... well everything | 20:51 |
| *** e0ne has quit IRC | 20:58 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add private key storage migration https://review.openstack.org/592213 | 20:59 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Add profile_tasks and timer callbacks for ansible https://review.openstack.org/597240 | 20:59 |
| *** rkukura has joined #openstack-infra | 20:59 | |
| *** trown is now known as trown|outtypewww | 21:00 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Add post job to push pbrx container images https://review.openstack.org/597171 | 21:00 |
| pabelanger | infra-root: ^might be interested to get some additional output from ansible, in our logs on bridge.o.o. | 21:00 |
| pabelanger | http://logs.openstack.org/69/593369/3/check/windmill-src-ubuntu-bionic/624fda5/job-output.txt.gz#_2018-08-23_19_43_36_205186 would be an example of the new output format | 21:01 |
| pabelanger | I noticed we didn't have date/time stamps in the crontab logs | 21:01 |
| fungi | pabelanger: ianw added timestamps at start/end of runs | 21:01 |
| pabelanger | fungi: yah, this is more for per task timestamps | 21:01 |
| fungi | but yeah, what we have doesn't help figure out what's slow within a run | 21:02 |
| pabelanger | timer gives you: http://logs.openstack.org/69/593369/3/check/windmill-src-ubuntu-bionic/624fda5/job-output.txt.gz#_2018-08-23_19_54_41_138899 | 21:02 |
| fungi | just challenging the assertion that we don't have _any_ date/time stamps in the crontab logs | 21:02 |
| pabelanger | to show you the longest running tasks in order | 21:02 |
| *** markvoelker has quit IRC | 21:05 | |
| *** bnemec has quit IRC | 21:12 | |
| pabelanger | cool | 21:12 |
| pabelanger | http://logs.openstack.org/40/597240/1/check/system-config-run-base/957a4b2/job-output.txt.gz#_2018-08-28_21_09_26_829385 | 21:12 |
| *** bnemec has joined #openstack-infra | 21:13 | |
| pabelanger | is new output we'd see on bridge.o.o | 21:13 |
| pabelanger | <3 the testing here for system-config | 21:13 |
| pabelanger | almost like one day, we'll see zuul kick ansible for the control plane :D | 21:13 |
| *** eharney has quit IRC | 21:16 | |
| clarkb | ianw: corvus ok I'm back to computer now ready to do puppet things with ansible when you are | 21:17 |
| *** jcoufal_ has quit IRC | 21:18 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Don't use loops with package task https://review.openstack.org/597241 | 21:18 |
| pabelanger | ^will be a little preformance increase too | 21:18 |
| clarkb | I'm working on an update to the testinfra change to make it not conflict with shrews change and to check the negative case | 21:18 |
| corvus | fungi, pabelanger: the ansible log has timestamps | 21:19 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Don't use loops with package task https://review.openstack.org/597241 | 21:20 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Test puppet is installed on base https://review.openstack.org/597195 | 21:21 |
| clarkb | I'm hoping ^ won't conflict and can merge as is without needing to rebase the parent change | 21:21 |
| pabelanger | corvus: we seem to have 2 logs, doing same thing but different output? | 21:22 |
| corvus | pabelanger: yep | 21:22 |
| pabelanger | k | 21:22 |
| corvus | yep | 21:22 |
| pabelanger | don't have time to look at why that is now, will dig more later | 21:22 |
| corvus | ++ | 21:22 |
| clarkb | ianw: corvus if concerned about it we could make a one off playbook that uses the role that merged, run that against the new mirror to confirm it works to bootstrap puppet then run it against logstash-worker01 to make sure it doens't regress existing servers, then merge it | 21:22 |
| corvus | i don't think it's intentional or finished :) | 21:23 |
| clarkb | corvus: pabelanger ya bridge.o.o is currently under construction, don't assume the entire road deck is safe just yet :) | 21:23 |
| corvus | clarkb: i have full confidence in the ci tests :) | 21:23 |
| clarkb | corvus: that also works for me assuming my latest change turns out happy | 21:24 |
| *** holser_ has quit IRC | 21:24 | |
| ianw | if i was doing it alone i'd probably do something like that, but if others are happy to be on stand-by i'm ok with it | 21:25 |
| fungi | my planet.o.o inventory globbing fix did eventually solve the overly-restrictive firewall rules, so all in all this seems to be working | 21:27 |
| fungi | no manual intervention was required, only a modicum of patience | 21:27 |
| *** ansmith has joined #openstack-infra | 21:28 | |
| clarkb | ianw: maybe wait for test results on my test update and approve if it comes back green? | 21:29 |
| clarkb | ianw: I'll be around to help with that | 21:29 |
| *** owalsh has quit IRC | 21:29 | |
| weshay | need a review on getting pending centos changes under ci so upstream doesn't fall flat | 21:29 |
| weshay | https://review.openstack.org/#/c/596799/ | 21:29 |
| weshay | mordred, ^ | 21:29 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: unbound: fix dhclient.conf copy https://review.openstack.org/597242 | 21:29 |
| weshay | thanks | 21:29 |
| corvus | pabelanger: https://review.openstack.org/597242 is a result of our earlier conversation | 21:30 |
| *** owalsh has joined #openstack-infra | 21:34 | |
| *** dpawlik has joined #openstack-infra | 21:35 | |
| *** jtomasek has quit IRC | 21:38 | |
| ianw | weshay: do you know how big it is? | 21:39 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: remove job settings for heat repositories https://review.openstack.org/597289 | 21:40 |
| weshay | ianw, I don't but can check | 21:40 |
| ianw | weshay: mirror has enough overhead anyway http://grafana.openstack.org/d/ACtl1JSmz/afs?orgId=1 | 21:40 |
| openstackgerrit | Ben Nemec proposed openstack-dev/pbr master: Ignore --find-links in requirements file https://review.openstack.org/597290 | 21:42 |
| *** boden has quit IRC | 21:42 | |
| ianw | infra-root / weshay : given the trivial nature of 596799 just removing an excludes from the rsync, i'll take the liberty of merging it, but take the mirror lock and do the first run manually just to be sure | 21:42 |
| clarkb | weshay: wouldn't jobs break when cr is updated if you start using it? or would you restrict that to a subset of jobs? | 21:43 |
| clarkb | (mostly just questioning the idea that if we mirror those packages somehow ci won't break anymore) | 21:43 |
| weshay | clarkb, ianw I have a test job that check it out.. it won't impact the normal check/gate jobs | 21:43 |
| clarkb | ianw: wfm | 21:44 |
| weshay | pabelanger, ok by you? | 21:44 |
| weshay | thanks all | 21:45 |
| *** rcernin has joined #openstack-infra | 21:46 | |
| ianw | it piqued my interest and might have some implications for dib testing too. i wasn't aware of it | 21:46 |
| weshay | cool | 21:47 |
| weshay | it delivers unicorns in rpm format | 21:47 |
| *** graphene has quit IRC | 21:48 | |
| *** rkukura has quit IRC | 21:49 | |
| *** graphene has joined #openstack-infra | 21:49 | |
| *** hasharAway has quit IRC | 21:50 | |
| clarkb | ianw: corvus https://review.openstack.org/#/c/597195/4 came back +1, ianw want to +A https://review.openstack.org/#/c/596997/3 ? | 21:51 |
| corvus | clarkb: i +3d both | 21:52 |
| clarkb | corvus: thanks! | 21:52 |
| *** rkukura has joined #openstack-infra | 21:56 | |
| *** weshay is now known as weshay|pto | 21:58 | |
| clarkb | ianw: fungi reviewing the git-review release notes change and its parent it doesn't look like the parent change renders docs using the new theme as expected: http://logs.openstack.org/86/594186/1/check/tox-docs/93effe7/html/ | 22:00 |
| clarkb | http://logs.openstack.org/86/594186/1/check/tox-docs/93effe7/job-output.txt.gz#_2018-08-21_14_04_46_092936 though that indicates it should be using the theme | 22:00 |
| clarkb | dhellmann: ^ does that look familiar to you? | 22:01 |
| dhellmann | it looks like the theme is being installed but not configured in conf.py? | 22:02 |
| dhellmann | it may rely on the old oslosphinx behavior, where the theme forced its use | 22:02 |
| dhellmann | the openstackdocstheme package doesn't do that | 22:02 |
| clarkb | dhellmann: https://review.openstack.org/#/c/594186/1 something like that? maybe that is insufficient | 22:03 |
| clarkb | or the name is wrong, gives me a place to look thanks | 22:03 |
| dhellmann | https://docs.openstack.org/openstackdocstheme/latest/ covers it | 22:03 |
| clarkb | aha needs html_theme | 22:03 |
| dhellmann | yeah | 22:04 |
| clarkb | now I wonder if that ommission is intentional | 22:04 |
| clarkb | fungi: ^ I'm happy to push up a new ps if you want, but any thoughts on if we want the default theme or not? current docs are openstacky | 22:05 |
| ianw | oh heh, sorry was too focused on thinking about the rtd issue with dependencies | 22:07 |
| fungi | clarkb: fine by me. i'm not too concerned with the docs looking *temporarily* openstacky since winterscale will have us revisiting all of that soonish (i hope) | 22:07 |
| fungi | having docs that build and that we can refer users to is more important to me | 22:08 |
| openstackgerrit | Merged openstack-infra/system-config master: allow cr repo mirror for early testing https://review.openstack.org/596799 | 22:10 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add private key storage migration https://review.openstack.org/592213 | 22:10 |
| openstackgerrit | Clark Boylan proposed openstack-infra/git-review master: Use doc/requirements and openstackdocstheme https://review.openstack.org/594186 | 22:13 |
| openstackgerrit | Clark Boylan proposed openstack-infra/git-review master: Start using reno for releasenotes https://review.openstack.org/593670 | 22:13 |
| clarkb | fungi: ianw ^ ok fixed | 22:13 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Add mitogen support for ansible on bridge.o.o https://review.openstack.org/597297 | 22:16 |
| *** dpawlik has quit IRC | 22:18 | |
| openstackgerrit | Merged openstack-infra/system-config master: Add install-puppet to base playbook https://review.openstack.org/596997 | 22:18 |
| *** gema has quit IRC | 22:19 | |
| clarkb | ianw: corvus ^ here we go | 22:19 |
| clarkb | hrm /me needs ssh key to tail logs for that | 22:19 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Add mitogen support for ansible on bridge.o.o https://review.openstack.org/597297 | 22:19 |
| corvus | clarkb: yell if you need me :) | 22:20 |
| openstackgerrit | Merged openstack-infra/system-config master: Test puppet is installed on base https://review.openstack.org/597195 | 22:21 |
| ianw | speaking of that logfile, i haven't found anything ansible quite as nice as the puppet logrotate classes | 22:22 |
| ianw | i guess can just manually drop a template file in | 22:23 |
| clarkb | puppet is currently running on the else set of servers | 22:25 |
| clarkb | I'm guessing we'll see the above chagne in action at the beginning of the next hour | 22:25 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Change project.private_key to private_secrets_key https://review.openstack.org/597302 | 22:27 |
| clarkb | fungi: ianw http://logs.openstack.org/86/594186/2/check/tox-docs/715cd06/html/ looks better | 22:31 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Add profile_tasks and timer callbacks for ansible https://review.openstack.org/597240 | 22:32 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Don't use loops with package task https://review.openstack.org/597241 | 22:32 |
| clarkb | ianw: corvus if I am reading logs correctly 22:45UTC will begin the run that includes the check puppet is installed role | 22:33 |
| pabelanger | infra-root: https://review.openstack.org/597297/ might be of interest, if we want to try using mitogen from bridge.o.o. Should results in a large performance of ansible, at least the docs say so. | 22:36 |
| pabelanger | https://review.openstack.org/597241/ is also an optimzation of package tasks, we can directly pass a list of packages to avoid multiple package calls with loop | 22:37 |
| clarkb | we probably want to instrument where the time losses are before we start throwing things at it? (small changes like handing package a list of packages is probably fine though) | 22:38 |
| corvus | clarkb: indeed -- 597240 does that :) | 22:38 |
| mnaser | fwiw i didn't see a huge impact in our usage of ansible involving mitogen tbh | 22:39 |
| mnaser | that being said, 'dw' from #mitogen is super super happy to take on these things and find space for improving performance | 22:39 |
| corvus | pabelanger, clarkb: and i believe the base-test job will allow us to preview all of these (at a smaller scale, obviously) | 22:39 |
| *** sshnaidm is now known as sshnaidm|afk | 22:39 | |
| pabelanger | corvus: ++ that is the exciting part for me | 22:40 |
| clarkb | ya I'm reading docs and comparing against the base test job now | 22:44 |
| clarkb | I've approved the profiling change | 22:45 |
| clarkb | ansible is running again and should include the new role | 22:46 |
| clarkb | watching tail output setting user groups is really slow | 22:48 |
| clarkb | the profiling change should give us much better info though | 22:49 |
| *** agopi has quit IRC | 22:51 | |
| *** signed8bit is now known as signed8bit_Zzz | 22:51 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: base-test: iptables: allow zuul console streaming https://review.openstack.org/597305 | 22:53 |
| corvus | pabelanger: ^ that's the best i've come up with so far. when we get around to implementing mordred's forwarded-tcp-socket idea for zuul console logs, we shouldn't need that anymore. but that's unlikely in the short term. | 22:54 |
| corvus | (maybe that's what we should do on friday in denver) | 22:54 |
| clarkb | unbound is being configured now, next should be puppet install | 22:58 |
| clarkb | puppet-install is running now | 22:59 |
| clarkb | ianw: corvus the only suprising thing to me is the remove server line from config showed as changed for all the nodes but the sjc1 mirror | 23:01 |
| clarkb | looks like we failed to git clone system-config because it already exists | 23:02 |
| clarkb | we may not want to manage those directly and instead rely on ansible-role-puppet to rsync them over? | 23:03 |
| pabelanger | yah, if we rsync them over, shouldn't need to re git clone them | 23:03 |
| *** tpsilva has quit IRC | 23:04 | |
| corvus | why would it fail though? | 23:04 |
| clarkb | 2018-08-28 23:02:01,643 p=29810 u=root | fatal: [mirror01.cn1.linaro.openstack.org]: FAILED! => {"changed": false, "cmd": "/usr/bin/git clone --origin origin https://git.openstack.org/openstack-infra/system-config /opt/system-config", "msg": "fatal: destination path '/opt/system-config' already exists and is not an empty directory.", "rc": 128, "stderr": "fatal: destination path '/opt/system-config' | 23:05 |
| clarkb | already exists and is not an empty directory.\n", "stderr_lines": ["fatal: destination path '/opt/system-config' already exists and is not an empty directory."], "stdout": "", "stdout_lines": []} | 23:05 |
| clarkb | this short circuited the run of disable-puppet-agent on all those nodes (which is fine because we already ran it on those nodes, but we should figure it out) | 23:05 |
| pabelanger | what is origin set for /opt/system-config? | 23:06 |
| corvus | clarkb: that's this task? http://git.openstack.org/cgit/openstack-infra/system-config/tree/roles/puppet-install/tasks/main.yaml#n22 | 23:06 |
| pabelanger | I've seen some issues with git task, if origin is missing or different | 23:07 |
| clarkb | corvus: ya | 23:07 |
| pabelanger | oh | 23:07 |
| pabelanger | isn't it /opt/system-config/production? | 23:07 |
| clarkb | looking at /etc/puppet/puppet.conf they differ between node that just had puppet installed and existing mirror node. Howver I think this is fine because ansible-role-puppet writes out the config we want | 23:07 |
| corvus | pabelanger: yep | 23:07 |
| clarkb | aha | 23:08 |
| pabelanger | I can push up a patch, 1 sec | 23:08 |
| clarkb | looks like puppet successfully ran on zuul01 | 23:10 |
| *** r-daneel has quit IRC | 23:10 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Fix git clone issue with puppet-install https://review.openstack.org/597308 | 23:10 |
| clarkb | ianw: we might be able to remove some of the puppet.conf munging based on what ansible-role-puppet does which may speed things up slightly | 23:10 |
| *** signed8bit_Zzz is now known as signed8bit | 23:11 | |
| ianw | the config removal bits? | 23:11 |
| fungi | wow, just got 28 messages from the zk servers about hostname resolution failures for sudo along with all the resulting bounces from the rest of the sysadmin forwarding addresses who reject e-mail from systems with no resolvable helo | 23:11 |
| corvus | fungi: me too. (i get the bounces for the messages i reject, since the bounces have no envelope sender). | 23:12 |
| ianw | clarkb: i think also disable-puppet-agent might be able to be removed and just in puppet-install now? | 23:12 |
| clarkb | ianw: the role removes the server line from puppet.conf and we also clone system-config and clone install /etc/puppet/modules but ansible-role-puppet will set its own puppet.conf and it will copy over the /etc/puppet/modules and system-config to match what is on bridge (so that things work in sync) | 23:13 |
| clarkb | ianw: ya that too | 23:13 |
| *** jcoufal has joined #openstack-infra | 23:13 | |
| *** holser_ has joined #openstack-infra | 23:13 | |
| clarkb | ianw: I think if we ensure the service is disabled and the default file is in place and the service isn't running we are good for ansible-role-puppet to do what it needs from there | 23:14 |
| clarkb | fungi: I don't think ansible was talking to the zk servers yet | 23:15 |
| clarkb | oh though the puppet install was to all of them | 23:16 |
| clarkb | and may have sudo'd things | 23:16 |
| fungi | that's probably what triggered it | 23:16 |
| pabelanger | the zk sudo issue was to deal with zookeeper using hostname in /etc/hosts when binding | 23:16 |
| pabelanger | we've never fixed that | 23:17 |
| pabelanger | I guess we could stop those servers, we haven't migrated zookeeper yet either | 23:17 |
| clarkb | ianw: I htink those were bootstrap speedups that make less sense running every cron trigger | 23:18 |
| *** holser_ has quit IRC | 23:20 | |
| ianw | happy to have it culled ... i just took a best guess at what of install_puppet was still relevant and what wasn't :) | 23:20 |
| pabelanger | clarkb: ianw: https://review.openstack.org/597308/ should fix our puppet-install issue | 23:20 |
| clarkb | pabelanger: I think we actually want to just not clone it at all there | 23:21 |
| clarkb | ansible-role-puppet manages that itself | 23:21 |
| openstackgerrit | Merged openstack-infra/system-config master: Add profile_tasks and timer callbacks for ansible https://review.openstack.org/597240 | 23:22 |
| pabelanger | clarkb: trying to see when we run that | 23:22 |
| *** agopi has joined #openstack-infra | 23:23 | |
| clarkb | pabelanger: https://git.openstack.org/cgit/openstack-infra/ansible-role-puppet/tree/tasks/main.yaml#n116 is part of it | 23:23 |
| clarkb | pabelanger: https://git.openstack.org/cgit/openstack-infra/ansible-role-puppet/tree/tasks/main.yaml#n82 too | 23:24 |
| pabelanger | clarkb: right, but we don't run that in base playbooks, from what I see. It seems to still be remote_puppet_adhoc / remote_puppet_afs | 23:24 |
| pabelanger | etc | 23:24 |
| pabelanger | I guess we need to pull that role into base? | 23:25 |
| pabelanger | or just let later roles do it | 23:25 |
| clarkb | ya its in the later playbooks | 23:25 |
| clarkb | https://git.openstack.org/cgit/openstack-infra/system-config/tree/playbooks/remote_puppet_else.yaml | 23:26 |
| clarkb | we do it that way so that each group of servers (git, afs, else) use a consistent set of git repos for their runs | 23:26 |
| clarkb | so the repo setup in base is redundant | 23:26 |
| pabelanger | clarkb: so I think we can delete the whole play for puppet_install_system_config_modules | 23:27 |
| pabelanger | as ansible-puppet-role pushed both system-config and puppet modules | 23:27 |
| clarkb | pabelanger: yes | 23:27 |
| clarkb | the ansible run got really slow... hasn't logged anything new in about 10 minutes but ps shows it talking ssh to one node at a time, weird | 23:28 |
| clarkb | hrm now its stuck on the new mirror node maybe | 23:28 |
| clarkb | oh puppet is running and its slow because afs compile? | 23:29 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Don't git clone system-config in puppet-install https://review.openstack.org/597308 | 23:29 |
| * clarkb practices a patience | 23:29 | |
| pabelanger | clarkb: yah, kernel compiling for afs | 23:29 |
| pabelanger | takes 10mins last time I did a mirror | 23:29 |
| clarkb | oh and the become there is likely what made zk unhappy | 23:29 |
| clarkb | so I think this may fix all the things we noticed :) | 23:29 |
| clarkb | pabelanger: +2 but ianw and corvus should double check | 23:30 |
| notmyname | pabelanger: follow-up from earlier, the swiftdsvm-functional-ipv6 test failed for etcd3 in ovh but a recheck ran it in rax where it passed. https://review.openstack.org/#/c/591497/ | 23:31 |
| fungi | we have global ipv6 working in rackspace but not in ovh, i wonder if that's related | 23:32 |
| clarkb | puppet ran on new sjc1 mirror, but afs doesn't seem to be working | 23:33 |
| clarkb | possibly need to reboot or modprobe/insmod? | 23:33 |
| pabelanger | server should have been rebooted | 23:33 |
| notmyname | fungi: yeah, it's not something that *should* impact swift. well, that is swift doesn't use etcd3 so the devstack provisioning of it is useless to us. the ipv6 test is good otherwise, but that's why I'm not too bent out of shape with this particular issue. just wanted to pass it up the chain | 23:34 |
| clarkb | pabelanger: it is when launch node runs but puppet runs afterwards once ansible cron goes by | 23:34 |
| pabelanger | clarkb: Oh | 23:34 |
| clarkb | pabelanger: I think this is just going to be a new gotcha with the transition off puppet | 23:34 |
| pabelanger | hmm | 23:34 |
| clarkb | I'm going to reboot the server manually | 23:34 |
| pabelanger | clarkb: check what we do for wheel build servers again | 23:34 |
| pabelanger | we don't reboot them and AFS works | 23:34 |
| clarkb | we probably have to modprobe the module. I checked lsmod and it didn't seem to be there | 23:35 |
| clarkb | but a reboot seems fine too | 23:35 |
| ianw | clarkb / pabelanger: yeah, hsouldn't need a reboot for afs modules. in fact, it's tested with the integration jobs where we build and install then check the mounts | 23:36 |
| fungi | yeah, old launch-node.py rebooted after initial puppet run | 23:36 |
| *** owalsh_ has joined #openstack-infra | 23:36 | |
| clarkb | ianw: pabelanger hrm well a reboot fixed it | 23:36 |
| fungi | which is likely why we missed the need to explicitly modprobe | 23:36 |
| clarkb | http://38.108.68.37/ is up and running now | 23:36 |
| clarkb | I'm going to update dns now | 23:36 |
| clarkb | then we can merge mnaser's change to put sjc1 into nodepool | 23:36 |
| *** owalsh has quit IRC | 23:37 | |
| clarkb | ok new thing discovered I think rackdns client is python2 only | 23:38 |
| clarkb | (we can install python2 on bionic, maybe we should consider it afterall?) | 23:38 |
| clarkb | I'll use web ui for now since its a single record | 23:38 |
| ianw | weshay: cr repo should be active now | 23:39 |
| mnaser | woo, so close :D | 23:40 |
| ianw | clarkb: it definitely is, this was something i mentioned in the letsencrypt spec i think | 23:40 |
| clarkb | http://mirror01.sjc1.vexxhost.openstack.org/ works now | 23:42 |
| clarkb | working on the cname | 23:42 |
| clarkb | http://mirror.sjc1.vexxhost.openstack.org/ resolves and works for me now too. Alright now to find mnasers change and get that in | 23:44 |
| clarkb | https://review.openstack.org/#/c/596830/2 if anyone else wants to take a look | 23:46 |
| clarkb | ianw: on https://review.openstack.org/#/c/597308/ are you waitning on ci before approving? | 23:47 |
| * mnaser holds breath | 23:47 | |
| mnaser | wonder if there's anyone we can poke to get the second +A | 23:48 |
| *** stakeda has joined #openstack-infra | 23:48 | |
| *** eernst has joined #openstack-infra | 23:48 | |
| ianw | clarkb: yeah but don't expect it will trigger anything | 23:48 |
| mnaser | ou there it goes | 23:50 |
| mnaser | we need a puppet run after project-config merges, right? | 23:50 |
| *** signed8bit is now known as signed8bit_Zzz | 23:51 | |
| *** sthussey has quit IRC | 23:52 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Don't manage puppet.conf during puppet-install https://review.openstack.org/597325 | 23:52 |
| clarkb | mnaser: yes | 23:52 |
| clarkb | ianw: pabelanger ^ is another followup on puppet-install | 23:52 |
| clarkb | I'm double checking if we can remove the disable puppet server/agent role now too | 23:53 |
| pabelanger | +2 | 23:53 |
| *** threestrands has joined #openstack-infra | 23:54 | |
| *** dingyichen has joined #openstack-infra | 23:55 | |
| clarkb | ianw: pabelanger Ithink we want to keep the disable puppet agent role as it actually installs the /etc/default/puppet file if necessary | 23:55 |
| clarkb | so its a superset of what we do in install-puppet | 23:55 |
| *** dingyichen has quit IRC | 23:56 | |
| clarkb | pabelanger: current ansible run is running with the profilinglogging data | 23:57 |
| ianw | should we be pulling ansible-role-puppet into our system-config testing? | 23:57 |
| clarkb | ianw: we sort of already do it with the apply tests iirc | 23:57 |
| clarkb | which is somewhat disconnected from the base.yaml testing though | 23:58 |
| clarkb | (so its probably not majorly urgent as it is tested, but wouldn't hurt to have it more copled with how we run it in production) | 23:58 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!