Wednesday, 2018-03-14

*** dingyichen has joined #openstack-infra00:06
*** bobh has joined #openstack-infra00:07
*** HeOS has quit IRC00:08
*** asilenkov has quit IRC00:11
ianwclarkb: so what's the general gist with openstack-infra/gerrit ... upstream imported on upstream/stable-* and then branched to openstack/x.y.z ?00:11
*** asilenkov has joined #openstack-infra00:11
ianwi'm just wondering how to move the jobs to native.  i guess put them on the openstack/ branch00:12
*** jamesmcarthur has quit IRC00:12
*** germs has quit IRC00:15
*** salv-orlando has joined #openstack-infra00:15
*** germs has joined #openstack-infra00:17
*** jamesmcarthur has joined #openstack-infra00:17
*** gouthamr has joined #openstack-infra00:19
clarkbyup and ya I'd  put them on the openstack branch00:20
*** gyee has quit IRC00:20
*** claudiub has quit IRC00:20
*** salv-orlando has quit IRC00:21
*** bobh has quit IRC00:22
*** jamesmcarthur has quit IRC00:31
*** germs has quit IRC00:32
*** germs has joined #openstack-infra00:42
*** germs has quit IRC00:42
*** germs has joined #openstack-infra00:42
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: zk: use kazoo retry facilities  https://review.openstack.org/53553700:45
*** eernst has joined #openstack-infra00:47
*** germs has quit IRC00:48
*** germs has joined #openstack-infra00:49
*** yamahata_ has quit IRC00:49
*** yamahata_ has joined #openstack-infra00:50
*** olaph has joined #openstack-infra00:50
*** olaph1 has quit IRC00:53
*** germs has quit IRC00:53
*** odyssey4me has quit IRC00:53
*** odyssey4me has joined #openstack-infra00:53
*** wolverineav has quit IRC00:56
*** wolverineav has joined #openstack-infra00:57
*** jbadiapa_ has quit IRC01:00
*** wolverineav has quit IRC01:02
*** r-daneel has quit IRC01:02
*** signed8bit_Zzz is now known as signed8bit01:03
*** aeng has quit IRC01:06
*** pahuang has quit IRC01:07
*** verdurin has quit IRC01:11
*** hongbin has joined #openstack-infra01:12
*** bobh has joined #openstack-infra01:13
*** eernst has quit IRC01:14
*** dizquierdo has quit IRC01:15
*** jbadiapa_ has joined #openstack-infra01:16
*** salv-orlando has joined #openstack-infra01:17
*** liujiong has joined #openstack-infra01:18
*** harlowja has quit IRC01:20
*** ihrachys has quit IRC01:20
*** olaph has quit IRC01:20
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: mqtt: add basic reporter  https://review.openstack.org/53554301:21
*** salv-orlando has quit IRC01:21
*** olaph has joined #openstack-infra01:22
*** pahuang has joined #openstack-infra01:24
*** olaph1 has joined #openstack-infra01:25
*** edmondsw has joined #openstack-infra01:25
*** olaph has quit IRC01:26
*** gongysh has quit IRC01:27
*** rossella_s has quit IRC01:27
*** edmondsw has quit IRC01:29
*** rossella_s has joined #openstack-infra01:30
*** rosmaita has quit IRC01:33
*** rossella_s has quit IRC01:35
*** gongysh has joined #openstack-infra01:39
*** rossella_s has joined #openstack-infra01:39
*** jamesmcarthur has joined #openstack-infra01:41
openstackgerritIan Wienand proposed openstack-infra/project-config master: Remove gerrit legacy jobs  https://review.openstack.org/55272301:42
*** bobh has quit IRC01:42
*** aeng has joined #openstack-infra01:43
*** jamesmcarthur has quit IRC01:46
fungiclarkb: ianw: looks like neutron-grenade jobs failed on the test change, something about...01:46
fungi/opt/stack/new/devstack/inc/python: line 561: /opt/stack/new/requirements/.venv/bin/edit-constraints: No such file or directory01:47
fungihappens while upgrading keystone01:47
*** rossella_s has quit IRC01:47
*** liujiong has quit IRC01:48
*** liujiong has joined #openstack-infra01:49
*** rossella_s has joined #openstack-infra01:49
fungidoesn't look related to new pip, though i see signs in the grenade log that there are going to be additional places we need to add the escape hatch option01:50
fungianyway, i take this as a sign of success and will press forward initiating a pr against pip tomorrow01:50
ianwfungi: ok ... i guess just pushing the warning out isn't going to fly this time01:51
*** liujiong has quit IRC01:51
ianwif a new flag is what it takes ... at least that's an auditable point01:51
fungii think a --sorry-yes-i-know option is likely to be the most agreeable solution01:52
openstackgerritIan Wienand proposed openstack-infra/openstack-zuul-jobs master: Remove gerrit legacy jobs  https://review.openstack.org/55272401:52
fungibut all bets are off until they start to review my python terribleness01:53
ianwcool, thanks will be watching01:53
*** rlandy|afk is now known as rlandy01:53
ianwpabelanger: hmm, an emjoi release name ... that's going to be up there with when Fedora chose "Schrödinger" for one of the releases01:54
fungiyou didn't know whether or not it was a live release until you opened the box?01:56
ianwmore that you're the cat -- you don't know if you're unicode safe until it either explodes or not :)01:58
fungioof02:00
Jeffrey4linfra-root, could anyone empty the files in http://tarballs.openstack.org/kolla/images/ and leave a README.md file with the content like this http://paste.openstack.org/show/700789/02:01
ianwJeffrey4l: sure ... do you want them archived somewhere, or just gone?02:02
Jeffrey4lianw, just remove them. we moved all the kolla image to hub.docker.com now. the tar files are out of update and useless.02:03
Jeffrey4lit is updated. thanks ianw02:05
*** bobh has joined #openstack-infra02:05
*** rlandy has quit IRC02:06
*** bobh has quit IRC02:07
*** bobh has joined #openstack-infra02:07
*** aeng has quit IRC02:09
*** bobh has quit IRC02:12
*** jamesmcarthur has joined #openstack-infra02:13
*** myoung|snow|bbl is now known as myoung|rover02:15
*** myoung|rover is now known as myoung|afk02:17
*** germs has joined #openstack-infra02:20
*** germs has quit IRC02:20
*** germs has joined #openstack-infra02:20
*** aeng has joined #openstack-infra02:22
*** r-daneel has joined #openstack-infra02:26
*** pcichy has quit IRC02:28
*** aeng has quit IRC02:29
*** yamahata_ has quit IRC02:35
*** eernst has joined #openstack-infra02:36
*** mriedem has quit IRC02:37
*** signed8bit is now known as signed8bit_Zzz02:38
*** hongbin_ has joined #openstack-infra02:42
*** hongbin has quit IRC02:43
*** germs has quit IRC02:43
*** germs has joined #openstack-infra02:43
*** germs has quit IRC02:43
*** germs has joined #openstack-infra02:43
*** psachin has joined #openstack-infra02:47
*** germs has quit IRC02:48
*** psachin has quit IRC02:59
*** psachin has joined #openstack-infra03:02
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: WIP: Handle ZK session loss during node launch  https://review.openstack.org/55253803:04
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: zk: use kazoo retry facilities  https://review.openstack.org/53553703:04
*** auristor has quit IRC03:04
*** psachin has quit IRC03:05
*** psachin has joined #openstack-infra03:08
*** olaph has joined #openstack-infra03:08
*** olaph1 has quit IRC03:09
*** auristor has joined #openstack-infra03:11
*** shu-mutou has joined #openstack-infra03:13
*** edmondsw has joined #openstack-infra03:14
*** vivsoni has quit IRC03:15
*** gongysh has quit IRC03:17
*** psachin has quit IRC03:17
*** jamesmcarthur has quit IRC03:17
*** edmondsw has quit IRC03:18
*** salv-orlando has joined #openstack-infra03:18
*** salv-orlando has quit IRC03:23
*** nicolasbock has quit IRC03:26
*** signed8bit_Zzz is now known as signed8bit03:26
*** vivsoni has joined #openstack-infra03:27
*** esberglu has quit IRC03:32
*** dhajare has joined #openstack-infra03:32
*** psachin has joined #openstack-infra03:35
*** cshastri has joined #openstack-infra03:36
*** aeng has joined #openstack-infra03:37
*** olaph has quit IRC03:38
*** olaph has joined #openstack-infra03:39
*** vivsoni has quit IRC03:45
*** aeng has quit IRC03:47
*** signed8bit is now known as signed8bit_Zzz03:48
*** ykarel|afk has joined #openstack-infra03:51
*** yamahata_ has joined #openstack-infra03:51
*** david-lyle has joined #openstack-infra03:55
*** vivsoni has joined #openstack-infra03:58
*** pahuang has quit IRC03:59
*** gongysh has joined #openstack-infra04:00
*** hongbin_ has quit IRC04:02
*** janki has joined #openstack-infra04:02
*** esberglu has joined #openstack-infra04:05
*** signed8bit_Zzz is now known as signed8bit04:05
*** udesale has joined #openstack-infra04:05
*** dhill__ has quit IRC04:05
*** namnh has joined #openstack-infra04:09
*** esberglu has quit IRC04:09
*** harlowja has joined #openstack-infra04:12
*** ianychoi_ has joined #openstack-infra04:12
*** pahuang has joined #openstack-infra04:13
*** aeng has joined #openstack-infra04:13
openstackgerritKazunori Shinohara proposed openstack-infra/project-config master: Add jobs of xstatic-** for heat-dashboard  https://review.openstack.org/55273504:14
*** ianychoi has quit IRC04:15
*** wolverineav has joined #openstack-infra04:17
*** dave-mccowan has quit IRC04:17
*** salv-orlando has joined #openstack-infra04:19
*** signed8bit is now known as signed8bit_Zzz04:21
*** wolverineav has quit IRC04:21
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: WIP: Handle ZK session loss during node launch  https://review.openstack.org/55253804:21
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Refactor run_handler to be generic  https://review.openstack.org/53555404:21
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Refactor NodeLauncher to be generic  https://review.openstack.org/53555504:21
openstackgerritKazunori Shinohara proposed openstack-infra/project-config master: Add jobs of xstatic-** for heat-dashboard  https://review.openstack.org/55273504:22
*** links has joined #openstack-infra04:22
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: zk: use kazoo retry facilities  https://review.openstack.org/53553704:22
*** salv-orlando has quit IRC04:24
*** signed8bit_Zzz is now known as signed8bit04:35
*** eernst has quit IRC04:39
*** harlowja has quit IRC04:40
*** mrphs has joined #openstack-infra04:44
*** signed8bit has quit IRC04:46
*** andreas_s has joined #openstack-infra04:48
*** mrphs has quit IRC04:50
*** ramishra has quit IRC04:51
*** andreas_s has quit IRC04:52
*** ramishra has joined #openstack-infra04:58
*** pgadiya has joined #openstack-infra04:59
*** esberglu has joined #openstack-infra04:59
*** alexchadin has joined #openstack-infra05:03
*** esberglu has quit IRC05:03
*** shu-mutou is now known as shu-mutow05:16
*** salv-orlando has joined #openstack-infra05:20
*** salv-orlando has quit IRC05:24
*** esberglu has joined #openstack-infra05:34
*** esberglu has quit IRC05:38
*** dhajare has quit IRC05:38
*** dhajare has joined #openstack-infra05:39
*** aeng has quit IRC05:45
*** ykarel|afk is now known as ykarel05:47
*** claudiub has joined #openstack-infra05:48
*** dhajare_ has joined #openstack-infra05:50
*** ramishra has quit IRC05:51
*** dhajare has quit IRC05:51
openstackgerritMerged openstack-infra/zuul master: Add zuul-tox-remote to gate pipeline  https://review.openstack.org/55269205:53
*** dhajare_ has quit IRC05:57
*** ramishra has joined #openstack-infra05:58
*** aviau has quit IRC06:04
*** e0ne has joined #openstack-infra06:04
*** aviau has joined #openstack-infra06:06
*** kiennt26__ has quit IRC06:14
*** kiennt26 has joined #openstack-infra06:14
*** pcichy has joined #openstack-infra06:21
*** salv-orlando has joined #openstack-infra06:21
*** salv-orlando has quit IRC06:25
*** esberglu has joined #openstack-infra06:28
*** alexchadin has quit IRC06:28
*** dbecker has quit IRC06:28
*** alexchadin has joined #openstack-infra06:29
*** esberglu has quit IRC06:32
*** jbadiapa_ has quit IRC06:33
*** salv-orlando has joined #openstack-infra06:34
openstackgerritSimon Westphahl proposed openstack-infra/zuul master: Fix zuul_json callback plugin problem with loops  https://review.openstack.org/55279906:37
*** rossella_s has quit IRC06:38
*** rossella_s has joined #openstack-infra06:38
*** dhajare_ has joined #openstack-infra06:39
*** dims has quit IRC06:41
*** dims has joined #openstack-infra06:43
*** ianychoi__ has joined #openstack-infra06:43
*** ianychoi_ has quit IRC06:46
*** jbadiapa_ has joined #openstack-infra06:48
*** dims has quit IRC06:48
*** dims has joined #openstack-infra06:49
*** edmondsw has joined #openstack-infra06:50
*** ianychoi__ is now known as ianychoi06:51
*** pcichy has quit IRC06:52
*** edmondsw has quit IRC06:54
*** tojuvone has joined #openstack-infra06:56
openstackgerritMerged openstack-infra/project-config master: Add nodepool-linaro dashboard to grafana.o.o  https://review.openstack.org/55270506:59
*** pcaruana has joined #openstack-infra07:04
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul master: Use re2 for change_matcher  https://review.openstack.org/53638907:09
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul master: Add irrelevant-branches negative matcher  https://review.openstack.org/55280907:09
*** andreas_s has joined #openstack-infra07:14
*** alexchadin has quit IRC07:16
*** alexchadin has joined #openstack-infra07:17
*** salv-orlando has quit IRC07:28
*** andreas_s has quit IRC07:31
*** danpawlik has joined #openstack-infra07:32
*** andreas_s has joined #openstack-infra07:32
*** salv-orlando has joined #openstack-infra07:34
*** armaan has quit IRC07:36
*** armaan has joined #openstack-infra07:36
*** alexchadin has quit IRC07:39
Jeffrey4linfra-root, how i can check whether post-timeout=3600 parameter works here? https://review.openstack.org/55245007:41
Jeffrey4lfrom the logs info, seems it doesn't work http://logs.openstack.org/50/552450/1/check/kolla-build-centos-binary/929d5c6/zuul-info/inventory.yaml07:43
*** hashar has joined #openstack-infra07:43
*** alexchadin has joined #openstack-infra07:47
*** rossella_s has quit IRC07:51
*** rossella_s has joined #openstack-infra07:53
*** alexchadin has quit IRC07:57
*** alexchadin has joined #openstack-infra07:58
*** rossella_s has quit IRC08:01
*** xarses has quit IRC08:02
*** shardy has joined #openstack-infra08:03
*** dbecker has joined #openstack-infra08:04
*** rossella_s has joined #openstack-infra08:05
*** chrisyang_0660 has joined #openstack-infra08:09
*** dsariel has quit IRC08:11
*** tesseract has joined #openstack-infra08:14
*** salv-orlando has quit IRC08:17
*** florianf has joined #openstack-infra08:21
*** yamahata_ has quit IRC08:24
*** jpena|off is now known as jpena08:25
*** electrofelix has joined #openstack-infra08:33
*** mdbooth has quit IRC08:33
*** ari[m] has quit IRC08:33
*** dingyichen has quit IRC08:34
*** amoralej|off is now known as amoralej08:36
*** e0ne has quit IRC08:37
*** edmondsw has joined #openstack-infra08:38
*** edmondsw has quit IRC08:43
*** kjackal has joined #openstack-infra08:54
*** ralonsoh has joined #openstack-infra08:54
*** kiennt26 has quit IRC08:55
*** rossella_s has quit IRC08:55
*** rossella_s has joined #openstack-infra08:57
*** panda|off is now known as panda09:01
*** jpich has joined #openstack-infra09:04
*** salv-orlando has joined #openstack-infra09:07
*** rossella_s has quit IRC09:07
*** rossella_s has joined #openstack-infra09:08
*** dbecker has quit IRC09:10
*** lucas-afk is now known as lucasagomes09:11
*** dbecker has joined #openstack-infra09:11
*** shu-mutow has quit IRC09:14
openstackgerritSorin Sbarnea proposed openstack-infra/project-config master: add tox risk check for jenkins-job-builder  https://review.openstack.org/55284309:18
*** yamamoto_ has joined #openstack-infra09:18
*** hemna_ has quit IRC09:20
*** yamamoto has quit IRC09:22
*** gongysh has quit IRC09:22
*** mgoddard_ has joined #openstack-infra09:24
*** udesale_ has joined #openstack-infra09:27
*** udesale__ has joined #openstack-infra09:29
*** udesale has quit IRC09:29
*** rossella_s has quit IRC09:29
*** rossella_s has joined #openstack-infra09:30
*** udesale_ has quit IRC09:32
*** derekh has joined #openstack-infra09:34
*** lpetrut has joined #openstack-infra09:34
*** rossella_s has quit IRC09:35
*** rossella_s has joined #openstack-infra09:36
ssbarneacorvus: hi! i am back now. if you could give me some hints on tox risk gate it would highly appreciate, here was my first attempt: https://review.openstack.org/#/c/552843/09:36
*** gfidente has joined #openstack-infra09:37
*** jcoufal has joined #openstack-infra09:38
*** efoley has joined #openstack-infra09:39
AJaegerssbarnea: please add this directly to jjb in-tree, no need to add such a specific job to project-config09:39
AJaegerssbarnea: and that is completely wrong syntax, will fail to run.09:40
ssbarneaAJaeger: i observed that but I am not sure what you mean by it, i am *very* new to zuul, this being the first job change mainly.09:40
*** tosky has joined #openstack-infra09:41
ssbarneai plan to setup my own test instance to play with it during weekend.09:41
evrardjphey folks, we got 2 roles in openstack-ansible that we can probably share with you: systemd-mount and systemd-service stuff09:43
evrardjpshould I import them as ansible-role-systemd_mount and ansible-role-systemd_service ?09:44
evrardjpclarkb: ^09:44
*** dsariel has joined #openstack-infra09:46
openstackgerritSlawek Kaplonski proposed openstack-infra/project-config master: Change names of new voting Neutron jobs  https://review.openstack.org/55285709:47
*** zoli is now known as zoli|lunch09:49
*** zoli|lunch is now known as zoli09:49
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: added location images endpoints  https://review.openstack.org/55245809:54
openstackgerritMerged openstack-infra/openstackid-resources master: added location images endpoints  https://review.openstack.org/55245809:54
*** jcoufal_ has joined #openstack-infra09:56
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: fix on images upload/update  https://review.openstack.org/55285909:59
*** jcoufal has quit IRC09:59
*** namnh has quit IRC10:00
openstackgerritMerged openstack-infra/openstackid-resources master: fix on images upload/update  https://review.openstack.org/55285910:01
*** rossella_s has quit IRC10:04
*** rossella_s has joined #openstack-infra10:05
openstackgerritJean-Philippe Evrard proposed openstack-infra/project-config master: Import systemd_mount and systemd_service to OSA  https://review.openstack.org/55286110:09
*** rcernin has quit IRC10:13
openstackgerritJean-Philippe Evrard proposed openstack-infra/project-config master: Import systemd_mount and systemd_service to OSA  https://review.openstack.org/55286110:17
*** alexchadin has quit IRC10:18
*** pbourke has joined #openstack-infra10:19
openstackgerritJean-Philippe Evrard proposed openstack-infra/project-config master: Add systemd roles base jobs  https://review.openstack.org/55286710:22
*** nicolasbock has joined #openstack-infra10:25
*** priteau has joined #openstack-infra10:26
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: fixed error on room floor update  https://review.openstack.org/55286810:26
*** edmondsw has joined #openstack-infra10:26
openstackgerritMerged openstack-infra/openstackid-resources master: fixed error on room floor update  https://review.openstack.org/55286810:27
*** rossella_s has quit IRC10:28
*** vivsoni has quit IRC10:30
*** edmondsw has quit IRC10:31
*** rossella_s has joined #openstack-infra10:32
*** esberglu has joined #openstack-infra10:41
persiaCould someone who has permissions (on the list at https://launchpad.net/~openstack-ci-core/+members#active) please enable different architectures at https://launchpad.net/~openstack-ci-core/+archive/ubuntu/openafs-amd64-hwe/+edit ?10:42
persiaThis is required to build ianw's upload of OpenAFS for ARM64 (which seems to be the root cause of wheel build testing failures)10:43
*** cshastri has quit IRC10:43
persiaI think we need arm64 and don't need i386, unless I misunderstand the current architectures for OpenStack CI.10:43
persiappc64el may also be interesting, to avoid needing to do this again if a cloud of that architecture becomes available.10:44
fricklerinfra-root: seems that list is mostly a list of infra-root members, could I get added to it, please? https://launchpad.net/~openstack-ci-core/+members10:45
fricklerpersia: I'd like to, but ^^10:45
persiaOther PPAs at https://launchpad.net/~openstack-ci-core may benefit from the same adjustment10:45
persiafrickler: Thanks, and totally understood :)10:45
*** pbourke has quit IRC10:45
*** esberglu has quit IRC10:45
*** alexchadin has joined #openstack-infra10:48
*** mgoddard_ has left #openstack-infra10:49
*** alexchadin has quit IRC10:51
*** mikal_ has joined #openstack-infra10:52
*** alexchadin has joined #openstack-infra10:53
*** mikal has quit IRC10:53
persiaOr maybe it is https://launchpad.net/~openstack-ci-core/+archive/ubuntu/openafs-arm64/+edit that needs adjustment (or maybe these should be consolidated, or something else which probably needs to wait for AU day to discuss properly).10:53
*** alexchadin has quit IRC10:55
*** HeOS has joined #openstack-infra10:57
*** esberglu has joined #openstack-infra10:58
*** esberglu has quit IRC11:03
*** alexchadin has joined #openstack-infra11:04
*** pbourke has joined #openstack-infra11:04
*** ianychoi has quit IRC11:07
tobiashJeffrey4l: post-timeout is probably just not injected as a variable (it technically doesn't need to be a var inside ansible)11:09
tobiashJeffrey4l: to test that you could just add a post playbook with a sleep longer than the timeout11:09
*** olaph1 has joined #openstack-infra11:10
*** janki has quit IRC11:10
*** olaph has quit IRC11:10
*** JI has joined #openstack-infra11:13
*** ianychoi has joined #openstack-infra11:13
*** rossella_s has quit IRC11:14
*** rossella_s has joined #openstack-infra11:17
*** tmorin has joined #openstack-infra11:18
tmorinhi AJaeger11:18
tmorinI'm seeing this error: http://logs.openstack.org/69/552769/1/gate/openstack-tox-pep8/5803386/job-output.txt.gz#_2018-03-14_11_01_42_72714011:18
tmorinI am guessing this may be related to work about https://review.openstack.org/#/c/550387/11:19
tmorin(the neutron/horizon tox_install things)11:19
*** bobh has joined #openstack-infra11:19
tmorinI also have questions on https://review.openstack.org/#/c/550387/11:20
tmorinI have to go now, but will get back to you later today...11:20
JIHi.11:22
JII have a question about Zuul v3 - Jenkins integration.11:22
JIDoes anyone know what's the current state of this feature?11:22
JIDoes it work already? Or is it something that is really going to happen soon?11:23
tobiashJI: mordred is the person to ask about this but he's afk this week11:24
*** armaan has quit IRC11:25
JItobiash: Thanks.11:25
*** dtantsur|afk is now known as dtantsur11:26
AJaegertmorin: I don't get that one - let'S hope mordred can help with it.11:27
AJaegertmorin: indeed, there's a relation, 550387 will solve it.11:28
tobiashJI: I think electrofelix and hughsaunders are also involved in this topic11:28
AJaegertmorin: mordred is travelling this week, you might need others to help - or wait ;(11:28
toskyAJaeger: is that 550387 a PoC which will be extended to the other users of tox_install.sh? (in my case, sahara-dashboard)?11:29
electrofelixJI I did some initial looking at a dedicated zuul-trigger plugin and worked out that for zuulv3 integration with jenkins, you really need to use nodepool with jenkins, v3 can manage static slaves as well11:29
*** bobh has quit IRC11:30
*** jcoufal_ is now known as jcoufal11:31
electrofelixJI: hughsaunders has some initial work done on a nodepool plugin, going to get the link11:31
electrofelixJI: might be worth joining the #zuul channel as that's where any chat on this occurs11:32
JI@electrofelix: I will. Thank you. As I understand, "hughsaunders has some initial work done" actually means that it is far from being merged. Is that correct?11:33
*** rossella_s has quit IRC11:36
electrofelixJI: https://github.com/rcbops/nodepool-plugin is the work done so far, hoping to start contributing sometime over the next 3 weeks myself (just need to find some time in upcoming sprint)11:36
*** sambetts|afk is now known as sambetts11:36
*** ldnunes has joined #openstack-infra11:36
*** rossella_s has joined #openstack-infra11:37
JIelectrofelix: I'll have a look. Thanks for help.11:39
*** janki has joined #openstack-infra11:43
*** salv-orlando has quit IRC11:44
*** udesale__ has quit IRC11:45
*** camunoz has joined #openstack-infra11:47
openstackgerritXing Zhang proposed openstack-infra/project-config master: Retire puppet-apps_site  https://review.openstack.org/55288411:48
*** dsariel has quit IRC11:50
*** armaan has joined #openstack-infra11:53
openstackgerritJoshua Hesketh proposed openstack-infra/zuul master: Decode gearman function into utf  https://review.openstack.org/55288811:54
*** gfidente is now known as gfidente|not11:58
*** e0ne has joined #openstack-infra12:01
evrardjptonyb: could you ping me when you're available?12:02
*** lucasagomes is now known as lucas-hungry12:03
*** armaan has quit IRC12:05
*** dsariel has joined #openstack-infra12:05
*** armaan has joined #openstack-infra12:06
*** trown|outtypewww is now known as trown12:08
openstackgerritJoshua Hesketh proposed openstack-infra/zuul master: Decode gearman function into utf  https://review.openstack.org/55288812:08
*** trown is now known as trown|ruck12:08
*** kjackal has quit IRC12:09
*** dprince has joined #openstack-infra12:10
*** armaan has quit IRC12:10
openstackgerritJoshua Hesketh proposed openstack-infra/zuul master: Decode gearman function into utf  https://review.openstack.org/55288812:11
*** zhongjun has quit IRC12:14
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: [WIP] Zuul Web: Add 'projects' endpoint  https://review.openstack.org/55289112:16
*** yamamoto_ has quit IRC12:17
*** yamamoto has joined #openstack-infra12:18
*** rossella_s has quit IRC12:18
*** rossella_s has joined #openstack-infra12:21
*** armaan has joined #openstack-infra12:24
*** armaan has quit IRC12:24
*** edmondsw has joined #openstack-infra12:24
*** bobh has joined #openstack-infra12:27
*** jpena is now known as jpena|lunch12:27
*** armaan_ has joined #openstack-infra12:29
*** kjackal has joined #openstack-infra12:30
*** dave-mccowan has joined #openstack-infra12:32
*** rlandy has joined #openstack-infra12:32
*** rossella_s has quit IRC12:34
*** rossella_s has joined #openstack-infra12:35
*** bobh has quit IRC12:36
*** bobh has joined #openstack-infra12:36
AJaegertosky: Yes!12:37
AJaegertosky: https://review.openstack.org/#/q/topic:tox-siblings+(status:open+OR+status:merged)12:38
AJaegertosky: so, not a POC - it's working ;)12:38
*** yamamoto has quit IRC12:38
*** claudiub has quit IRC12:39
*** dsariel has quit IRC12:39
*** yamamoto has joined #openstack-infra12:39
*** bobh has quit IRC12:41
*** derekh has quit IRC12:41
*** yamamoto has quit IRC12:44
*** salv-orlando has joined #openstack-infra12:44
openstackgerritMerged openstack-infra/project-config master: Change names of new voting Neutron jobs  https://review.openstack.org/55285712:46
*** odyssey4me has quit IRC12:47
*** odyssey4me has joined #openstack-infra12:47
*** salv-orlando has quit IRC12:49
*** rosmaita has joined #openstack-infra12:49
*** pblaho has joined #openstack-infra12:50
*** panda is now known as panda|lunch12:52
*** dsariel has joined #openstack-infra12:52
*** armaan_ has quit IRC12:53
*** armaan has joined #openstack-infra12:53
*** r-daneel has quit IRC12:53
*** zoli is now known as zoli|afk12:54
*** zoli|afk is now known as zoli12:55
*** yamamoto has joined #openstack-infra12:55
*** edmondsw_ has joined #openstack-infra12:55
*** pbourke has quit IRC12:56
*** mriedem has joined #openstack-infra12:57
*** edmondsw has quit IRC12:58
*** olaph1 is now known as olaph12:58
*** yamamoto has quit IRC12:59
*** alexchadin has quit IRC13:01
*** pblaho has quit IRC13:03
*** jamesmcarthur has joined #openstack-infra13:04
*** rossella_s has quit IRC13:05
*** chrisyang_0660 has quit IRC13:05
*** alexchadin has joined #openstack-infra13:06
*** pblaho has joined #openstack-infra13:07
*** lucas-hungry is now known as lucasagomes13:07
*** rossella_s has joined #openstack-infra13:07
toskyAJaeger: oh, thanks13:09
*** yamamoto has joined #openstack-infra13:10
*** HeOS has quit IRC13:10
*** HeOS has joined #openstack-infra13:10
*** Goneri has joined #openstack-infra13:13
*** sree has joined #openstack-infra13:13
*** yamamoto has quit IRC13:15
*** amoralej is now known as amoralej|lunch13:15
*** rossella_s has quit IRC13:15
*** rossella_s has joined #openstack-infra13:17
*** jcoufal_ has joined #openstack-infra13:20
*** rossella_s has quit IRC13:22
*** jcoufal has quit IRC13:22
fungi#status log added frickler to https://launchpad.net/~openstack-ci-core and set as administrator to enable ppa management13:22
openstackstatusfungi: finished logging13:22
*** rossella_s has joined #openstack-infra13:23
*** felipemonteiro_ has joined #openstack-infra13:23
rosmaitalooks like things are kind of calm ... could someone other than AJaeger take a look at https://review.openstack.org/#/c/550048/ ? thanks!13:24
*** yamamoto has joined #openstack-infra13:25
*** e0ne has quit IRC13:25
*** eharney has joined #openstack-infra13:25
*** dhill_ has joined #openstack-infra13:26
*** jpena|lunch is now known as jpena13:27
*** edmondsw_ is now known as edmondsw13:27
*** pgadiya has quit IRC13:28
*** jamesmca_ has joined #openstack-infra13:29
*** yamamoto has quit IRC13:29
*** xarses has joined #openstack-infra13:30
*** katkapilatova has joined #openstack-infra13:31
*** myoung|afk is now known as myoung|rover13:35
slaweqhi13:36
slaweqare there some guides/docs how to migrate job templates which uses "legacy-dsvm-base" as parent, like: https://github.com/openstack/neutron-tempest-plugin/blob/master/.zuul.yaml#L24 to use "devstack-tempest"?13:37
*** yamamoto has joined #openstack-infra13:40
AJaegerslaweq: best talk with andreaf on #openstack-qa13:40
slaweqAJaeger: ok, thx13:40
slaweqI will ask him13:40
*** derekh has joined #openstack-infra13:40
*** florianf has quit IRC13:41
*** dsariel has quit IRC13:41
*** pbourke has joined #openstack-infra13:42
*** ykarel is now known as ykarel|afk13:43
*** yamamoto has quit IRC13:44
*** zhipeng has joined #openstack-infra13:45
*** salv-orlando has joined #openstack-infra13:45
AJaegertosky: https://review.openstack.org/552895 is your change and ready for review (no urgency on that one - just because you asked ;)13:46
*** olaph has quit IRC13:46
*** rossella_s has quit IRC13:47
*** olaph has joined #openstack-infra13:48
toskyAJaeger: seen that, thanks, and you also migrated to the new PTI (almost)13:48
*** salv-orlando has quit IRC13:50
*** rossella_s has joined #openstack-infra13:51
*** felipemonteiro_ has quit IRC13:51
*** kiennt26 has joined #openstack-infra13:52
*** amoralej|lunch is now known as amoralej13:52
*** zhongjun has joined #openstack-infra13:53
*** jcoufal has joined #openstack-infra13:54
*** yamamoto has joined #openstack-infra13:54
*** yamamoto has quit IRC13:55
*** yamamoto has joined #openstack-infra13:55
*** yamamoto has quit IRC13:55
zhipenghi , anyone knows hot to reboot the meetbot ?13:56
*** dsariel has joined #openstack-infra13:56
*** jamesmca_ is now known as jamesmcarthur_13:56
*** jcoufal_ has quit IRC13:57
mnaserzhipeng: what issues are you having with meetbot?13:57
AJaegertosky: what did I miss?13:58
zhipengi cannot end the meeting13:58
zhipengmeetbot seems non-responsive13:58
mnaserzhipeng: were you the one who started it?13:58
AJaegertosky: feel free to comment if I should add anything..13:58
mnaseri think only after an hour, anyone can end it13:58
zhipengyep13:58
zhipengi also tried with my matrix irc nick13:58
zhipengno response13:59
zhipengi started like, last week...13:59
mnaserwhich channel is this zhipeng ?13:59
toskyAJaeger: shouldn't "python setup.py build_sphinx" be changed too?13:59
mnaserit may have died mid-meeting and so now its stuck but not really in a meeting13:59
zhipengopenstack-cyborg13:59
zhipengyep my guess as well13:59
*** esberglu has joined #openstack-infra14:00
AJaegertosky: le me  remove the docs environment completly - you have no docs ;)14:00
AJaegertosky: updated14:01
*** camunoz has quit IRC14:02
toskyAJaeger: oh, right - probably copied from other repositories :(14:04
*** kgiusti has joined #openstack-infra14:09
AJaegertosky: everybody starts with "I want good docs" ;)14:09
*** neiljerram has joined #openstack-infra14:10
*** bobh has joined #openstack-infra14:11
*** ykarel|afk has quit IRC14:12
*** dsariel has quit IRC14:13
*** electrofelix has quit IRC14:16
fungidocumentation-driven development process ;)14:16
*** gouthamr has quit IRC14:16
*** camunoz has joined #openstack-infra14:17
*** trown|ruck is now known as trown|brb14:18
corvusJeffrey4l: i think the post-timeout just isn't supplied as an ansible variable, so won't show up in the inventory.  do you have another reason to suspect it isn't working?14:18
*** rossella_s has quit IRC14:19
*** rossella_s has joined #openstack-infra14:19
*** trown|brb is now known as trown|ruck14:20
openstackgerritMerged openstack-infra/zuul master: Add zuul_json tests  https://review.openstack.org/55268814:26
*** rossella_s has quit IRC14:26
*** dsariel has joined #openstack-infra14:26
*** salv-orlando has joined #openstack-infra14:27
*** rossella_s has joined #openstack-infra14:30
*** psachin has quit IRC14:30
*** alexchadin has quit IRC14:32
*** fresta has quit IRC14:33
*** pblaho has quit IRC14:34
*** fresta has joined #openstack-infra14:35
*** germs has joined #openstack-infra14:35
*** germs has quit IRC14:35
*** germs has joined #openstack-infra14:35
*** fresta has quit IRC14:37
*** gfidente|not is now known as gfidente14:37
*** fresta has joined #openstack-infra14:40
*** signed8bit has joined #openstack-infra14:40
*** eernst has joined #openstack-infra14:42
*** danpawlik has quit IRC14:43
clarkbslaweq: AJaeger the docs for migrating those jobs should be in the devstack docs now14:46
*** pblaho has joined #openstack-infra14:47
slaweqclarkb: yes, frickler already gave me the link https://docs.openstack.org/devstack/latest/zuul_ci_jobs_migration.html14:47
slaweqthx14:47
*** felipemonteiro_ has joined #openstack-infra14:49
*** felipemonteiro__ has joined #openstack-infra14:51
clarkbzhipeng: mnaser after an hour anyone can end a meeting but if the bot was restarted then that implicitly ends the meeting14:53
*** dizquierdo has joined #openstack-infra14:54
clarkbsomeone with channel logs may have to provide that info though as meetbot is also our channel logger14:54
zhipengthx clarkb :)14:54
*** germs has quit IRC14:55
*** felipemonteiro_ has quit IRC14:55
*** yamamoto has joined #openstack-infra14:55
*** hongbin has joined #openstack-infra14:55
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Add support for review-dev01.o.o to site.pp  https://review.openstack.org/55291914:57
*** e0ne has joined #openstack-infra14:58
*** ykarel|afk has joined #openstack-infra14:58
*** ykarel|afk is now known as ykarel14:58
*** yamamoto has quit IRC15:02
fungiclarkb: ianw: as predicted, getting pushback on any attempt to work around pip 10 breaking... https://github.com/pypa/pip/pull/506615:03
openstackgerritMerged openstack-infra/zuul master: Fix zuul_json callback plugin problem with loops  https://review.openstack.org/55279915:04
fungithe main objection seems to be, "but if we add a way for users to make it not break, then they'll do that"15:04
*** numans_ has joined #openstack-infra15:04
*** numans has quit IRC15:05
clarkbimagine that users prefer not breaking :/15:06
fricklerfungi: you used my old account, somehow couldn't change that. correct lp account is "j-harbott"15:07
fungifrickler: oh, sorry about that. fixing now15:08
fungifrickler: okay, should be all set now15:11
*** ihrachys has joined #openstack-infra15:12
fricklerfungi: yep, thx15:12
*** germs has joined #openstack-infra15:13
fungimy pleasure!15:13
*** jamesmcarthur has quit IRC15:13
*** germs has quit IRC15:14
*** germs has joined #openstack-infra15:15
*** germs has quit IRC15:15
*** germs has joined #openstack-infra15:15
pabelangerfungi: clarkb: yah, reading https://github.com/pypa/pip/issues/4805 again, it does seem a revert is still on the table. Especially if https://github.com/pypa/pip/pull/5066 needs to be written not to break end-users15:15
clarkbfungi: any idea if distros have it on their radar to update packaging?15:15
*** panda|lunch is now known as panda15:16
*** dhajare_ has quit IRC15:16
fungiclarkb: absolutely no clue. it's likely something that has to be fixed on a package-by-package basis which, for a lot of distros, means communicating with lots of different package maintainers to get them to update their packaging scripts15:17
fungipabelanger: all a revert buys us is that we get to go through this _again_ for pip 1115:17
pabelangerfungi: agree15:17
fungiunfortunately, it seems like they wouldn't want to support an escape hatch cli option indefinitely either15:17
fungiso we likely get to go through it again at pip 11 (or pip 12, or...) either way15:18
*** links has quit IRC15:18
clarkbwe (openstack) should probably start communicating this to the dostros we care about since it doesnt appear upstream is doing that15:19
clarkbzigo and jamespage and dirk and dmsimard can probably get us at least moving in the right direction?15:20
dmsimardhi15:20
fungithough even if we find problem packages in, e.g., ubuntu 18.04 now it'll be another two years before we even have the option of switching to another lts release that includes the fixed packaging15:20
dmsimardI haven't caught up with backscroll yet15:20
fungisame for rhel/centos 815:20
persiaVery likely, but in some cases "lots" might be hundreds, and may include comittees, so it is unlikely to happen quickly.15:20
pabelangerclarkb: or we (openstack) could ask upstream pip to start doing it? I mean, it doesn't seem unreasonable to ask that15:21
clarkbfungi: yup but we have to get distros fixed starting somewhere15:21
pabelangerespecially since they are expect people to change / reach to breaking changes in pip1015:21
clarkbpabelanger: they explicitly seem to say in that bug that it is their users duty to do this15:21
fricklerpersia: so https://launchpad.net/~openstack-ci-core/+archive/ubuntu/openafs-arm64 does seem to have arm packages built. is that what you need or do you (also) need the older version in AMD64 HWE? anyway if it's not very urgent, I'd indeed like to defer that to ianw15:21
* persia thinks that "now" is always the best time to fix distros, as the other choice is usually "a couple years ago", which is harder to arrange.15:21
fungisure, i expect they're (slowly) fixing this themselves but as i said, lots of packages with different mostly-autonomous maintainers15:21
clarkbbut I agree they should be communicating it15:21
clarkbpersia: let me know if you solve that arrangement it would be quite useful15:22
persiafrickler: As it turns out, the baseline code depends on https://launchpad.net/~openstack-ci-core/+archive/ubuntu/openafs-amd64-hwe/+packages , and there is a special case for aarch64 to use https://launchpad.net/~openstack-ci-core/+archive/ubuntu/openafs-arm64 which I missed in my first look (as I keep failng to properly distinguish "arm64" and "amd64")15:22
persiafrickler: I still think it might be ideal to just have the OpenStack CI PPAs all support all target architectures, rather than having arch-specific workarounds in arch-specific PPAs, but that is probably a wider discussion.15:23
*** rlandy has quit IRC15:24
persiaclarkb: I have only ever managed to do it by accident in the past, so no particular hints to share.  I shall let you know if I encounter any.15:24
fungiwatching python library package uploads in debian, i see some slow churn of maintainers redoing old `setup.py install` packaging to use dh_python and such, but there's a long tail of packages which aren't getting frequent (where frequency is on the order of several years or longer) upstream releases so tend to just not get rebuilt15:24
fungii'm hoping the pressure to stamp out python 2.x packages will accelerate a lot of that15:24
*** rlandy has joined #openstack-infra15:24
clarkbdmsimard: the tl;dr is that distro packaging for python packages must stop using distutils (eg setup.py install) to make python packages to continue to be compatibile with future pip15:25
dmsimardcan someone tl;dr the changes ?15:25
dmsimardclarkb: wow, thanks15:25
fungidmsimard: the recommendation is for newer packaging to use pip or something else which will generate equivalent egg-info metadata so that pip knows where to find the package's files15:26
dmsimardI'll get in touch with our packaging senpai15:26
clarkbdmsimard: pip appears to be taking the stance that they are willing to just break people that interact with distro packages in an effort to get distros to move towards fixing this15:26
clarkbbut I'm not even sure all the distros are even aware that this is a thing15:26
dmsimardoh actually there's the RDO meeting going on right now so great opportunity to ask about it.15:27
* dmsimard does that15:27
fungidmsimard: basically, `sudo pip install --upgrade foo` will start breaking with pip 10 if you have a distro-installed python-foo built with distutils mechanisms15:27
clarkbdmsimard: https://github.com/pypa/pip/issues/4805 should capture the issue reasonably well on the pip side if people want more details15:28
* persia remembers a time when distros were excitingly all switching to distutils15:28
pabelangerfungi: clarkb: looping back to the discussion about virtualenvs for devstack, would that help avoid the breakage in pip10?15:28
clarkbpabelanger: yes15:28
clarkbpabelanger: that would solve the problem for devstack in particular15:28
clarkbon the infra side of things we'll likely have to go through each bit of config mgmt and switch to using venvs too15:29
persiaAt least for Debian, Fedora, Ubuntu, there are mechanisms in place for folk to trigger binary rebuilds without a source change (I don't know about SuSE), which can help for the case where the package depends on some distro build tooling that has been updated.  Using these often doesn't require interaction with each package maintainer.15:29
fungiunder pip 9 it issued a deprecation warning like "Uninstalling a distutils installed project ({0}) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project." but https://github.com/pypa/pip/commit/fabb739 means that anywhere you see that under pip 9.x will start unconditionally breaking under15:29
clarkbthough I think it less likely to affect us broadly15:29
fungipip 10 (real soon now)15:29
pabelangerclarkb: okay, I did talk with andreaf (I think) at PTG about that devstack change, and didn't get a no :)15:29
clarkbpabelanger: in the past major issues with needing to allow system packages in the virtualenvs to get libvirt to work sort of made using virtualenvs in devstack a waste of time. This is no longer a problem (hasn't been for a couple years)15:30
clarkbI'm not sure if pip10 would also break in ^ if we still had to allow system packages15:31
clarkbbut I don't know of any cases where we do so should be fine15:31
* jamespage reads backscroll15:31
clarkbjamespage: I think fungi and ianw have this paged in the best but there is also a pip bug above with the details. Let us know if we can help sort this out in any way.15:33
*** esberglu has quit IRC15:33
Jeffrey4lcorvus, re post-timeout, some jobs in stable branch failed. i can not sure whether i increase jobs.post-timeout works before it is merged. i will try tobiash's solution. thanks all guys.15:33
*** rkukura has joined #openstack-infra15:33
corvusJeffrey4l: it takes effect before it merges.  can you point me at an error?  and what was tobiash's solution?15:34
fungia lot of the backpressure on the distro side has been a general distaste for enabling language ecosystem package managers, as well as the fact that they're generally going to recommend if you do want to use one you should use the version of it that they explicitly package in the distro15:34
*** ihrachys_ has joined #openstack-infra15:34
*** felipemonteiro__ has quit IRC15:34
Jeffrey4lcorvus, here is the error http://lists.openstack.org/pipermail/openstack-stable-maint/2018-March/007559.html15:34
fungipatient: "it hurts when i do this"15:35
Jeffrey4ltobiash told me to add a "sleep" task to check.15:35
fungidoctor: "then stop doing it"15:35
*** felipemonteiro__ has joined #openstack-infra15:35
corvusJeffrey4l: oh, that's a periodic job.  the change won't take effect for that before it merges.  it will affect the check results though.15:35
persiafungi: Except in this case, both parties have hurt and are asked to stop.15:35
Jeffrey4lyep corvus15:36
Jeffrey4li will try to push a patch with a sleep long, then check. if it works, we will merge it.15:36
*** ihrachys has quit IRC15:37
*** dizquierdo has quit IRC15:38
*** ykarel is now known as ykarel|afk15:39
tobiashJeffrey4l: well I wouldn't call that 'solution' but just a DNM change exceeding the timeout to prove that this works, but I think that also should be tested already in the zuul tests itself ;)15:39
corvusJeffrey4l: we already have a unit test that does that, so i'm pretty sure it works :)15:39
pabelangerinfra-root: could I get a few reviews on https://review.openstack.org/552919/ adds review-dev01.o.o to site.pp15:40
pabelangerneeded to start work on bringing xenial server online15:40
Jeffrey4lcool. thanks a lot. we will try to merge the patch.15:40
corvuspabelanger: is that stanza any different than the one underneath it?15:42
*** lpetrut has quit IRC15:42
*** e0ne has quit IRC15:43
clarkbfungi: ya, I kind of get that (re patient doctor) but at the same time users use distros and if distros don't work with their tools I imagine there will be sadness. FOr us we can likely work with it as we understand the issue but many won't15:44
*** kiennt26 has quit IRC15:45
clarkbpabelanger: there is a USE_VENV option in devstack today that we can try turning on on top of the use pip10 patch15:45
*** JI has quit IRC15:46
pabelangercorvus: no, only the node changed. I was planing on doing a 2 step dance, but have no issues just updating our existing review-dev.o.o, just means puppet won't run on that server any more15:46
clarkbpabelanger: I think the potential problem we'll have is in the bootstrapping steps where we might chicken and egg installing things like virtualenv15:47
pabelangerclarkb: k, I'll look in a bit to see how that work15:47
*** esberglu has joined #openstack-infra15:48
clarkbfixup_virtualenv is a function that will most definitely need to be rewritten if still used15:48
clarkbsince it installs virtualenv from distro then installs it with pip15:48
clarkbfungi: ^15:48
dmsimardfungi: great comments on that issue/pr btw15:48
*** hemna_ has joined #openstack-infra15:48
fricklerwoa, just received a release-announce-digest with 164 entries since Feb 28. was there something stuck that got resolved?15:48
*** ihrachys__ has joined #openstack-infra15:49
dmsimardclarkb, fungi: I'm caught up and I brought it up at the RDO meeting -- we don't really have a solution right now. Realistically (even if it'd be a lot of work), we can work something out for the packages /we/ are accountable for (RDO packages) but the base distro packages would be quite problematic since they use the same tooling.15:49
clarkbpabelanger: looks like we explicitly don't call fixup_virtualenv on our test nodes so that one might be hard to test in CI15:49
clarkbpabelanger: in any cases USE_VENV is probably somewhere to start15:50
dmsimardWe'll keep that in mind, but it seems like the best case scenario is probably to pin to pip9 for the time being15:50
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Add support for review-dev01.o.o to site.pp  https://review.openstack.org/55291915:50
dmsimardThere's no public information about the rhel8 release but this might come far too late in the process to be taken into account -- i.e, I doubt given the current timelines that rhel8 would ship pip10 *and* "patched" packaging15:51
*** ihrachys_ has quit IRC15:51
corvuspabelanger: it's possible to write a regex that works for both.  but i expect we don't want to have the old one running anymore anyway.15:51
dmsimardand it's highly unlikely that rhel7/centos7 would rebuild all the python packages to take that into account15:51
clarkbdmsimard: ya I think at this point its mostly about communicating things so that progress can be made at some point15:51
clarkband not necessarily expecting any revamp of existing or soon to exist packaging15:52
dmsimardI can chime in the issue just to share our perspective but I don't have a solution for us or them15:52
clarkbfor us (openstack) we'll likely just end up using virtualenvs for everything15:52
dmsimardyeah15:54
*** yamamoto has joined #openstack-infra15:58
*** e0ne has joined #openstack-infra15:58
*** dsariel has quit IRC15:59
*** felipemonteiro_ has joined #openstack-infra16:01
pabelangercorvus: yah, I think once we power down review-dev.o.o and detach volumes, puppet stops running anyways. So I'm okay with removing the old node from site.p.p16:01
*** felipemonteiro_ has quit IRC16:02
*** zhipeng has quit IRC16:03
*** yamahata has joined #openstack-infra16:03
*** felipemonteiro_ has joined #openstack-infra16:03
fungiclarkb: ultimately, i would consider switching to virtualenvs a win anyway. i get that there are a lot of people who feel that since pip used to at least pretend to work installing directly into the system context that should be supported forever, but i always felt that mixing different package managers in a single context is a recipe for disaster since they never share any common state information16:03
*** yamamoto has quit IRC16:04
clarkbfungi: ya I think the libvirt issue was the only real reason we had to not do it and that is no longer an issue16:04
toskyfungi: summarized as "never cross the streams"16:04
fungiit's only a matter of time before one package manager upgrades a package and blows away assumed state for the other16:04
*** felipemonteiro__ has quit IRC16:04
fungiand then, yes, total protonic reversal16:04
clarkbfungi: on debuntu that largely isn't an issue due to the use of /usr/local16:05
openstackgerritJean-Philippe Evrard proposed openstack-infra/project-config master: Import systemd_mount and systemd_service to OSA  https://review.openstack.org/55286116:05
clarkbas long as you don't break your paths everything largely works16:05
fungiright16:06
clarkbthe bigger issue thee is when multiple /usr/local/ python installs fight for the same names due to multiple versions of python16:06
*** felipemonteiro__ has joined #openstack-infra16:06
fungithough you still have the problem that newer packages in /usr/local/lib may not completely shadow modules from /usr/lib they're intended to replace16:07
fungiyou may end up importing modules from different versions of the package16:07
clarkbpabelanger: corvus to be clear this is the expectation on the review-dev change that regex will work on old and new hosts.16:08
clarkbpabelanger: is that the expectation?16:08
fungie.g., if a module in the package has been renamed in newer versions and your application does an import under try/expect to attempt to maintain backwards-compatibilty16:08
openstackgerritZhao Chao proposed openstack-infra/irc-meetings master: Change Trove meeting time  https://review.openstack.org/55295016:09
openstackgerritMerged openstack-infra/zuul master: Import Zuul modules at top of files  https://review.openstack.org/54973016:09
openstackgerritMatthew Thode proposed openstack/diskimage-builder master: enable systemd profile for Gentoo  https://review.openstack.org/55295216:09
*** felipemonteiro_ has quit IRC16:09
clarkbpabelanger: also looking at review-dev memory history on cacti we might consider using a much smaller flavor for the new host?16:10
clarkbmaybe 8GB?16:10
openstackgerritJean-Philippe Evrard proposed openstack-infra/project-config master: Add systemd roles base jobs  https://review.openstack.org/55286716:11
pabelangerclarkb: yah, we can do 8GB16:11
pabelangerclarkb: no, the regex now will only work with review-dev01.o.o hostnames. review-dev.o.o will stop getting puppet updates, but think that is fine as once review-dev01.o.o is online, we'll power off other and move volumes16:12
clarkbpabelanger: by using the * it should continue to work with review-dev.o.o too16:12
pabelangeroh, k. I didn't test16:12
clarkb* will match no \d as well as more than one \d16:12
clarkb+ is one or more16:13
pabelangerack16:13
*** dprince has quit IRC16:13
clarkbI remember the other issue with USE_VENV in devstack as is right now. It doesn't test coinstallability of openstack services16:13
evrardjpOur systemd roles for network management, systemd services, and mount points management, written by cloudnull, are up for sharing. Just saying.16:13
clarkbit also has the downside of being a per service check against USE_VENV to use a service specific VENV which means plugins will all have to implement it themselves too16:13
pabelangerclarkb: I'll be using https://etherpad.openstack.org/p/jgLaT4MRuC to make notes for migrating review-dev.o.o to review-dev01.o.o16:14
clarkbfungi: pabelanger so I think we likely want a new devstack option called GLOBAL_VENV or similar that updates the pip install function and just installs everything to a common virtualenv16:14
pabelangerclarkb: yah, USE_VENV seem to setup per service virutalenvs16:14
*** r-daneel has joined #openstack-infra16:15
fungithat'll be a problem for python 3 testing while the services don't all support python 3. we'd still need separate python 3 and python 2 virtualenvs at a minimum, right?16:16
clarkbfungi: yup good point, but we could do one global venv for each python16:16
pabelangeryah16:16
clarkband slowly over time that should converge to single for python3 hopefully16:16
fricklermight also be interesting to see how grenade can deal with that change16:16
clarkbfrickler is asking the hard questions16:19
clarkbfrickler: I think it should be functionally similar to having system global installs?16:19
fricklerclarkb: sorry, bad habit ;)16:19
clarkbso hopefully nothing really changes there other than to say use this virtualenv16:19
*** felipemonteiro__ has quit IRC16:20
*** udesale has joined #openstack-infra16:20
pabelangerokay, grabbing some food16:21
fricklerhmm, might actually be that easy, yes16:21
*** panda is now known as gabriele16:23
*** olaph1 has joined #openstack-infra16:23
*** erlon has joined #openstack-infra16:23
*** olaph has quit IRC16:25
openstackgerritMerged openstack-infra/zuul master: Match github model of granting admins write  https://review.openstack.org/54985316:26
*** udesale has quit IRC16:26
openstackgerritMerged openstack-infra/zuul master: Make sure we fail because of path checks  https://review.openstack.org/55250216:26
openstackgerritMerged openstack-infra/zuul master: Decode gearman function into utf  https://review.openstack.org/55288816:26
*** rossella_s has quit IRC16:26
*** tpsilva has joined #openstack-infra16:27
*** gfidente is now known as gfidente|not16:27
*** rossella_s has joined #openstack-infra16:29
*** gfidente has joined #openstack-infra16:29
*** andreas_s has quit IRC16:29
tmorinAJaeger: I've had a look at tox-siblings...  and I have a question16:29
tmorinAJaeger:16:29
*** andreas_s has joined #openstack-infra16:30
*** ykarel|afk has quit IRC16:30
tmorinAJaeger: https://review.openstack.org/#/c/550387/ solves the issue only for the dependency to neutron, but networking-bgpvpn has other dependencies... (to networking-odl and to networking-bagpipe for instance)16:30
tmorinAJaeger: will all these dependencies be added to global requirements ?16:31
*** dprince has joined #openstack-infra16:32
*** olaph has joined #openstack-infra16:35
rm_workIs anyone else seeing stuff like http://logs.openstack.org/85/550085/4/check/openstack-tox-pep8/e5f9981/job-output.txt.gz#_2018-03-14_15_42_23_051295 suddenly?16:35
*** olaph1 has quit IRC16:36
clarkbrm_work: I think what is happening there is neutron is newly published to pypi and has been added to constraints16:37
clarkbrm_work: I believe the intent is to allow projects like yours to stop managing out of bandish neutron installs and just list it normally like any other dependency which should fix that issue16:37
odyssey4mepabelanger there may be an opportunity to have some of the ansible role you're working on (zuul, nodepool, etc) to make use of the roles we're importing in https://review.openstack.org/552861 ?16:38
rm_workso we'll need to do something? because currently literally every one of our checks are breaking16:38
clarkbrm_work: yes, list neutron as a normal test-requirement16:38
rm_workk16:39
clarkbrm_work: which will install it from https://pypi.python.org/pypi/neutron/12.0.0 using whatever version is in constraints16:39
rm_workkk16:39
clarkbthis wasn't an issue before because neutron wasn't in the constraints file so wasn't attempted to be constrained by pip (it essentially just ignored it)16:39
clarkbhrm the construction hasn't been so bad but they are digging a trench at the base of my driveway now and my monitor is shaking16:40
pabelangerodyssey4me: good to know, will keep an eye out16:40
clarkbif this gets any worse I may have to figure out relocating (which is made difficult by not having a driveway I can drive out of)16:41
odyssey4mepabelanger clarkb we're also working on a standard ansible role to construct a venv - perhaps that's of interest to use more broadly too?16:41
*** arxcruz is now known as arxcruz|off16:42
*** yamahata has quit IRC16:42
odyssey4methe intent is to have it construct a venv at run time, then package it into a tarball - it can then re-use that venv and do the needed fixup when re-using it16:42
clarkbodyssey4me: be careful doing that, virtualenvs aren't really relocatable in practice16:42
clarkbthere are tools to make it less painful though16:43
pabelangerodyssey4me: for now, I've had good success using the virtualenv flags for the pip module, seems to be working well16:43
clarkbif your distro updates their python packages existing virtualenvs tend to have to be rebuilt16:43
odyssey4meclarkb yes, there are limits for its usage - the caveat is that the same c bindings as were used to construct it are available when re-using it16:43
odyssey4methe intent of the role is just to do it - how it's used is up to the consumer and if anything special needs to be in place then that can be done in the play16:44
rm_workIIRC there was some flag too during venv creation that was supposed to help with relocatability16:46
rm_workwe did our deploys that way for a little bit (tarball of a venv) but eventually switched to container images as they tend to be more reliable16:46
clarkbrm_work: ya the relocatable flag deals with paths changing under the venv iirc16:46
pabelangerodyssey4me: take a look giftwrap: http://git.openstack.org/cgit/openstack/giftwrap I think that does something closely to what you are suggesting. But don't think it has anything related to ansible16:46
clarkbso you can move it from /foo/venv to /bar/venv?16:46
rm_workyeah16:47
*** janki has quit IRC16:47
rm_workhmm i guess `--no-site-packages` is default now too, cool16:47
rm_workit's been a while since i looked16:47
*** andreas_s has quit IRC16:48
rm_workthe `--relocatable` thing is apparently for running on an existing venv? which is news to me, lol16:48
clarkbrm_work: yup and openstack's reliable on site pckages for libvirt bindings was a big thing that made virtualenvs less useful for us in the past. But that hasn't been an issue for a while now so we can move past that and embrace the virtualenv16:48
rm_workwe may have just been wildly lucky stuff worked :P16:48
odyssey4mepabelanger yeah, giftwrap did some clever things and we have discussed some ideas learned from that which we may put into the roadmap16:48
rm_workyeah I've been virtualenv only for ... a long time16:48
rm_workdevstack still pains me with lack of venv16:48
rm_worki just spin it up in a VM and revert to a clean snapshot every single run <_<16:49
pabelangerodyssey4me: nice16:49
corvusi think it'd be okay for devstack to use *one* virtualenv16:49
*** andreas_s has joined #openstack-infra16:50
clarkbcorvus: ya in fact I think it is preferred in this case to make it easier on plugin writers and to help check coinstallability16:50
fungiwell, *one* virtualenv per major python interpreter version you're testing16:50
odyssey4mecorvus for now, perhaps - and only if you exclude things like tempest/rally/gnocchi which don't abide by the g-r process16:50
corvus(this is distinct from venv-per-service which is a choice that has everything to do with compatibility and nothing to do with installation mechanics)16:50
corvusodyssey4me: yeah, tempest already has its own venv for $legit_reasons16:50
tmorinrm_work: I've hit the same issue as yours16:50
tmorinrm_work, clarkb: I ended up proposing https://review.openstack.org/#/c/552975/16:51
clarkbtmorin: I feel like that is a step backward, we are at the beginning of the cycle we should just roll forward and add neutron to test requirements instead?16:52
rm_worktmorin: I think clarkb is right though, we just need to adjust our tox.ini to stop trying to do crazy weird installation methods for neutron that we used to do as a workaround for the lack of thing they just fixed :P16:52
tmorinrm_work, clarkb: AJaeger told me this should be handled with mordred, but that he was travelling16:52
clarkbrm_work: right16:52
rm_workso yeah, I would not recommend we move backwards16:52
rm_workthis is the time to break stuff16:52
clarkbat the very least push a change up that swaps it out for pypi neutron and see if it just works (tm)16:52
rm_workstart of cycle16:52
tmorinclarkb: I don't disagree with the principle, but the work goes beyond neutron16:53
rm_workyep that is our plan16:53
clarkbif it is that easy I would definitely fight for rolling forward16:53
*** tosky has quit IRC16:53
rm_worki'll report back16:53
tmorinclarkb: e.g. networking-bgpbpn has dependencies on neutron, but also networking-odl, and networking-bagpipe16:53
tmorinand adding these to global requirements will not be sufficient16:53
openstackgerritMerged openstack-infra/zuul master: Skip autohold if no autohold was requested  https://review.openstack.org/54730216:54
openstackgerritMerged openstack-infra/zuul master: Add queue size to tenant overview  https://review.openstack.org/54824816:54
openstackgerritMerged openstack-infra/zuul master: Don't store references to secret objects from jobs  https://review.openstack.org/54642816:54
clarkbyou won't need to add those to requirements to fix the current issue16:54
openstackgerritMerged openstack-infra/zuul master: Add test for fingergw on ipv4 / ipv6 addresses  https://review.openstack.org/55264516:54
tmorinat least for networking-bgpvpn and networking-bagpipe: they are co-developped and networking-bgpvpn typically needs  networking-bagpipe master16:54
clarkbconstraints only applies to explicitly listed package names16:54
rm_workif they aren't in g-r this issue shouldn't affect them16:54
tmorinclarkb: true16:54
*** andreas_s has quit IRC16:54
tmorinclarkb: but the whole tox-siblings intent would mean putting all these projects in requirements, right ?16:54
rm_workwe're just removing the weird tox_install.sh thing and putting it in our requirements16:55
fungialso, tox-siblings is the answer to the lockstep development issue. you list them as required projects in your jobs and they'll get equivalent branches installed from git instead of pypi16:55
tmorinclarkb: I wanted to bring this up, but I actually need to go right now ...16:55
rm_worktmorin: i would assume you could continue to install those using the old method, it'd just be `neutron` that would need to switch16:55
tmorinclarkb: will get back to the topic tomorrow16:55
*** trown|ruck is now known as trown|lunch16:55
tmorinI'll think about all that in the meantime16:55
tmorinbye16:55
rm_workhere's the change we're trying out: https://review.openstack.org/#/c/552978/16:56
pabelangerclarkb: fungi: corvus: frickler: I've started iterating on USE_VENV in devstack at https://review.openstack.org/552939/ it's already exposed a few bugs.  While that is per service, should yield some good first results, then we can see how to do the global virtualenv for all services16:56
*** witek has quit IRC16:56
rm_workpabelanger: ooh, i didn't realize that was in-progress or even semi-working, that's awesome16:56
pabelangerrm_work: yah, USE_VENV hasn't been part of devstack for a while, but I don't know the history of why it hasn't been used more. Figure enabling it will quickly learn what works / doesn't16:57
*** iyamahat has joined #openstack-infra16:57
clarkbpabelanger: the history of not using it in the gate is beacuse it doesn't check coinstallability16:58
clarkbat least that is what I seem to remember the big issue being16:58
pabelangerk16:58
*** iyamahat has quit IRC16:58
*** tmorin has quit IRC16:58
*** witek has joined #openstack-infra16:59
*** armaan has quit IRC16:59
*** yamamoto has joined #openstack-infra16:59
*** iyamahat has joined #openstack-infra16:59
fungialso we've had an unfortunate history of some services importing modules from the codebase of other services (and as mentioned earlier some of the extension/plugin projects still rely on this to varying degrees) so they need to share an installation context to work at all17:02
rm_worki mean for extensions ... yes17:03
rm_workit makes sense, they're supposed to install on top of a primary project17:03
*** jamesdenton has joined #openstack-infra17:04
rm_workthat's kinda the point :P17:04
corvusyes, coinstallability is a goal17:04
rm_workother things doing it though ... is a little sketchy17:04
corvusso regardless of other issues (plugins), we'd still want to have no more than a single venv for services17:04
*** yamamoto has quit IRC17:05
*** ralonsoh has quit IRC17:08
pabelanger++17:10
*** armaan has joined #openstack-infra17:13
*** claudiub has joined #openstack-infra17:14
*** jpich has quit IRC17:14
*** yamahata has joined #openstack-infra17:15
*** pcaruana has quit IRC17:18
*** e0ne has quit IRC17:20
fungiin the interim, we'd need two if we're testing python 3 and integrating with non-py3k-supporting services17:22
fungiinstall most things into the python3 venv, and only what won't work there into the python2 venv instead17:22
*** zoli is now known as zoli|gone17:23
fungieven when we do that in the system context today we're not testing coinstallability between those two groups of services for py3k compat jobs, since they end up with separate sets of dependencies installed under the lib search path for each major interpreter version17:24
odyssey4meis there any tooling available to do bug trend reporting for launchpad/storyboard?17:24
clarkbfungi: yup17:24
*** zoli|gone is now known as zoli17:24
clarkbodyssey4me: elastic-recheck ?17:24
clarkbodyssey4me: openstack-health has support for it built in now too17:24
odyssey4meI'm thinking about something project drivers may use to monitor project health - things like monitoring bug throughput, incoming vs outgoing, etc17:24
fungidepends on what aspects of bugs you want to trend, i guess17:24
clarkboh purely on the bug data17:25
clarkbthere is bugday17:25
fungiyeah, for open/closed rate info, i think bugday did/does that?17:25
*** Swami has joined #openstack-infra17:25
fungithough almost certainly only for lp at the moment17:25
fungisb wouldn't be hard to do what with the straightforward rest api, but would need someone to write that17:26
evrardjpwe are still on LP so we are good on that side odyssey4me  :)17:26
clarkbhttp://status.openstack.org/bugday/17:27
clarkbI guess it only trends over a day? so may need to be updated to get longer term info17:27
odyssey4melooks like bugday is a bit dated17:28
odyssey4mewell, this page at least: https://wiki.openstack.org/wiki/Documentation/BugDay17:28
fungiit hails from near the beginning of openstack17:28
fungiand probably hasn't had any commits in years17:28
*** rossella_s has quit IRC17:30
clarkbits actually ~4 days?17:30
clarkber 317:31
odyssey4meyeah, I'm thinking more along the lines of something useful over the course of an entire cycle17:31
clarkbttx has/had tools for doing that not sure if they are hosted or just run locally17:31
*** jamesmcarthur_ has quit IRC17:32
odyssey4memanaging a project has many factors... it looks like most of the tooling development has gone into managing getting new features in - bug management seems to be a bit of a second class citizen17:32
openstackgerritMerged openstack-infra/irc-meetings master: Change Trove meeting time  https://review.openstack.org/55295017:32
*** rossella_s has joined #openstack-infra17:32
odyssey4meeven feature pipeline management isn't great... the LP stuff is a bit weird, so I can understand why it's not always used by projects17:33
clarkbsdague has/had tooling too17:33
* odyssey4me makes a mental note to look back at storyboard again17:33
clarkbI think people end up writing their own and then they don't get hosted/published necessarily?17:33
odyssey4meja, and that gives me the sads17:33
odyssey4mewe all have our hacky scripty things which we're too embarrased to share because they're horrible :p17:34
clarkbI'd definitely catch up with ttx when he is awake tomorrow17:34
odyssey4methanks for the perspective :) I'll see if I can get hold of sdake and ttx to discuss what they've got - I like to think we could perhaps put together a discussion about it at the next PTG, and hopefully we can do something together to improve the situation17:35
*** jamesmcarthur has joined #openstack-infra17:36
*** gfidente has quit IRC17:36
evrardjpodyssey4me: sdague not sdake :)17:38
*** gfidente|not is now known as gfidente17:38
evrardjpodyssey4me: thanks for the work there17:39
odyssey4meoops, apologies to both :p17:39
*** jamesmcarthur has quit IRC17:40
*** jamesmcarthur has joined #openstack-infra17:42
*** hashar is now known as hasharAway17:43
*** gabriele is now known as panda17:43
*** panda is now known as panda|off17:43
*** dizquierdo has joined #openstack-infra17:44
*** imacdonn has quit IRC17:44
*** sree has quit IRC17:44
*** imacdonn has joined #openstack-infra17:44
*** sree has joined #openstack-infra17:45
*** dtantsur is now known as dtantsur|afk17:47
pabelangerfungi: corvus: mind a review on https://review.openstack.org/552919/ brings review-dev01.o.o online17:49
pabelangerin site.pp17:49
*** sree has quit IRC17:50
clarkbcorvus: did you want to weigh in on the python-jenkins core reviewer updates email thread? as fungi pointed out its not an infra project proper and I think you were involved in ressurrecting it off of lp17:50
*** bhavik1 has joined #openstack-infra17:52
*** claudiub|2 has joined #openstack-infra17:53
*** e0ne has joined #openstack-infra17:53
fungiclarkb: https://review.openstack.org/550092 "Stop embedding (outated) PTL contact info in docs" probably needs your input17:54
*** claudiub has quit IRC17:55
*** wolverineav has joined #openstack-infra17:57
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Exclude api-ref changes from py checks  https://review.openstack.org/55004817:57
*** links has joined #openstack-infra17:57
*** yamamoto has joined #openstack-infra18:01
*** jpena is now known as jpena|off18:01
*** lucasagomes is now known as lucas-afk18:02
*** wolverineav has quit IRC18:03
*** derekh has quit IRC18:04
openstackgerritMerged openstack-infra/zuul master: Unset finger client timeout after connect  https://review.openstack.org/54673518:05
*** gyee has joined #openstack-infra18:05
*** yamamoto has quit IRC18:06
*** wolverineav has joined #openstack-infra18:09
*** harlowja has joined #openstack-infra18:11
*** pblaho has quit IRC18:11
*** felipemonteiro_ has joined #openstack-infra18:12
openstackgerritMerged openstack-infra/system-config master: Add support for review-dev01.o.o to site.pp  https://review.openstack.org/55291918:14
*** wolverineav has quit IRC18:14
*** felipemonteiro__ has joined #openstack-infra18:15
jlvillalgerritbot review request: https://review.openstack.org/#/c/545469/  Some cleanup/refactoring and adding unit tests. Thanks.18:15
*** e0ne has quit IRC18:16
*** bhavik1 has quit IRC18:17
*** felipemonteiro_ has quit IRC18:19
*** myoung|rover is now known as myoung|lunch18:23
*** lpetrut has joined #openstack-infra18:28
corvuspabelanger: to your question in #zuul -- project-config doesn't have any secrets affected by the bug18:29
corvuspabelanger: i don't plan to audit all of openstack's usage, though if someone wants to, be my guest :)18:30
*** felipemonteiro__ has quit IRC18:30
corvuspabelanger: i think just an email to openstack-dev to let folks know is sufficent18:30
*** felipemonteiro__ has joined #openstack-infra18:30
*** neiljerram has quit IRC18:30
pabelangerthanks18:33
pabelangerI can't remember any place where we did a loop with a secret, but unsure of other projects18:34
tobiashwe craft a .netrc in a loop :/18:38
*** trown|lunch is now known as trown|ruck18:38
*** wolverineav has joined #openstack-infra18:43
*** shardy has quit IRC18:48
fungiooh, .netrc and rexec... that takes me back18:51
beisnerhi all: do you know the guidelines for announcing releases on the openstack-announce ML?  ie. openstack-charms releases, etc?18:54
*** e0ne has joined #openstack-infra18:54
fungibeisner: it's moderated, and there is also a separate ml for per-project release announcements18:56
beisnerfungi: is that release-announce?18:57
*** ociuhandu has joined #openstack-infra18:57
fungibeisner: yes, http://lists.openstack.org/pipermail/release-announce/ is mostly posted to by our release automation (via announce-release ci jobs for projects), but you could probably also send directly to it if you're doing your own manual releasing18:58
fungibeisner: for sending a once-per-cycle release announcement along the lines of http://lists.openstack.org/pipermail/openstack-announce/2018-February/002012.html that might be okay, but i'd ask in #openstack-release for guidance18:59
*** pbourke has quit IRC19:00
fungimight be okay to send to the openstack-announce ml i mean19:00
openstackgerritMerged openstack/boartty master: Remove archived lanes and worklist items  https://review.openstack.org/55271419:00
openstackgerritMerged openstack/boartty master: Display story access level  https://review.openstack.org/55271519:00
fungiwe try to keep it very, very, very low-traffic though19:00
*** salv-orlando has quit IRC19:00
*** salv-orl_ has joined #openstack-infra19:01
*** ihrachys has joined #openstack-infra19:02
*** yamamoto has joined #openstack-infra19:02
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Create review-dev for hieradata  https://review.openstack.org/55301519:02
*** ihrachys__ has quit IRC19:02
pabelangerfungi: clarkb: corvus: ^follow up to review-dev01.o.o patch, moves our hieradata from fqdn to group entries19:03
*** ociuhandu has quit IRC19:03
fungioh, right, probably also need to do that in the private hieradata on pmaster19:04
pabelangerYah, can make that change once we are ready19:04
*** camunoz has quit IRC19:05
*** ihrachys has quit IRC19:07
*** ihrachys has joined #openstack-infra19:07
*** yamamoto has quit IRC19:07
*** salv-orl_ has quit IRC19:08
AJaegerclarkb, tmorin, rm_work , back online and reading backscroll...19:08
*** salv-orlando has joined #openstack-infra19:08
rm_worki think it's fine for us AJaeger19:09
rm_workwe're merging fixes now that remove the old workaround stuff19:10
*** olaph1 has joined #openstack-infra19:10
*** andreas_s has joined #openstack-infra19:11
*** olaph has quit IRC19:11
AJaegerrm_work: great!19:12
AJaegerrm_work: anything you need my help with?19:12
AJaegerrm_work: I apologize for that breakage, we didn't expect that ;(19:12
rm_workah but you just commented and looks like we need to respin and wait ANOTHER hour+ T_T19:12
rm_workwe've not had working gates for over a week now19:12
rm_workbecause it's just been one thing after another19:13
AJaegerrm_work: what fun...19:13
rm_worklongest they've been working is about 2 hours between breakage19:13
AJaegerrm_work: merge what you have and let me do a followup - ok?19:13
rm_workeh19:13
openstackgerritMatthieu Huin proposed openstack-infra/zuul master: web: Add /projects, /projects/{project} endpoints  https://review.openstack.org/55289119:13
AJaegerrm_work: which channel are you hanging out? I can join...19:13
rm_workwe can just do it right, i think -- just need to make sure we get ALL of it correct and don't wait again19:13
rm_work#openstack-lbaas19:13
*** salv-orl_ has joined #openstack-infra19:18
*** myoung|lunch is now known as myoung|rover19:18
*** salv-orlando has quit IRC19:19
*** camunoz has joined #openstack-infra19:20
*** esberglu has quit IRC19:20
*** tesseract has quit IRC19:20
*** signed8bit is now known as signed8bit_Zzz19:21
*** esberglu has joined #openstack-infra19:23
*** andreas_s has quit IRC19:25
*** signed8bit_Zzz is now known as signed8bit19:30
clarkbfungi: I'll take a look. Just catching back up again after moving some tree rounds around and deciding that I should just have Chris Aedo do that for me >_>19:32
*** links has quit IRC19:34
amotokiis there any change in zuul? http://logs.openstack.org/08/552808/1/gate/openstack-tox-pep8/d50b4e3/job-output.txt.gz#_2018-03-14_18_27_30_84078719:34
clarkbamotoki: that is a change to openstack/requirements to add neutron and horizon to the globale requirements and upper constraints file19:35
clarkbfungi: do you think it likely that we would publish more contact info on the governance pages? I do worry that that may end up creating a communications barrier if we don't19:35
amotokiclarkb: I see. can we consume horizon latest commit?19:35
clarkbamotoki: if you use tox-siblings (which I think is default for all tox jobs) then yes19:35
*** e0ne has quit IRC19:36
clarkbamotoki: you just have to list the project as a required-project in the job then list horizon or neutron as a normal dependency (in test-requirements probably?)19:36
amotokiclarkb: thanks for the info. perhaps all neutron related projects and horizon plugins needs to be updated.19:37
AJaegeramotoki: just writing an email about it19:37
amotokiAJaeger: that helps all of us.19:38
AJaegerclarkb: https://etherpad.openstack.org/p/zuulv3-remove-zuul-cloner - and see https://review.openstack.org/#/q/topic:tox-siblings19:38
amotokihopefully it would be nicer before we have the change.19:38
openstackgerritMerged openstack-infra/zuul master: configloader: check for unknown shadow projects  https://review.openstack.org/55076319:38
fungiclarkb: i think it's a good opportunity for pushing to have the project pages on the governance.o.o site to start displaying the e-mail addresses we already have in the structured data we're building them from19:40
AJaegeramotoki, clarkb http://lists.openstack.org/pipermail/openstack-dev/2018-March/128310.html19:40
openstackgerritMerged openstack-infra/system-config master: Create review-dev for hieradata  https://review.openstack.org/55301519:41
clarkbAJaeger: thanks19:41
amotokiAJaeger: is it okay to have horizon/neutron queens release as g-r even if the code totally depends on Rocky one?19:41
*** hemna_ has quit IRC19:43
openstackgerritTobias Rydberg proposed openstack-infra/irc-meetings master: Change of day and channel for bi-weekly meetings for Public Cloud WG.  https://review.openstack.org/55302219:44
*** jcoufal has quit IRC19:44
*** amoralej is now known as amoralej|off19:45
AJaegeramotoki: yes! We install those from git thanks to tox-siblings19:45
fungi(assuming they're included in required-projects for the job that is)19:46
amotokiAJaeger: but this change makes it difficult to test a patch which depends on the master locally19:46
amotokiAJaeger: is there any trick?19:46
clarkbthis is the big problem with tox-siblings, You essentially have to run tox the same way I think. Might be worth a rethink on that? The method is run tox to install the venv only, then do manual install of "sibling" to the venv then rerun tox to execute the tests19:47
AJaegeramotoki: you can always install in your environment. and mordred wanted to write a tool for that19:47
AJaegeramotoki: as clarkb said19:47
fungii think the solution projects who've already had to deal with this problem have been defaulting to is to provide a convenience script to local developers for cloning and checking out the right repositories and installing them into the virtualenv and editing them out of the constraints and requirements files19:47
*** e0ne has joined #openstack-infra19:48
fungibut yeah, ideally we'll have some tool for locally running job playbooks19:48
fungiwhich is mostly blocked on being able to grab the right roles from other repos to assemble that19:48
fungiif memory serves19:48
AJaegerfungi: the change to tox-siblings removes the tools/tox_install.sh script that you mention as local tool19:48
fungiAJaeger: ahh, as opposed to simply not running tools/tox_install.sh in ci jobs19:49
clarkbpabelanger: corvus fungi dmsimard do you know off the top of your head if we run ansible on the zuul executors with python3 or with python2?19:49
*** salv-orl_ has quit IRC19:50
amotokiso, isn't it the right solution to drop tox_install.sh now?  it seems better tox_install.sh just calls pip install if it is called in CI.19:50
*** salv-orlando has joined #openstack-infra19:50
pabelangerclarkb: python319:50
AJaegerwhy is mordred not around when we finally merge his requriements change ;)19:50
clarkbpabelanger: thanks19:50
pabelangerhowever on the nodepool side, it is using python219:50
AJaegerthat breakage wasn't expected ;(19:51
pabelangerwe hardcode that in ansible.cfg IIRC19:51
clarkbpabelanger: nodepool side? we don't run ansible from nodepool do we?19:51
pabelangerclarkb: no, but their is python that runs on VM side too, which we force python219:51
AJaegeramotoki: if you have some working code, please tell - and we can always readd19:51
pabelangerclarkb: I want to say because of python3 bugs / crashes19:51
clarkbpabelanger: oh gotcha, is this just in the image builds? or something that happens post boot?19:51
pabelangerclarkb: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/executor/server.py#n121419:52
*** jamesmcarthur has quit IRC19:52
*** Marx314 has joined #openstack-infra19:52
*** Marx314 has quit IRC19:52
clarkbwait so we run the ansible-playbook command on executor as python3 which spawns python2 ansible processes on the test nodes?19:53
pabelangerwe should see if that is still an issue on bionic nodes19:53
*** Marx314 has joined #openstack-infra19:53
pabelangerclarkb: yes19:53
clarkbthats a bit mind bending19:53
amotokiAJaeger: i just noticed it. do you mean some script by "working code"?19:53
AJaegeramotoki: I thought you talked about tox_install.sh19:53
*** sambetts is now known as sambetts|afk19:53
amotokiAJaeger: yes.19:53
AJaegeramotoki: sorry, trying to push out changes quickly...19:53
*** jamesmcarthur has joined #openstack-infra19:53
amotokiAJaeger: previously tox-install.sh checks zuul-cloner to determines whether CI env or local env. can we do similar thing?19:54
AJaegeramotoki: and I have one idea - remove neutron/horizon from constrainsts for now19:54
*** salv-orlando has quit IRC19:54
AJaegeramotoki: maybe - didn't dig into that. mordred wanted to work on some other tool so didn't dig into that ;(19:54
amotokiAJaeger: the main reason I raise this is mox-removal effort in horizon plugins completely depends on horizon master. they cannot be mox free with horizon queens.19:55
corvusclarkb: think of ansible modules running in python as an implementation detail... (recall that they don't actually have to be implemented in python, they can even be bash)19:55
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Fix safe path check for directories containing symlinks  https://review.openstack.org/55302919:55
clarkbcorvus: right its like a remote fork()19:56
corvusya19:56
clarkbstill mind bending :)19:56
corvusclarkb: note that because of delegation, we can end up running modules on the executor under py3 and running the same modules under py2 on a remote node19:56
corvusso in general, ansible *itself* is py3 safe, modules are not19:57
pabelangeryah, this was one of the concerns about python3 only nodes for bionic, right now zuul would need to remove the hardcoded python319:57
pabelangererr19:57
pabelangerpython2*19:57
tobiashhopefully that gets better with 2.419:57
AJaegeramotoki: and in CI it will work correctly.19:57
AJaegeramotoki: but I understand your concerns for local development19:57
clarkbpabelanger: bionic has python2 though19:58
pabelangerclarkb: yah, I think tonyb was asking about creating python3 only bionic, then allow jobs to install python2.19:59
pabelangerbut our DIB elements don't support that today, they do both20:00
clarkbpabelanger: if we end up pushing everything into venvs anyways that may be a better answer to the ignore certain python installs by default case20:00
pabelangeryah20:00
*** yamamoto has joined #openstack-infra20:03
amotokiAJaeger: hopefully you can send some follow-up mail on local development.20:04
amotokior we can encourage to push code without running local tests :(20:05
AJaegeramotoki, I don't have an answer, hope mordred can give one20:06
*** dizquierdo has quit IRC20:07
*** yamamoto has quit IRC20:08
*** jamesmcarthur has quit IRC20:09
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Perform late validation of secrets  https://review.openstack.org/55304120:09
pabelangerlooks like our fedora mirror might be out of sync20:16
pabelangerconnection limits with kernel.org20:16
mnaserpabelanger: ^ i might have someone to talk to about that, but i don't know if they'd be willing to adjust things20:21
mnaser(if you'd like)20:22
clarkbpabelanger: I was going to switch fedora when I switched centos for that reason but didn't bceause of the atomic image mirroring20:22
clarkbpabelanger: it seems that fewer places mirror the atomic images20:23
clarkband we couple the two things together20:23
clarkbused limestone networks mirror since we know logan- there now :) and the mirror site claimed they have more bandwidth than kernel.org and its closer geographically to our mirror update node20:24
openstackgerritMerged openstack-infra/zuul master: Fix safe path check for directories containing symlinks  https://review.openstack.org/55302920:24
pabelangerclarkb: mnaser: I think I found a place for atomic rsync, testing now20:25
clarkbpabelanger: one idea I had at the time and have not had time to think about much since is decoupling atomic mirroring from fedora mirroring20:25
clarkbalso is anyone even using the atomic images?20:25
ianwspeaking of mirrors; re our discussion i've backported and built bionic-era afs 1.8 packages in https://launchpad.net/~openstack-ci-core/+archive/ubuntu/openafs-1.8-xenial.  i've had it running a recursive md5 on the mirrors overnight on a host, only reading, but no issues or warnings20:26
*** jamesmcarthur has joined #openstack-infra20:27
*** e0ne has quit IRC20:27
clarkbthat sounds promising20:27
*** jamesmca_ has joined #openstack-infra20:29
*** jamesmcarthur has quit IRC20:29
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Update fedora mirroring to pubmirror1.math.uh.edu  https://review.openstack.org/55305220:30
pabelangerclarkb: ianw: mnaser: testing ^ now20:30
mnaserclarkb: afaik at the time i got the atomic images for magnum to use20:31
* mnaser looks if they're still using them20:31
*** camunoz has quit IRC20:32
mnaserclarkb: https://github.com/openstack/magnum/blob/master/magnum/tests/contrib/gate_hook.sh#L88 yes they are20:36
*** dizquierdo has joined #openstack-infra20:41
corvusfungi: did we ever consider using shadow *projects* in the same gerrit?20:43
*** salv-orlando has joined #openstack-infra20:43
corvuseg, review.openstack.org/security/openstack/nova.git ?20:43
*** efoley has quit IRC20:45
fungicorvus: interestingly, i don't think that was ever brought up as a possibility20:45
fungiit's certainly worth considering20:46
pabelangerokay, review-dev01.o.o failed to launch, I see some java exceptions. I'll dive into them after I get some food20:47
fungipabelanger: bootstrapping a new gerrit server may be not entirely possible from puppet too... i think we end up copying/moving in the static data and database20:48
*** eharney has quit IRC20:49
pabelangerfungi: yah, it would be helpful to see how we've done it in the past. Do you know of any etherpads with that info?20:50
fungii don't, but should just be /home/gerrit2 which needs to be copied over20:51
*** esberglu has quit IRC20:52
*** dprince has quit IRC20:52
fungiit'll end up wanting to connect to the existing trove db though, so you may need to turn down the gerrit service on old review-dev if bringing up one on review-dev0120:52
pabelangerokay20:53
*** _jbadiapa has joined #openstack-infra20:54
pabelangerfungi: ianw: clarkb: mnaser: https://review.openstack.org/553052/ works as expected to mirror fedora / atomic via rsync. If you'd like to review / approve20:54
tonybevrardjp: I'm around for the next 30mins but I suspect that you're don for the day?20:54
tonybpabelanger: Yup that is indeed what I was asking for20:55
*** esberglu_ has joined #openstack-infra20:55
*** esberglu_ has quit IRC20:56
*** esberglu_ has joined #openstack-infra20:56
*** jbadiapa_ has quit IRC20:57
evrardjptonyb: I am close to done indeed20:58
evrardjptonyb: was wondering what's the status of our EOLing20:58
*** rossella_s has quit IRC20:58
tonybevrardjp: SOrry I didn't get to it yesterday.  I'll do it now, if you're around for the next 10mins you can ack on the list and then I can try to finish it today.  DOes that work for you?20:59
evrardjpyeah sure20:59
* fungi disappears to find food, bbiaw21:01
*** rossella_s has joined #openstack-infra21:02
*** trown|ruck is now known as trown|outtypewww21:03
*** kgiusti has left #openstack-infra21:04
*** yamamoto has joined #openstack-infra21:04
*** dizquierdo has quit IRC21:05
*** rcernin has joined #openstack-infra21:08
*** tojuvone has quit IRC21:08
*** tojuvone has joined #openstack-infra21:09
*** yamamoto has quit IRC21:10
*** Goneri has quit IRC21:12
*** rossella_s has quit IRC21:12
*** rossella_s has joined #openstack-infra21:17
clarkblooks like I can get out of my driveway again21:18
*** tojuvone_ has joined #openstack-infra21:20
*** tojuvone has quit IRC21:20
tonybevrardjp: So all my tools failed so I created the list manually, please check it carefully if you're still functional21:20
tonybevrardjp: the mail shoudl hit the list RSN21:20
*** lpetrut has quit IRC21:22
corvusokay, both zuul security fixes for today have landed21:22
corvusi'll start working on restarting the executors21:22
clarkbcorvus: thanks, let me know if I can help21:24
evrardjptonyb: will have a look21:24
corvusclarkb: thx, should be straightforward21:24
evrardjptonyb: I am expecting the fastest email delivery right now! I trust SMTP for getting my emails as soon as possible!21:25
evrardjpthat's a terrible joke, but I like it.21:25
*** dave-mccowan has quit IRC21:26
tonybevrardjp: Always as son as possible not always as soon as you'd like ;P21:26
evrardjptonyb: that's fairly true!21:27
tonyb:)21:27
*** rossella_s has quit IRC21:28
AJaegerclarkb, mordred, send patches for nearly all repos that we broke with tox-siblings. Exception: networking-cisco since it installs from old branch. Added to https://etherpad.openstack.org/p/zuulv3-remove-zuul-cloner21:29
* AJaeger is tired now ;(21:29
clarkbAJaeger: sambetts|afk was tlking about stuff like that in the ironic channel earlier today21:29
clarkbI think at the very least they are aware of what is going on so we likely don't have to solve that for them?21:29
AJaegerclarkb: I solved it for those issues that i was aware21:30
AJaegerclarkb: let me check ironic on eavesdrop...21:30
evrardjptonyb: some of those don't have the branch stable/newton, but if your code is resilient, it should be fine, right?21:30
evrardjp:D21:30
evrardjplet me confirm by email21:31
clarkbAJaeger: also you should get some sleep if you are tired :)21:31
*** andreas_s has joined #openstack-infra21:31
tonybevrardjp: Yeah the EOL tool will handel that.  Sorry I had to do it my hand it seems quicker than debugging my tool :/21:31
* tonyb takes kids to school21:31
corvusstarting executors now21:32
AJaegerclarkb: will do...21:32
AJaegerclarkb: hope TheJulia will followup to my email with her concerns21:32
corvusup and running21:32
*** rossella_s has joined #openstack-infra21:32
TheJuliaAJaeger: Can now be nap time instead of email time?21:33
clarkbfor me its try to think with the loud construction outside21:33
clarkbtime21:33
*** dave-mccowan has joined #openstack-infra21:33
AJaegerTheJulia: we can followup tomorrow. Let me just assure you that the change does test exactly the same as it did before - it installs neutron from git without any changes.21:34
AJaegerTheJulia: http://lists.openstack.org/pipermail/openstack-dev/2018-March/128311.html - and yes tomorrow, please ;)21:34
TheJuliaAJaeger: I now understand that after clarkb stepped in, it is all the other things that doesn't have our internal magic that is concerning21:35
TheJuliaI'll try to put a chain of words together tomorrow that will hopefully make sense21:35
AJaegerSure - and sorry for breaking some stuff ;( There was an unexpected side effect :(21:36
*** andreas_s has quit IRC21:36
TheJuliaAJaeger: such is life :)21:36
*** aeng has joined #openstack-infra21:36
* AJaeger calls it a day now and waves good night21:36
TheJuliagodnight21:36
TheJuliagoodnight21:36
evrardjpgood nigh AJaeger21:37
*** esberglu_ has quit IRC21:38
*** ihrachys_ has joined #openstack-infra21:38
*** ihrachys has quit IRC21:40
clarkbOh! before I forget I did sign up infra for an onboarding session in vancouver. They are 40 minute sessions and dmsimard has volunteered to help out with it21:40
dmsimardclarkb: are the schedules due this week ?21:41
TheJuliadmsimard: I think they are just trying to figure out what teams want them right now21:41
*** edmondsw has quit IRC21:41
clarkbdmsimard: no they are due in a couple weeks iirc21:42
clarkbseems like one of the emails siad when we'd figure that all out but ya for right now they just want signups21:42
dmsimardclarkb: couple weeks ? Ouch. That doesn't leave a lot of time to prepare things if you get accepted talks21:43
*** erlon has quit IRC21:43
clarkbdmsimard: is the concern scheduling conflicts or knowing to prep for it?21:44
clarkbdmsimard: the onboarding sessions is basically confirmed at this point so we should be prepared for it21:44
clarkbas for avoiding conflicts I think that is why they ask for our names upfront21:45
dmsimardMostly knowing whether or not we need to prepare talks yeah21:45
clarkbtalks are separate21:45
clarkband notifications of acceptance go out this week aiui21:45
*** jamesmca_ has quit IRC21:45
dmsimardAhhh TIL.21:46
clarkbfungi: I've gone ahead and approved the docs update for PTL reference. Should we work to update the governance publications?21:48
clarkbdmsimard: one of the things they do at the summits is give projects update time slots and onboarding time slots. Those are assigned via a queue until they run out of room. This is separate of the cfp process21:49
dmsimardYeah I just didn't realize the project slots were done that way21:50
*** esberglu has joined #openstack-infra21:52
*** ricardoas has joined #openstack-infra21:55
*** iyamahat has quit IRC21:59
*** wolverineav has quit IRC21:59
*** wolverineav has joined #openstack-infra22:00
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Perform late validation of secrets  https://review.openstack.org/55304122:01
*** wolverineav has quit IRC22:01
*** wolverin_ has joined #openstack-infra22:01
ianwpabelanger: mind a poke at https://review.openstack.org/550683 to add a no_timeout flag to the reprepo script?  i always end up copying the script and editing it22:03
*** iyamahat has joined #openstack-infra22:03
clarkbianw: are you planning to update afs client on mirror-update soon? or are you going to keep doing "burn in" first?22:03
ianwclarkb: first i was going to go through all the current errors, just sshing in now :)22:05
*** kjackal has quit IRC22:05
*** bobh has quit IRC22:06
*** owalsh_ has joined #openstack-infra22:06
*** yamamoto has joined #openstack-infra22:06
*** kjackal has joined #openstack-infra22:08
*** owalsh has quit IRC22:10
*** owalsh_ has quit IRC22:11
*** yamamoto has quit IRC22:12
*** myoung|rover is now known as myoung|afk22:13
ianwi'm pulling apart what's supposed to be happening with the mariadb mirrors; they're using the wrong volume name, at least for the vos release22:13
*** priteau has quit IRC22:15
*** owalsh has joined #openstack-infra22:16
pabelangerianw: +222:20
*** felipemonteiro_ has joined #openstack-infra22:20
*** e0ne has joined #openstack-infra22:21
ianwis there any standard puppet way of injecting some randomness into cron job timing?22:21
ianwwe've got two mariadb syncs for 10.x and 10.y ... it would be good if they ran offset from each other22:22
*** e0ne has quit IRC22:22
pabelangerianw: sleep $((RANDOM\%600) && foo22:23
pabelangeris what we use in other cases in system-config22:23
*** felipemonteiro__ has quit IRC22:23
openstackgerritIan Wienand proposed openstack-infra/system-config master: Fix mariadb sync  https://review.openstack.org/55308522:24
ianwpabelanger: ^ hmm, worth adding in there?22:24
ianwmaybe we should see if there is an actual problem before solving it :)22:25
clarkbhrm if using the same volume they could could fight each other22:26
ianwthe vos release should be atomic, though?22:28
clarkbianw: one vos release could happen while reprepro is updating and that will potentially release a halfway completed repo22:28
clarkbwhich could in theory lead to errors updating packages in jobs?22:28
ianwurgh, yeah.  maybe it would be easier to just make new volumes22:29
ianwwhere are these actually used?  we're not carrying them for no reason are we?22:30
pabelangeryah, new volumes is likely safest22:30
openstackgerritMerged openstack-infra/system-config master: Stop embedding (outated) PTL contact info in docs  https://review.openstack.org/55009222:30
pabelangerand pretty cheap22:30
clarkbor combine them all into a single lock and update script22:30
clarkbsimilar to how fedora and atomic happen together22:30
ianwexport NODEPOOL_MARIADB_MIRROR=${NODEPOOL_MARIADB_MIRROR:-http://$NODEPOOL_MIRROR_HOST/ubuntu-mariadb}22:31
ianwi'm not seeing any users of that...22:31
pabelangerianw: I want to say OSA is using it22:32
clarkbya I think it was OSA that we set it up for22:32
*** owalsh has quit IRC22:32
*** ldnunes has quit IRC22:32
*** owalsh has joined #openstack-infra22:33
*** ldnunes has joined #openstack-infra22:33
clarkbcodesearch can't seem to find anything using it though22:33
clarkbhttp://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/tree/roles/nodepool-setup/templates/mirror_info.sh.j2 is interesting though22:34
ianwhow about i send a openstack-dev mail about it, and in a few days i'll either fix it, or remove them?22:34
clarkbya and definitely ping osa to follow up22:35
pabelangermaybe on stable branches?22:36
clarkblooks like mariadb is available on trusty and xenial and bionic22:37
*** jamesmcarthur has joined #openstack-infra22:40
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Added endpoint to get/delete rsvp template by id  https://review.openstack.org/55308722:40
clarkbI guess it could be a version thing22:41
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Perform late validation of secrets  https://review.openstack.org/55304122:41
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Perform late validation of nodesets  https://review.openstack.org/55308822:41
*** danpawlik has joined #openstack-infra22:41
openstackgerritMerged openstack-infra/openstackid-resources master: Added endpoint to get/delete rsvp template by id  https://review.openstack.org/55308722:42
*** jamesmcarthur has quit IRC22:45
ianwclarkb / pabelanger: sound ok -> http://paste.openstack.org/show/701002/  ?22:46
clarkbdoing a quick scan of their repos (at least on master) they appear to install galera from percona/galera's repos?22:46
clarkbI don't think that needs a mariadb to boostrap22:46
*** signed8bit is now known as signed8bit_Zzz22:46
clarkbianw: lgtm22:46
*** dave-mccowan has quit IRC22:47
*** felipemonteiro_ has quit IRC22:49
pabelangerianw: wfm22:51
ianwok, moving on ... http://paste.openstack.org/show/700292/ all the rest seem to be stopped due to lockfiles, with unknown db consistency states, i'll just try them one-by-one22:51
*** danpawlik has quit IRC22:52
pabelangerclarkb: ianw: mind a +3 on https://review.openstack.org/549643/22:52
pabelangerfor bionic testing22:52
openstackgerritMerged openstack-infra/system-config master: Update fedora mirroring to pubmirror1.math.uh.edu  https://review.openstack.org/55305222:53
clarkbpabelanger: at ptg I thought it would be good to not gate on bionic. I sort of still feel that way.22:54
*** iyamahat has quit IRC22:54
clarkbAt least in this case it is infra itself which should address problems22:54
clarkbpabelanger: concern is blocking updates to zuul-jobs due to beta being broken out side of our control22:55
clarkbbut I guess even if check fails we won't gate so maybe that is moot22:55
clarkbianw: one thing I keep pondering and worrying about is these one off mirrors for things like mariadb and ceph and so on. We don't seem to do a great job keeping them up to date because each new software version requires new mirror configs22:57
clarkbianw: it would be nice to get away from mirroring them and instead focusing on the distro releases which are released on a fairly well known cadence that we have to prepare for anyways and should in most cases have this software also packaged up22:57
clarkband maybe we point at tumbleweed for people wanting latest and greatest testing (fedora too)22:58
pabelangerclarkb: yah, feel the same myself. However, if jobs are running against bionic, we likely want to ensure some of our playbooks are working.  Given how smooth testing of bionic has been so far, I think right now it is looking unlikely that something will break (famous last words) :)22:58
clarkbpabelanger: in this case we should probably be prepared to remove these jobs if they start failing due to a broken update22:58
clarkbthat is easy enough to do so I'm ok moving ahead, just wanted to call that out22:59
*** hongbin has quit IRC22:59
pabelangersure, I think that is fair22:59
ianwclarkb: i agree; i think maybe jobs are setup and then removed and forgotten.  i guess if we get more requests, we should consider that carefully23:00
*** iyamahat has joined #openstack-infra23:00
diablo_rojoWould fungi, or some other wonderful infra friend be able to run the migration for sahara on friday? Assuming https://review.openstack.org/#/c/552651/ gets +W before then.23:04
persiaclarkb: pabelanger: Maybe non-voting jobs, which won't wedge anything, but will provide forward guidance?23:05
clarkbpersia: ya, at this point I think I'm happy to operate under the idea that they shouldn't change much at this point and things should just work (tm) and address it if they break23:06
*** tellesnobrega has joined #openstack-infra23:06
clarkbfor project outside the bootstrapping of infra I think being a little more conservative is still worthwhile because they don't want to debug random distro fails23:06
*** yamamoto has joined #openstack-infra23:07
persiahttps://wiki.ubuntu.com/BionicBeaver/ReleaseSchedule suggests to me waiting until at least the 22nd, but maybe folk are more conservative than they once were.23:07
clarkbwe are past the debian import freeze23:09
clarkbso at this point it should be relatively minor changes?23:09
clarkbya that is feature freeze time too so probably we are ok23:09
*** wolverin_ has quit IRC23:09
clarkband it would just be bugs breaking us at this point23:09
persiaclarkb: At least when I was an Ubuntu Developer, this just meant that we were supposed to think for a few minutes before pressing the LP button to sync "manually", key being not to introduce any known breaking changes at this point (new versions are fine).23:10
clarkbah23:10
*** wolverineav has joined #openstack-infra23:10
persiaIt was UIFreeze that we started needing to file freeze exceptions for new upstream packages as a general rule (which discourages doing so)23:10
persiaHence me suggesting "wait one more week" :)23:10
*** wolverineav has quit IRC23:11
*** wolverin_ has joined #openstack-infra23:11
persia5 April is the hard freeze, after which it is stable, but I expect fewer folk to file all the paperwork necessary to sync random things after UIFreeze.23:11
openstackgerritMerged openstack-infra/project-config master: Add zuul-jobs for ubuntu-bionic  https://review.openstack.org/54964323:11
*** armaan has quit IRC23:11
*** yamamoto has quit IRC23:12
clarkbpersia: I guess we'll see how hard people think about merging changes :)23:13
clarkbit is straightforward to undo so not a major concern23:13
persiaFair :)23:14
pabelangeryah, we should be at beta1 for bionic now, if their release schedule is correct.23:15
persiaI thought betafreeze became opt-in at the last UDS, such that it only applied to a narrow subset of packages, but I've been away from that long enough that I defer to folk paying more attention.23:17
*** david-lyle has quit IRC23:21
*** rossella_s has quit IRC23:21
*** rossella_s has joined #openstack-infra23:22
*** hasharAway has quit IRC23:23
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Fix on multiresponse parser  https://review.openstack.org/55309523:24
openstackgerritMerged openstack-infra/openstackid-resources master: Fix on multiresponse parser  https://review.openstack.org/55309523:25
*** david-lyle has joined #openstack-infra23:26
*** olaph1 has quit IRC23:27
*** rossella_s has quit IRC23:28
*** olaph has joined #openstack-infra23:28
*** david-lyle has quit IRC23:28
*** rossella_s has joined #openstack-infra23:30
*** david-lyle has joined #openstack-infra23:30
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Allows null on file param on location image/map update method  https://review.openstack.org/55309823:30
openstackgerritMerged openstack-infra/openstackid-resources master: Allows null on file param on location image/map update method  https://review.openstack.org/55309823:31
*** jamesmcarthur has joined #openstack-infra23:36
*** kmalloc has quit IRC23:38
persiaianw: Are you sufficiently satisfied with https://gerrit.openafs.org/#/c/11940/ to think it worth addressing https://bugs.launchpad.net/ubuntu/+source/openafs/+bug/1364544 before bionic closes?23:39
openstackLaunchpad bug 1364544 in openafs (Ubuntu) "Openafs Not Building on arm64" [Undecided,New]23:39
ianwpersia: yeah, everything seems to be ok, for read operations at least.  i think it's definitely worth getting in, the patches are minor really23:41
ianwwe have them built in a ppa; i can actually provide a patch on the existing package23:42
ianwi just built the bionic package into a ppa for xenail, which we're going to try on mirror-update.  that will stress the read & write paths, and should provide good feedback23:43
clarkbianw: thats x86 though right? or are you using the same set of patches for both?23:43
pabelangerHmm, I'm seeing some synchronize tasks failing for some reason23:45
ianwclarkb: that's x86 ... our arm64 packages are a little further on from them, as i just took them straight from git after the arm64 patches merged23:45
pabelangertrying to debug why23:45
pabelangerin zuul jobs23:45
ianwboo ... afs just died on mirror-update, during debian update23:47
clarkbpabelanger: http://logs.openstack.org/89/549789/7/check/tempest-full-py3/ecad750/ara/result/40702208-7a5b-4349-8659-8bec2dff950d/ that?23:47
ianwubuntu uca & debian-ceph-jewel i restored successfully23:47
pabelangerclarkb: no, working on getting logs23:47
ianwsince i'm geting into the big ones (debian/ubuntu) i think i'll take this point to update the afs client, and see what happens23:47
pabelangerhard to tell right now because I am using no_log: True with synchronize23:48
persiaianw: My thought for easiest-acceptance-path was just to add the patch from 11940 to 1.8.0~pre5-1, as straight-from-git requires me to do a more complicated dance.23:49
ianwoh wait ... mirror-update is trusty still?23:49
pabelangeryah23:49
*** esberglu has quit IRC23:50
ianwany reason why?23:50
pabelangerianw: no, we haven't updated it yet23:50
clarkbthe apache thing above appears to be proeprly broken ansible? its stating a file then trying to read the results as registered_result.results.stat23:50
pabelangerclarkb: okay, think there is a regression in recent zuul update23:50
pabelangerhttp://git.openstack.org/cgit/openstack/windmill/tree/tests/base-minimal.yaml#n1123:50
pabelangerno longer works23:50
clarkbbut reading docs it should just be registered_result.stat23:50
ianwwell i guess i know what i'm doing for the next hour or so :)23:50
pabelanger2018-03-14 23:50:19.566129 | bastion |   "msg": "Syncing files from outside the working dir /var/lib/zuul/builds/4d1b0a0210b6427f8e07d6a0d0899140/work is prohibited",23:51
*** dprince has joined #openstack-infra23:51
clarkbI mean yes23:52
clarkbpabelanger: you have to do that in a trusted context don't you? which our normal repo syncing does?23:52
pabelangerhttp://logs.openstack.org/00/553100/1/check/windmill-ubuntu-xenial/4d1b0a0/ara/result/2d0a122b-cedf-49eb-9dd5-ba61561abb0b/23:52
pabelangerclarkb: up until this afternoon, untrusted worked23:52
pabelangerso, does that mean because what it was doing was wrong?23:53
pabelangerbecause I'd expect {{ zuul.executor.src_root }} to be fine in an untrusted context, on executor23:53
clarkbdo you know what the actual path is for src_root?23:54
clarkb/var/lib/zuul/builds/4d1b0a0210b6427f8e07d6a0d0899140/src_root ?23:54
pabelanger        src_root: /var/lib/zuul/builds/4d1b0a0210b6427f8e07d6a0d0899140/work/src23:54
pabelangerhttp://logs.openstack.org/00/553100/1/check/windmill-ubuntu-xenial/4d1b0a0/zuul-info/inventory.yaml23:55
clarkbhrm that is in the work dir23:55
* clarkb needs to figure out what the homedir is23:56
pabelangerlet me look at recent zuul changes23:56
*** hamzy has quit IRC23:56
clarkbenv_copy['HOME'] = self.jobdir.work_root23:56
clarkbthe intent is definitely to allow the homedir and the trusted/ dir23:56
clarkbfor reads at least23:56
openstackgerritJames E. Blair proposed openstack-infra/storyboard master: WIP: test subscribers and permissions  https://review.openstack.org/55310223:57
*** ldnunes has quit IRC23:58
*** hamzy has joined #openstack-infra23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!