Wednesday, 2020-03-04

*** felixedel has joined #zuul00:01
openstackgerritMerged zuul/zuul master: Update k8s log streaming release note  https://review.opendev.org/71112400:03
*** Defolos has quit IRC00:16
*** igordc has joined #zuul00:21
*** marvs has quit IRC00:39
*** igordc has quit IRC00:40
*** igordc has joined #zuul01:20
*** jamesmcarthur has joined #zuul01:30
*** igordc has quit IRC01:33
*** jamesmcarthur has quit IRC01:39
*** jamesmcarthur has joined #zuul01:40
*** jamesmcarthur has quit IRC01:54
*** jamesmcarthur has joined #zuul01:57
*** threestrands has joined #zuul02:06
*** jamesmcarthur has quit IRC02:08
*** jamesmcarthur has joined #zuul02:16
*** mhu has quit IRC02:23
*** Goneri has quit IRC02:41
*** jamesmcarthur has quit IRC03:13
*** jamesmcarthur has joined #zuul03:14
*** jamesmcarthur has quit IRC03:31
*** jamesmcarthur has joined #zuul03:53
*** rlandy has quit IRC04:03
*** sgw has quit IRC04:05
*** sgw has joined #zuul04:05
*** jamesmcarthur has quit IRC04:50
*** raukadah is now known as chandankumar05:09
*** evrardjp has quit IRC05:35
*** evrardjp has joined #zuul05:35
*** igordc has joined #zuul05:38
*** igordc has quit IRC06:11
*** dpawlik has joined #zuul06:33
*** threestrands has quit IRC06:34
*** threestrands has joined #zuul06:35
*** threestrands has quit IRC06:36
*** threestrands has joined #zuul06:36
*** threestrands has quit IRC06:37
*** threestrands has joined #zuul06:38
*** threestrands has quit IRC06:39
*** michael-beaver has joined #zuul06:42
*** felixedel has joined #zuul06:43
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: ensure-tox: add prefer_python2  https://review.opendev.org/71105607:21
*** felixedel has quit IRC07:27
*** AJaeger has quit IRC07:32
*** AJaeger has joined #zuul07:35
*** AJaeger has joined #zuul07:36
openstackgerritMerged zuul/zuul-jobs master: ensure-tox: add prefer_python2  https://review.opendev.org/71105607:39
*** felixedel has joined #zuul07:51
*** zxiiro has quit IRC07:54
*** Defolos has joined #zuul07:58
*** jcapitao has joined #zuul08:09
*** tosky has joined #zuul08:32
*** jpena|off is now known as jpena08:48
openstackgerritFelix Edel proposed zuul/zuul master: Make github file annotation levels configurable via zuul return  https://review.opendev.org/71117909:00
*** mhu has joined #zuul09:10
*** michael-beaver has quit IRC09:11
*** hashar has joined #zuul09:12
*** felixedel has quit IRC09:16
*** avass has joined #zuul09:21
*** bhavikdbavishi has joined #zuul09:23
*** avass has quit IRC09:52
*** hashar has quit IRC10:46
*** felixedel has joined #zuul10:47
*** felixedel has quit IRC11:02
*** Miouge has quit IRC11:08
*** Miouge has joined #zuul11:08
*** felixedel has joined #zuul11:08
*** felixedel has quit IRC11:28
*** felixedel has joined #zuul11:40
*** jcapitao is now known as jcapitao_lunch11:41
*** avass has joined #zuul11:49
*** bhavikdbavishi has quit IRC11:50
*** openstackstatus has joined #zuul11:51
*** ChanServ sets mode: +v openstackstatus11:51
*** AJaeger_ has joined #zuul11:55
*** AJaeger has quit IRC11:56
*** AJaeger_ is now known as AJaeger11:57
avasshmm, I'm trying to install zuul-web from source but I'm getting 40412:11
avassI'm following this: https://zuul-ci.org/docs/zuul/howtos/zuul_install.html12:11
tristanCavass: what is yielding 404?12:26
avassthe zuul-web server (?). With a message "The path '/' was not found" I'm probably missing something.12:27
*** jpena is now known as jpena|lunch12:32
*** bhavikdbavishi has joined #zuul12:50
*** Shrews has quit IRC12:54
*** Shrews has joined #zuul12:55
*** rlandy has joined #zuul12:56
*** jcapitao_lunch is now known as jcapitao13:14
*** Goneri has joined #zuul13:16
avasstristanC: looks like zuul/web/static is empty after installing zuul13:16
avasstristanC: and I get the same effect if i install it locally without building the web components first.13:21
avassaccording to this: https://zuul-ci.org/docs/zuul/reference/developer/javascript.html?highlight=web%20static#deploying that's supposed to be built during installation if I'm reading that right.13:21
avassand the Dockerfile builds it explicitly: https://review.opendev.org/gitweb?p=zuul/zuul.git;a=blob;f=Dockerfile;h=a1d4636340799e6a2c30ac6cce6fcdf1bd1d4858;hb=refs/heads/master#l2913:23
*** jamesmcarthur has joined #zuul13:23
*** jamesmcarthur has quit IRC13:23
*** jamesmcarthur has joined #zuul13:24
*** felixedel has quit IRC13:27
*** michael-beaver has joined #zuul13:31
*** bhavikdbavishi has quit IRC13:31
*** jamesmcarthur has quit IRC13:32
*** jamesmcarthur has joined #zuul13:33
*** jpena|lunch is now known as jpena13:33
*** jamesmcarthur has quit IRC13:38
avassYep that solved it.13:42
*** felixedel has joined #zuul13:46
openstackgerritFelix Edel proposed zuul/zuul master: Report aborted changes via Github checks API  https://review.opendev.org/71102313:50
*** nhicher has quit IRC13:51
*** nhicher has joined #zuul13:51
openstackgerritFelix Edel proposed zuul/zuul master: Report aborted changes via Github checks API  https://review.opendev.org/71102313:56
mordredavass: yes - that's is :)14:04
mordredavass: it _is_ supposed to be built for you if you have the javascript tools installed14:05
*** armstrongs has joined #zuul14:08
*** jamesmcarthur has joined #zuul14:11
*** arxcruz is now known as arxcruz|rover14:13
*** jamesmcarthur has quit IRC14:18
*** sgw has quit IRC14:32
*** sgw has joined #zuul14:33
*** sgw has quit IRC14:39
openstackgerritBenedikt Löffler proposed zuul/zuul master: Fix override variables in zuul_return  https://review.opendev.org/71100214:40
openstackgerritFelix Edel proposed zuul/zuul master: Allow check runs to be configured as required status in pipeline config  https://review.opendev.org/71124114:41
*** jamesmcarthur has joined #zuul14:53
*** jamesmcarthur has quit IRC14:58
*** sgw has joined #zuul15:02
*** felixedel has quit IRC15:05
*** felixedel has joined #zuul15:14
*** swest has quit IRC15:19
*** felixedel has quit IRC15:20
*** chandankumar is now known as raukadah15:20
*** jamesmcarthur has joined #zuul15:28
*** jamesmcarthur has quit IRC15:33
*** michael-beaver has quit IRC15:40
*** jamesmcarthur has joined #zuul15:55
corvustristanC: are you planning on updating https://review.opendev.org/708686 or would you like me to?16:00
*** jcapitao is now known as jcapitao_afk16:02
*** jamesmcarthur has quit IRC16:05
avassmordred: I had the javascript tools isntalled, but it didn't work anyway :)16:06
mordredavass: that's ... disappointing16:07
*** avass has quit IRC16:23
*** jcapitao_afk is now known as jcapitao16:31
*** mattw4 has joined #zuul16:37
openstackgerritTristan Cacqueray proposed zuul/zuul master: docs: move components to reference  https://review.opendev.org/70868616:54
*** hashar has joined #zuul16:55
*** Defolos has quit IRC16:55
*** jamesmcarthur has joined #zuul17:11
openstackgerritJames E. Blair proposed zuul/zuul master: WIP: one option for scheduler test app manager  https://review.opendev.org/71128917:16
*** openstackgerrit has quit IRC17:20
*** igordc has joined #zuul17:33
*** evrardjp has quit IRC17:35
*** evrardjp has joined #zuul17:35
*** jcapitao is now known as jcapitao_off17:36
*** jpena is now known as jpena|off17:46
corvustristanC, clarkb, Shrews: what's the status of zk auth?17:46
clarkbcorvus: in general or in zuul?17:47
corvusin zuul17:47
corvuslike, what do we need to do to get to "zuul supports zk auth"?17:47
corvussince that's step on of the scale out scheduler sequence17:48
corvusstep one17:48
clarkbI know the first thing was getting support in kazoo, I believe that happened17:50
corvuslooks like https://review.opendev.org/619155 is relevant17:51
clarkbhttps://kazoo.readthedocs.io/en/latest/api/client.html#kazoo.client.KazooClient.__init__ shows that the client takes certs/keys for authentication17:52
corvusthat patch uses sasl17:52
clarkbya via auth_data17:53
tobiashcorvus: turned out that I was wrong about explitcly requiring check runs. It's implemented in 711289 today17:53
tristanCcorvus: i'm pretty sure topic:zk-auth implemented zk auth when it was proposed... not sure what needs to happen with it now17:53
clarkbthat is probably better since you can force auth on connection which I know tristanC is worried about17:53
corvustobiash: that change number is wrong :)17:53
tobiashwhoops17:53
clarkbtristanC: corvus that change doesn't seem to be in merge conflict. Maybe recheck?17:53
clarkband go from there?17:54
corvusclarkb: yeah, i rechecked the nodepool change17:54
tobiashI meant 71124117:54
corvushttps://review.opendev.org/619156 is the corresponding zuul change17:54
corvusit's green as of feb 2017:54
*** openstackgerrit has joined #zuul17:54
openstackgerritTristan Cacqueray proposed zuul/nodepool master: Implement zookeeper-auth  https://review.opendev.org/61915517:54
tristanCcorvus: unfortunately the zuul change is also un-tested17:55
corvustristanC: oh... there was testing in the nodepool change; can we do the same for zuul?17:55
corvustobiash did add https://review.opendev.org/657096  which is good, but we should have it in actual unit tests too17:56
tristanCcorvus: yes sure, though we can also merge the nodepool side first, then use the same testing in zuul17:57
corvustristanC: yeah, sounds like there are no blockers for the np change17:58
corvus(though a nodepool with zk auth but no zk auth in zuul is not useful -- maybe we want to wait until they're both ready to merge?)17:58
*** sshnaidm is now known as sshnaidm|afk17:59
corvustobiash: https://review.opendev.org/711241 lgtm but needs a doc update18:00
tobiashcorvus: oh yes, I think he said he'll add the docs tomorrow to that change18:01
Shrewsthe only reason i didn't merge the np change was wanting the zuul equivalent ready to go18:03
Shrewswhich i noted in my last +2 on it18:03
corvusShrews: ah yep.  i think i agree with that18:03
corvusso i think the blocker is now adding equivalent testing to zuul18:04
*** igordc has quit IRC18:04
*** igordc has joined #zuul18:05
corvustristanC: are you planning on doing that?18:10
corvustobiash: interesting, the github api uses the more-british spelling of cancelled (canceled is the more-american spelling and is what zuul uses)18:11
corvusenglish is neat18:11
tobiashcorvus: interesting18:13
clarkbcorvus: I seem to recall inconsistent application of z's and s's in places like dib where you had new zealand and australian english mixed with american english18:13
tobiashI thought it started in america18:13
clarkbdib was started by a kiwi18:14
clarkbthough really it started wtih a team18:14
clarkband so there was a mixture of english18:14
mordredya - but several of them enjoyed their more-british english and advocated for it in ways like using it in code18:14
tristanCcorvus: well i can work on that stack again, but please let's keep the feedback loop short. it has been more than a year i worked on that feature, up to a point where both zuul and nodepool implementation had +1 +218:16
tobiashI guess I mix that up by myself as well as I learned british english at school and transitioned more to american spelling at the university and at work18:16
corvustristanC: agreed.  i'll try to watch for changes, but if i miss an update, feel free to ping me and i'll review asap18:17
openstackgerritTristan Cacqueray proposed zuul/zuul master: Implement zookeeper-auth  https://review.opendev.org/61915618:18
corvusi assume the github checks api was implemented by their london office18:18
mordredcorvus: this is why we should implement all APIs in german18:20
*** armstrongs has quit IRC18:25
tobiashmordred: veto :)18:25
tobiashLuckily I can't even remember when I saw a program written in german the last time...18:27
*** igordc has quit IRC18:27
mordredtobiash: yeah ... we might have to increase the max line length ...18:27
clarkbjava melody is written in french18:28
tobiashlol18:28
*** sugaar has quit IRC18:29
openstackgerritTristan Cacqueray proposed zuul/zuul master: Implement zookeeper-auth  https://review.opendev.org/61915618:31
*** igordc has joined #zuul18:33
*** tosky has quit IRC18:44
openstackgerritTristan Cacqueray proposed zuul/nodepool master: DNM: debug zookeeper service flags  https://review.opendev.org/71130018:47
tristanChmm, it seems like something change in the way zookeeper is configured, the tests are now failing hard with kazoo.exceptions.NoAuthError18:49
*** jcapitao_off has quit IRC18:50
*** jamesmcarthur has quit IRC18:56
tristanCseems like it is a change only affecting ubuntu, the openshift test, which is running on fedora, doesn't have the issue18:58
clarkbpossible the ubuntu zk doesn't support sasl auth?19:03
clarkbwe can switch to the upstream provided tarball instead if so?19:03
corvusbut used to?19:03
clarkboh if it regressed that would be weird19:04
*** reiterative has quit IRC19:04
*** reiterative has joined #zuul19:04
tristanCfedora provides 3.4.9-13 , and the one in ubuntu is 3.4.10-319:05
corvusi'm poking at it on my local ubuntu machine19:09
mordredI mean - it's nice to know there are hard failures if auth isnt' there19:10
corvusi have replicated locally19:11
corvustristanC:  /usr/bin/java -cp /etc/zookeeper/conf:/usr/share/java/jline.jar:/usr/share/java/log4j-1.2.jar:/usr/share/java/xercesImpl.jar:/usr/share/java/xmlParserAPIs.jar:/usr/share/java/netty.jar:/usr/share/java/slf4j-api.jar:/usr/share/java/slf4j-log4j12.jar:/usr/share/java/zookeeper.jar -Djava.security.auth.login.config=/etc/zookeeper/auth.conf -Dcom.sun.management.jmxremote19:11
corvus-Dcom.sun.management.jmxremote.local.only=true -Dzookeeper.log.dir=/var/log/zookeeper -Dzookeeper.root.logger=INFO,ROLLINGFILE org.apache.zookeeper.server.quorum.QuorumPeerMain /etc/zookeeper/conf/zoo.cfg19:11
corvustristanC: that's my local ps19:11
corvustristanC: do we need to run the acl script?19:12
corvusin the test setup19:12
corvustristanC: from the log: http://paste.openstack.org/show/790310/19:13
corvushrm, it's in the test _setUp method19:14
tristanCcorvus: iiuc, you run the test-setup.sh from 619155 and got this paste after running the acl script?19:15
corvustristanC: no, i ran the test-setup.sh then the unit test.  i did not run the acl script.19:15
tristanC `SaslServerCallbackHandler@120] - Successfully authenticated client: authenticationID=super;  authorizationID=super.`  sounds like the configuration is correct19:16
*** Defolos has joined #zuul19:16
*** jamesmcarthur has joined #zuul19:22
*** dmellado has quit IRC19:24
*** dmellado has joined #zuul19:25
*** mgoddard has quit IRC19:27
*** mgoddard has joined #zuul19:28
openstackgerritTristan Cacqueray proposed zuul/nodepool master: DNM: investigate zookeeper service setup  https://review.opendev.org/71130019:28
*** jamesmcarthur has quit IRC19:33
*** jamesmcarthur has joined #zuul19:47
tristanCperhaps k8s is failing because of test-setup removal in pre ( https://review.opendev.org/#/c/619155/22..24/playbooks/nodepool-functional-k8s/pre.yaml )19:52
openstackgerritTristan Cacqueray proposed zuul/nodepool master: Implement zookeeper-auth  https://review.opendev.org/61915519:52
tristanCin which case, the issue may be a difference in the unittest operations19:53
corvusthere has been a kazoo release since patchset 2219:56
corvusand a couple of pure-sasl releases19:58
tristanCcorvus: it seems like zk-auth is working, at least for nodepool-functional-openshift19:59
corvusdowngrading pure-sasl and kazoo doesn't help19:59
tristanCi thought it was failing in nodepool-functional-k8s because of a diff with ubuntu, but i think it 's because the k8s pre stopped running test-setup role (which setup zk auth)20:00
corvustristanC: do you want to try running the unit tests on centos?20:00
tristanCcorvus: i don't think that's necessary20:01
corvusoh, i thought you were getting at the idea that the zk binary provided by ubuntu may be faulty20:01
openstackgerritTristan Cacqueray proposed zuul/nodepool master: Implement zookeeper-auth  https://review.opendev.org/61915520:02
corvustristanC: what do you think the problem is then?20:03
tristanCcorvus: i was, because openshift integration worked (where zk run on fedora) on not k8s (where zk run on ubuntu). but then, lookint at all the diff between PS24 and PS22 i noticed: the test-setup role got removed ( https://review.opendev.org/#/c/619155/22..24/playbooks/nodepool-functional-k8s/pre.yaml ), which would explain the difference20:03
corvustristanC: that doesn't explain the unit test failures, does it?20:03
tristanCno, there must be something in the 22..24 that would explains the unit test failure20:04
corvustristanC: (also see the comment i just left on ps25; i've verified it's not the current problem with the unit tests on ubuntu, but we should keep it in mind)20:04
corvustristanC: 22 was almost a year ago; a lot of external things could have changed20:05
corvusi have to get lunch now20:06
tristanCi think that line is the issue: https://review.opendev.org/#/c/619155/22..26/nodepool/cmd/config_validator.py@12420:10
tristanCor not *the* issue, but that seems wrong20:11
tobiashcorvus, tristanC: kazoo now needs pure-sasl to work with sasl: https://github.com/python-zk/kazoo/blob/master/requirements_sasl.txt20:28
tristanCtobiash: it is added yes https://review.opendev.org/#/c/619155/26/requirements.txt20:30
tobiashtristanC: at least the zuul change seems to miss it and the job failure indicates that it's missing: https://review.opendev.org/61915620:32
openstackgerritTristan Cacqueray proposed zuul/zuul master: Implement zookeeper-auth  https://review.opendev.org/61915620:40
tristanCtobiash: good catch. though we are looking at nodepool failures20:41
tobiashoh20:41
tristanCso, adding test-setup role back to nodepool-k8s pre run fixed the zk auth issue.20:41
openstackgerritTristan Cacqueray proposed zuul/nodepool master: Implement zookeeper-auth  https://review.opendev.org/61915520:45
*** jamesmcarthur has quit IRC20:49
tobiashcorvus: I like 711289, btw, Jan is ooo this week, so he'll answer next week on your review comments20:52
openstackgerritTristan Cacqueray proposed zuul/nodepool master: Implement zookeeper-auth  https://review.opendev.org/61915520:55
*** jamesmcarthur has joined #zuul21:12
corvustristanC: unit tests are still a problem, right?21:12
corvusyeah, looks like it21:12
corvustristanC: so is the test setup script doing something extra that the unit test fixture setup should be doing?21:13
corvus(but the test setup script *is* running for unit tests, we just also have the extra chroot21:14
corvusso maybe it's the chroot that's the issue?)21:14
tristanCthe test setup is doing the zookeeper auth configuration21:15
tristanCzookeeper service*21:15
corvusright.  that's running for the unit test jobs too, but they are failing21:16
tristanCi think the test setup is correct, it did fixed the auth error happening in the functional-k8s job21:17
corvustristanC: do you agree that the unit test jobs are failing?21:17
tristanCyes, and i haven't figure out why yet21:18
corvusok, just wanted to make sure we were on the same page :)21:18
corvusi'm trying to make a standalone reproducer script21:18
tristanCwould be nice to know if PS22 worked, then we could perhaps bisect what is the issue?21:20
corvushrm, i am unable to reproduce with a simple script that creates a lock21:29
tristanCme neither, the nodepool-update-zk-auth works fine locally. so zookeeper service and kazoo seems to be working as expected21:31
tristanCis it me or all the stack trace are passing through `zk.py", line 1048, in imageBuildLock` ?21:32
corvustristanC: yep, i think i'm narrowing down the problem -- it looks like the tests start a builder without auth info21:33
corvustristanC: nodepool/tests/__init__.py", line 345, in setUp21:34
tristanCi'm on the same line indeed :)21:34
corvusi've confirmed with extra debugging that auth_info is none in the connect call21:35
corvusso maybe that isn't getting the right secure.conf file21:35
tristanCperhaps it's only missing a `if not securefile: securefile = self.setup_secure('zookeeper-auth.yaml')`21:36
corvustristanC: yep, i think that may be it21:37
tristanCok, let me propose that shortly21:37
corvustristanC: yes, adding that to useBuilder fixes it21:37
corvustristanC: sounds good21:37
corvustristanC: i'm going to explore my question about the password now21:37
openstackgerritTristan Cacqueray proposed zuul/nodepool master: Implement zookeeper-auth  https://review.opendev.org/61915521:38
corvustristanC: my test script suggests that we don't need the password in the acl21:38
corvustristanC: so just this should work:  acl = (kazoo.security.make_acl("sasl", "super", all=True),)21:39
openstackgerritTristan Cacqueray proposed zuul/nodepool master: Implement zookeeper-auth  https://review.opendev.org/61915521:42
tristanCcorvus: alright, here is the acl change ^21:42
*** jamesmcarthur has quit IRC21:44
corvustristanC: i think there's one more spot -- see comment on ps29 update_zk_auth.py line 4621:44
*** jamesmcarthur has joined #zuul21:44
openstackgerritMerged zuul/zuul master: Make test setup_config more pure  https://review.opendev.org/70908421:46
openstackgerritTristan Cacqueray proposed zuul/nodepool master: Implement zookeeper-auth  https://review.opendev.org/61915521:46
corvustristanC: lgtm!  let's see what the tests say now :)21:48
tristanCcorvus: tobiash: I really like seing types in zuul source, but they are not checked everywhere, and they might become outdated over time. For example, in https://review.opendev.org/709084, the setup_config is missing a return type (-> str).21:50
*** jamesmcarthur has quit IRC21:56
*** jamesmcarthur has joined #zuul21:57
tristanCerr, actually (-> configparser.ConfigParser). So I think we should either add test, or avoid the annotations21:57
openstackgerritTristan Cacqueray proposed zuul/nodepool master: Implement zookeeper-auth  https://review.opendev.org/61915522:00
*** jamesmcarthur has quit IRC22:08
*** jamesmcarthur has joined #zuul22:10
*** tosky has joined #zuul22:11
*** hashar has quit IRC22:15
*** jamesmcarthur has quit IRC22:22
*** jamesmcarthur has joined #zuul22:32
openstackgerritTristan Cacqueray proposed zuul/zuul master: Implement zookeeper-auth  https://review.opendev.org/61915622:33
corvusjamesmcarthur: i just went through the survey againd and sent an email with a list of suggested changes... i think we're almost there :)22:35
*** jamesmcarthur has quit IRC22:37
openstackgerritTristan Cacqueray proposed zuul/zuul master: Implement zookeeper-auth  https://review.opendev.org/61915622:44
*** jamesmcarthur has joined #zuul22:44
*** jamesmcarthur has quit IRC22:47
*** jamesmcarthur has joined #zuul22:52
*** jamesmcarthur has quit IRC22:53
*** jamesmcarthur has joined #zuul22:54
*** jamesmcarthur has quit IRC23:02
*** jamesmcarthur has joined #zuul23:09
*** jamesmcarthur has quit IRC23:12
*** rlandy is now known as rlandy|bbl23:15
tristanCzuul-maint: it seems like topic:zk-auth is back to a verified state. Please review it, i'll try to address comments before it gets into merge conflict again :)23:17
*** threestrands has joined #zuul23:28
*** Goneri has quit IRC23:36
*** tosky has quit IRC23:40
*** Goneri has joined #zuul23:42
openstackgerritTristan Cacqueray proposed zuul/zuul master: Implement zookeeper-auth  https://review.opendev.org/61915623:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!