Tuesday, 2018-06-05

*** AJaeger has joined #zuul00:03
*** rlandy|rover is now known as rlandy|rover|bbl00:11
pabelangerhmm, was JOB_NAME a valid JJB variable we used in openstack-infra? Working on zuul_legacy_vars issue and can't remember00:15
pabelangerit is also just as easy for me to remove the usage of JOB_NAME00:15
clarkbyes I think it is/was00:16
clarkbwas something jenkins set00:16
pabelangerk, we seem to be missing it00:16
pabelangerI can look more00:16
pabelangerin the morning00:16
clarkbSince we sont do templates anymore it is less useful to switch on that in zuulv3 jobs. Also I thibk it was a bit if an antipattern to do so on v200:17
pabelangerright, the usage in RDO is pretty limited, it is likey easier to update JJB to remove the usage. But if we want to continue supporting JJB to ansible, we might want to account for it00:18
pabelangerah, that explains it. ZUUL_WORKSPACE is hardcoded to /home/zuul/workspace in zuul_legacy_vars00:27
pabelangertristanC: ^we'll need to fix that to use ansible_user_home or switch sf.io for rdoproject to use /home/zuul/workspace. Do you mind poking at it?00:28
tristanCpabelanger: why not using {{ ansible_env.HOME }} ?00:33
tristanCor did i missunderstood something?00:34
pabelangertristanC: because zuul-migrate hardcodes shell tasks to use zuul_legacy_vars00:34
pabelangerIdeally ya, we'd use ansible_env.HOME or ~00:35
pabelangerI'm testing a workaround for now00:35
tristanCcan't we update zuul-migrate to use ansible_env instead of zuul_legacy_vars for workspace location?00:35
pabelangeryes, but I am about to log for the night00:35
pabelanger:)00:35
pabelangeroh00:35
pabelangerhmm00:35
pabelangerI'll look in the morning for that00:36
pabelangerI was guess zuul_legacy_vars would be easier00:36
*** harlowja has quit IRC01:07
*** D3VIATION has quit IRC01:11
*** rlandy|rover|bbl is now known as rlandy|rover01:35
*** D3VIATION has joined #zuul01:49
*** GonZo2000 has joined #zuul03:01
*** GonZo2000 has quit IRC03:01
*** GonZo2000 has joined #zuul03:01
*** GonZo2000 has quit IRC03:06
*** GonZo2000 has joined #zuul03:07
*** myoung|off is now known as myoung03:08
*** GonZo2000 has quit IRC03:38
*** CrayZee has joined #zuul04:57
*** rlandy|rover has quit IRC05:22
*** yolanda_ is now known as yolanda05:48
*** myoung is now known as myoung|zzz05:49
*** D3VIATION has quit IRC05:57
*** sshnaidm has joined #zuul06:53
*** pcaruana has joined #zuul06:54
*** Rohaan has joined #zuul07:02
*** pcaruana is now known as pcaruana|worksho07:03
*** sshnaidm is now known as sshnaidm|brq07:05
RohaanHey guys, I was just trying out Zuul on my local machine. Is there any good step-by-step documentation for configuring Zuul for a sample project? I looked up in documentation but could not find any. I've setup zuul on my machine following https://zuul-ci.org/docs/zuul/admin/zuul-from-scratch.html07:06
RohaanI've also looked at https://docs.openstack.org/infra/zuul/admin/quick-start.html . I would really appreciate if someone could help me out here :)07:12
*** hashar has joined #zuul07:17
*** gtema has joined #zuul07:20
*** pcaruana|worksho is now known as pcaruana07:50
*** jpena|off is now known as jpena07:50
openstackgerritArtem Goncharov proposed openstack-infra/nodepool master: Add support for specifying security_group in nodepool  https://review.openstack.org/57085708:01
*** chmouel has joined #zuul08:11
*** electrofelix has joined #zuul09:03
openstackgerritAndrea Frittoli proposed openstack-infra/zuul-jobs master: DNM Fix etc/hosts  https://review.openstack.org/57231309:24
*** eandersson has quit IRC09:50
openstackgerritAndrea Frittoli proposed openstack-infra/zuul-jobs master: Include hostnames in /etc/hosts  https://review.openstack.org/57231310:14
*** bhavik1 has joined #zuul10:23
*** Rohaan has quit IRC10:35
*** bhavik1 has quit IRC10:50
*** _ari_ has quit IRC10:50
*** _ari_ has joined #zuul10:51
openstackgerritArtem Goncharov proposed openstack-infra/nodepool master: Use openstacksdk instead of os-client-config  https://review.openstack.org/56615810:54
*** myoung|zzz has quit IRC10:57
*** weshay has quit IRC10:57
*** myoung has joined #zuul10:58
*** weshay has joined #zuul10:58
*** jpena is now known as jpena|lunch11:04
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: Temporary fix static node assign race  https://review.openstack.org/57234211:11
*** ianw has quit IRC11:13
*** ianw has joined #zuul11:32
*** snapiri- has joined #zuul11:48
*** CrayZee has quit IRC11:50
*** snapiri- is now known as CrayZee11:53
*** CrayZee is now known as Guest9114511:53
*** Guest91145 has quit IRC11:53
*** snapiri- has joined #zuul11:54
*** snapiri- has quit IRC11:54
*** snapiri- has joined #zuul11:55
*** snapiri- has quit IRC11:55
*** jpena|lunch is now known as jpena11:59
*** eandersson has joined #zuul12:19
*** rlandy has joined #zuul12:31
*** rlandy is now known as rlandy|rover12:32
*** D3VIATION has joined #zuul13:00
*** acozine1 has joined #zuul13:02
*** CrayZee has joined #zuul13:15
*** Wei_Liu has quit IRC13:30
*** D3VIATION has quit IRC13:35
*** gtema has quit IRC13:45
openstackgerritFabien Boucher proposed openstack-infra/zuul master: Make Zuul able to start with a broken config  https://review.openstack.org/53551114:00
openstackgerritFabien Boucher proposed openstack-infra/zuul master: Make Zuul able to start with a broken config  https://review.openstack.org/53551114:01
openstackgerritMerged openstack-infra/nodepool master: Add support for specifying security_group in nodepool  https://review.openstack.org/57085714:12
openstackgerritMarkos Chandras (hwoarang) proposed openstack-infra/nodepool master: Add opensuse-150 to nodepool dsvm  https://review.openstack.org/57241614:16
*** gtema has joined #zuul14:24
*** eandersson_ has joined #zuul14:45
*** eandersson has quit IRC14:45
openstackgerritMichael Johnson proposed openstack-infra/zuul-jobs master: Collect the coverage report for npm test jobs  https://review.openstack.org/57026014:53
fungii'm pretty sure we don't have a formal policy, but do we generally consider cfp info for related conferences to be on-topic for the zuul-discuss ml?14:54
openstackgerritMarkos Chandras (hwoarang) proposed openstack-infra/nodepool master: Add opensuse-150 to nodepool dsvm  https://review.openstack.org/57241614:55
fungiin particular, wondering if it would be good to post a reminder there about the ansiblefest cfp being open, to urge some zuul folks to submit zuul-related talks for that14:55
Shrewsseems appropriate to me14:59
clarkbre Rohaan's message is anyone writing the configure a project with a job portion of the docs yet?15:00
Shrewsclarkb: we've discussed the need for it, but no one has started on it afaik15:01
clarkbok just making sure there wasn't a change i should be reviewing that I hadn't found.15:01
*** pcaruana has quit IRC15:05
gtemamordred: does it make sense to retire shade in nodepool like we did for Ansible? https://review.openstack.org/#/c/566158/ would not depend on releasing shade changes then and overall will be simplier (at least in not having mix of SDK and proxy to SDK).15:09
corvusi'm going to be afk for most of the day15:19
clarkbgtema: yes we should switch the dependency over, however I expect we'll want ot continue using the shade bits of the sdk lib15:25
gtemaclarkb: yes, it's clear. I will then start working on it15:27
gtemathanks15:27
*** nguyenhai93 has joined #zuul15:29
*** nguyenhai_ has quit IRC15:32
*** sshnaidm|brq has quit IRC15:33
clarkbgtema: trivia time, nodepool is where shade started life, we had so many things in nodepool to handle various clouds and we realized we needed them outside of nodepool so monty turned it into a proper library15:39
clarkbbut I agree we don't need to use this in a complicated amnner. should simplify15:39
gtemanice15:40
gtemawould be likely also to address some of the TODOs for shade (i.e. caching)15:40
*** yolanda has quit IRC15:50
mordredgtema: yes. there's actually a caching related task I'd love to chatwith you about a little later this week15:51
*** hwoarang has joined #zuul15:55
gtemamordred: great. Please remember I am in Berlin TZ, so probably until 17:00UTC (or per email)15:56
mordredgtema: ++15:56
*** yolanda has joined #zuul16:05
*** harlowja has joined #zuul16:28
*** myoung is now known as myoung|lunch16:43
*** gtema has quit IRC16:45
*** harlowja has quit IRC16:45
openstackgerritFabien Boucher proposed openstack-infra/zuul master: Make Zuul able to start with a broken config  https://review.openstack.org/53551117:03
openstackgerritFabien Boucher proposed openstack-infra/zuul master: Make Zuul able to start with a broken config  https://review.openstack.org/53551117:03
*** jpena is now known as jpena|off17:10
pabelangerI know openstack-infra does merge if necessary for projects, this is because that is what zuul does for testing, but could zuul do / support 'rebase if necessary' strategy with gerrit?17:11
clarkbpabelanger: I think it already does or did. We tested it and gerrit doesn't do the right thing in many cases iirc17:11
pabelangerI seem to remember a long time ago infra was doing cherry-picks, btut opted to stop doing that for some reason17:11
clarkbpabelanger: one major issue is it rewrite the commits themselves17:11
pabelangeryah17:11
clarkbya we also tried cherry picking too whihc rewrites commits too17:12
clarkbbut zuul has a flag to mimic different gerrit behavior in the mergers iirc17:12
pabelangerah, right. I remember that now17:12
*** gtema has joined #zuul17:14
openstackgerritMarkos Chandras (hwoarang) proposed openstack-infra/nodepool master: [DNM] Add opensuse-150 to nodepool dsvm  https://review.openstack.org/57241617:24
*** electrofelix has quit IRC17:27
*** gtema has quit IRC17:30
*** myoung|lunch is now known as myoung17:45
openstackgerritClark Boylan proposed openstack-infra/zuul master: Send keepalives on the log callback connection  https://review.openstack.org/57250317:47
clarkbtobiash: Shrews fungi ^ moving here. I don't really like this the more I think about it because the 5 second timeout seems too aggressive17:48
*** hashar is now known as LeftForDead17:49
fungiwhat was the reason for choosing 5 seconds? is that just a default value from somewhere?17:49
clarkbfungi: I4e395b650e83a78d66aa362a420ef0d836c80d35 is the change that added it I think it may have been arbitrary17:50
fungigot it17:50
Shrewsthat does seem pretty low17:51
clarkbtobiash: mentions in -infra we could see if increasing the timeout after connecting works17:51
Shrewsclarkb: maybe after the connection is established, we just unset the timeout on the socket?17:52
Shrewsi did this somewhere either in zuul or nodepool (i think)17:53
clarkbya let me throw that patch up as a followup17:53
Shrewssetsockopt() call or some such17:53
clarkbsettimeout()17:54
* tobiash switches to computer17:54
clarkbtobiash: if you are on vacation don't worry about it :P17:54
tobiashI'm not on vacation but after work ;)17:55
tobiashbut discussing is much easier at computer ;)17:55
tobiashclarkb: setting the timeout will cause python emit a timeout exception on every blocking socket operation17:55
tobiashclarkb: so I see two choices, first unset the timeout after establish or handling the timeout exception (independent of the tcp state) during reading17:56
openstackgerritClark Boylan proposed openstack-infra/zuul master: Disable timeouts on the log streaming connection  https://review.openstack.org/57250317:56
tobiashI think just sending keepalives just hides that problem17:56
clarkbtobiash: sending keepalives ensures that if the connection is valid we don't timeout, we only timeout if the connection breaks17:56
tobiashbecause in case a keepalive is delayed we still might need to handle that connection as it is still theoretical possible17:57
tobiashat least that's how I understood the docs17:57
clarkbthats true but in that case we might consider it broken if it takes many seconds to get a keepalive and we timeout anyways. In any case I've done what shrews and you suggest and just turn off the timeout17:57
Shrewsclarkb: yeah, that's what i did in the _fingerClient() method in zuul/web/__init__.py17:58
tobiashclarkb: https://docs.python.org/3/library/socket.html#socket.socket.settimeout17:58
*** myoung is now known as myoung|biaf17:59
tobiashclarkb: thinking more about this, do we need to cleanup the socket if we hit the establish timeout?18:01
tobiash(that would be a different topic)18:01
clarkbtobiash: hrm now that you ask that I'm not sure we handle the establish timeout properly at all we don't reraise or continue18:02
clarkbwe just log18:02
clarkbtobiash: in theory if we continue there are reraise the socket object referred to by the old s will be garbage collected18:02
tobiashclarkb: maybe we want to return if we hit that exception18:03
tobiashotherwise we try to send something over the socket which is in an unknown state18:03
clarkbtobiash: yup18:04
clarkbI expect that it will raise a new exception due to that and then effectively return18:04
clarkb(whcih will gc the socket object)18:04
tobiashyes, probably18:05
tobiashbut that exception will bubble up further than we might want18:05
tobiashor we may even want it to bubble up the same as it would on a read error18:06
tobiashclarkb: oh, in fact after if we hit the exception 's' is not even assigned18:12
clarkbgood point18:12
tobiashso we'll hit a undefined variable exception in that case18:12
clarkbreading this more I think there is a valid situation where you might timeout and that is waiting for the job side daemon to start listening on port 1988518:12
clarkbwe should be aware of that and we may need to icnrease the 5 second timeout if that isn't enough in all situation18:12
clarkbtobiash: read log runs in its own thread though so an exception there or a return should be equivalent. I will send a patch to return so that it is clearner18:13
clarkbalternative would be to continue and just try again in a loop18:14
tobiashclarkb: while at that do you think it makes sense to also emit that error messate to self._log_streamline ?18:14
tobiashclarkb: that patch was added to help diagnosing firewall problems18:14
clarkb++ that way the end user/ops can debug it more easily18:14
tobiashso a loop would circumvent the initial idea18:14
*** CrayZee has quit IRC18:15
clarkbtobiash: we could log and loop it though18:16
tobiashclarkb: a timeout most of the time indicates a filtered port (which is permanent)18:16
tobiashwe already loop on failed connections until the server is listening on the port18:17
openstackgerritClark Boylan proposed openstack-infra/zuul master: Cleanup log streaming error handling  https://review.openstack.org/57251318:17
tobiashbut yes, 5s seem to be tough if you're connecting around the world and low bandwith18:17
clarkbsomething like that maybe18:17
tobiash+2 from me18:19
clarkbhttp://logs.openstack.org/03/572503/2/check/tox-py35/47bac7d/testr_results.html.gz thats a new one. /me looks at fixing it18:21
tobiashclarkb: I've seen this test case failing earlier every now and then, but I don't remember if that exception was the same18:22
clarkbI see the problem will have a patch as soon as I can write down why I think this should fix it in the commit message18:24
tobiashclarkb: same pattern with the undefined variable :)18:24
openstackgerritClark Boylan proposed openstack-infra/zuul master: Fix bwrap leak test  https://review.openstack.org/57251418:25
tobiashclarkb: you made our third (and I think finally successful) attempt to fix that race :)18:27
clarkbha18:27
clarkblets hope18:27
tobiashI'm pretty sure that works out18:27
*** myoung|biaf is now known as myoung18:30
clarkbShrews: review on ^ would be appreciated as it kept the timeout fix from going into the gate18:32
*** EmilienM is now known as EmilienM|PTO18:43
openstackgerritClark Boylan proposed openstack-infra/zuul master: Make streaming test run longer than connection timeout  https://review.openstack.org/57252218:44
clarkbthat attempts to test this better18:44
tobiashclarkb: commented on that18:47
clarkbtobiash: that is good info, thanks. I've got to shift ot infra things now but will try to pick that back up again when I have time18:47
tobiashk18:47
*** yolanda has quit IRC18:49
clarkbtobiash: I guess localhost doesn't do log streaming at all regardless of task type right?18:57
clarkbso ya definitely need a remote shell task18:58
tobiashclarkb: yes18:58
tobiashI think so18:58
openstackgerritMerged openstack-infra/zuul master: Fix bwrap leak test  https://review.openstack.org/57251419:08
openstackgerritIan Y. Choi proposed openstack-infra/zuul-jobs master: WIP Support building pdfs of sphinx docs  https://review.openstack.org/57095719:15
*** yolanda has joined #zuul19:29
openstackgerritMerged openstack-infra/zuul master: Disable timeouts on the log streaming connection  https://review.openstack.org/57250319:49
*** sshnaidm|brq has joined #zuul19:53
openstackgerritAdam Gandelman proposed openstack-infra/zuul master: github: Optimize getPullReviews() a bit  https://review.openstack.org/57007719:55
adam_gtobiash: updated ^ to only cache and omit the comment filtering, tho IMO that should still happen19:57
tobiashadam_g: thanks, will have a look tomorrow (eod here)20:06
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Add a job variable to control javascript link copying  https://review.openstack.org/57254220:11
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Upgrade from angularjs (v1) to angular (v6)  https://review.openstack.org/55198920:14
* mordred waves to adam_g, hands him a mostly unused pie he found laying around20:26
adam_gmordred: thanks. i think i found crumbs to said pie in keyboard of your old thinkpad that ibm reissued to me20:33
fungibetter than ending up with one of those terrible newer thinkpads, i'll wager20:45
openstackgerritIan Wienand proposed openstack-infra/nodepool master: Add openSUSE Leap 15 to dib boot tests  https://review.openstack.org/57256120:57
*** acozine1 has quit IRC20:59
*** myoung is now known as myoung|off21:08
*** josefwells has joined #zuul21:41
openstackgerritMohammed Naser proposed openstack-infra/zuul master: Disable rewriting links in status page with HTML5 mode  https://review.openstack.org/57256621:44
mnaserclarkb, fungi: ^ fix for my little js broke21:44
mnaseroh quick sneak that in before mordred patches to go to angular 6 :P21:44
clarkbha21:45
clarkbI've +2'd it as it fixes issue we are seeing in live deploiyments21:45
josefwellshey guys, I'm trying to set up a zuul instance and I'm having a hard time getting zuul-scheduler up, doesn't like the zuul.yaml in my 'config' project21:45
josefwellsthe instructions here, don't seem to work: https://zuul-ci.org/docs/zuul/admin/github_setup.html21:46
josefwellsis there a way to test changes to that config other than having zuul-scheduler fetch the changes from github?21:47
clarkbjosefwells: thats typically hte easiest way since zuul has to conglomerate all the configs together and make sure they don't conflict or have errors21:48
clarkbjosefwells: it should log and report what the error is though, do you have thta info?21:48
josefwellsI keep getting errors of the form:   extra keys not allowed @ data['trigger']['github']21:48
clarkbjosefwells: ok thats helpful, any chance you can share your config as well?21:49
clarkbjosefwells: however, I think that means your zuul ini config doesn't have a github connection21:49
josefwellsI tried with the version listed here:21:49
josefwellshttps://zuul-ci.org/docs/zuul/admin/github_setup.html21:49
clarkbjosefwells: /etc/zuul/zuul.conf in that document21:50
josefwellsit is fetching the config from github (enterprise-hosted instance)21:50
clarkbif you can share the zuul.conf and zuul.yaml contents with anything sensitive removed that would be helpful21:51
andreafclarkb corvus to complete the tempest multinode job: https://review.openstack.org/563099 https://review.openstack.org/572313 https://review.openstack.org/#/c/564220/ :)21:51
clarkbI guess main.yaml too can't hurt21:51
josefwellsOn another note, I have been working on containerizing all of this, I hope to publish my changes, hopefully others will be able to bring it up more easily21:51
andreafupz wrong room sorry21:52
josefwellsclarkb: sure, lemme see what I can do21:52
andreafhello folks,  I'm interested in demonstrating zuul running on k8s and managing a pool of containers from k8s cluster(s)21:53
clarkbandreaf: have you seen the spec for that?21:54
andreaffor which I need an helm zuul chart and k8s support in nodepool21:54
andreafclarkb not yet, I was going to ask if there was any work already started21:54
andreafto contribute to21:55
andreafclarkb in infra-specs?21:55
clarkbandreaf: tobiash is running zuul in openshift/k8s I don't know if he is using helm for that, but it is generally doable with the gotcha of bwrap maybe needing privileged container21:55
clarkbandreaf: no its a change to zuul, not merged yet I will get a link21:55
clarkbandreaf: the spec covers the second thing of running jobs in k8s resources https://review.openstack.org/#/c/560136/21:56
josefwellsclarkb: I am being swept away right now, sadly.  I'll be back and put my configs in a gist or something when I have more time to sit down21:56
clarkbthey are independent as you can run zuul on k8s and have it talk to openstack VMs or run zuul outside of k8s and have it talk to k8s containers21:56
clarkbjosefwells: ok feel free to ping me and if I am around will take a look21:56
josefwellsclarkb: thanks for the help, I'll stay on it, zuul looks fantastic21:56
*** sshnaidm|brq has quit IRC21:58
andreafclarkb yeah they are independent indeed - I'll look into the spec tomorrow21:58
andreafthanks!21:59
*** josefwells has quit IRC22:01
funginot sure if it's the additional marketing via the osf and summit/opendev or just reaching critical mass but people are really starting to come out of the woodwork giving zuul a try22:03
openstackgerritClark Boylan proposed openstack-infra/zuul master: Make streaming test run longer than connection timeout  https://review.openstack.org/57252222:19
openstackgerritClark Boylan proposed openstack-infra/zuul master: Fix remote stream log test  https://review.openstack.org/57257422:19
*** LeftForDead has quit IRC22:24
openstackgerritMarkos Chandras (hwoarang) proposed openstack-infra/nodepool master: [DNM] Add opensuse-150 to nodepool dsvm  https://review.openstack.org/57241622:52
*** threestrands has joined #zuul23:08
openstackgerritMarkos Chandras (hwoarang) proposed openstack-infra/nodepool master: Add opensuse-150 to nodepool dsvm  https://review.openstack.org/57241623:09
*** harlowja has joined #zuul23:10
*** rlandy|rover is now known as rlandy|rover|bbl23:22
openstackgerritClark Boylan proposed openstack-infra/zuul master: Fix race in slow start test  https://review.openstack.org/57258423:30
clarkbmore test fixes23:30
openstackgerritMerged openstack-infra/zuul master: Disable rewriting links in status page with HTML5 mode  https://review.openstack.org/57256623:39

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!