Friday, 2018-05-11

*** yolanda_ has joined #zuul00:13
*** yolanda has quit IRC00:16
ianwcorvus: ok, so yeah it doesn't appear to be any redirect etc magic (which i didn't think it would be, as that all happens in the child, but anyway) --> http://logs.openstack.org/03/567703/2/check/dib-functests-python3/e4f0879/job-output.txt.gz#_2018-05-11_00_15_52_37720700:17
*** harlowja has joined #zuul01:06
*** swest has quit IRC01:56
*** harlowja has quit IRC02:07
*** swest has joined #zuul02:10
openstackgerritIan Wienand proposed openstack-infra/zuul master: [DNM] Testing for with_items log output  https://review.openstack.org/56772202:34
openstackgerritIan Wienand proposed openstack-infra/zuul master: [DNM] Testing for with_items log output  https://review.openstack.org/56772202:43
openstackgerritIan Wienand proposed openstack-infra/zuul master: [DNM] Testing for with_items log output  https://review.openstack.org/56772202:55
openstackgerritIan Wienand proposed openstack-infra/zuul master: [DNM] Testing for with_items log output  https://review.openstack.org/56772203:09
openstackgerritIan Wienand proposed openstack-infra/zuul master: [DNM] Testing for with_items log output  https://review.openstack.org/56772203:15
openstackgerritIan Wienand proposed openstack-infra/zuul master: [DNM] Testing for with_items log output  https://review.openstack.org/56772203:32
*** pcaruana has joined #zuul06:32
*** dmellado has quit IRC07:09
*** dmellado has joined #zuul07:10
*** gtema has joined #zuul07:11
openstackgerritArtem Goncharov proposed openstack-infra/nodepool master: fail to build image without known type  https://review.openstack.org/56643707:25
*** jpena|off has quit IRC07:40
*** ssbarnea_ has joined #zuul07:41
*** ssbarnea_ has quit IRC07:49
*** electrofelix has joined #zuul08:00
*** jpena has joined #zuul08:13
*** gouthamr has quit IRC08:34
*** gouthamr has joined #zuul08:36
*** openstack has joined #zuul09:29
*** ChanServ sets mode: +o openstack09:29
*** xinliang has quit IRC09:57
*** ssbarnea_ has joined #zuul09:57
*** xinliang has joined #zuul10:13
*** xinliang has quit IRC10:13
*** xinliang has joined #zuul10:13
*** bhavik1 has joined #zuul10:40
*** bhavik1 has quit IRC10:51
*** xinliang has quit IRC11:26
*** xinliang has joined #zuul11:29
*** xinliang has quit IRC11:29
*** xinliang has joined #zuul11:29
*** electrofelix has quit IRC11:35
*** jpena is now known as jpena|lunch11:43
*** jimi|ansible has joined #zuul12:20
*** jimi|ansible has joined #zuul12:20
*** rlandy has joined #zuul12:35
*** ssbarnea_ has quit IRC12:43
*** TheJulia is now known as needsleep12:47
*** jpena|lunch is now known as jpena12:51
*** ssbarnea_ has joined #zuul13:12
*** openstackstatus has joined #zuul13:13
*** ChanServ sets mode: +v openstackstatus13:13
-openstackstatus- NOTICE: Due to a Zuul outage, patches uploaded to Gerrit between 09:00UTC and 12:50UTC, were not properly added to Zuul. Please recheck any patches during this window and apologies for the inconvenience.13:17
pabelangerTrying to figure out why my email to zuul-discuss@lists.zuul-ci.org hasn't showed up yet, sent it almost 20hrs go13:32
*** openstackgerrit has joined #zuul13:42
openstackgerritArtem Goncharov proposed openstack-infra/zuul master: dont wait infinitely for the connection to zuul_console server  https://review.openstack.org/56786113:43
*** elyezer has quit IRC13:55
*** elyezer has joined #zuul13:58
*** pwhalen has quit IRC13:59
openstackgerritArtem Goncharov proposed openstack-infra/zuul master: fill `delta` with '0' for `creates` and `removes` command.  https://review.openstack.org/56786414:01
*** pwhalen has joined #zuul14:01
*** pwhalen has joined #zuul14:03
*** pwhalen has quit IRC14:09
*** pwhalen has joined #zuul14:13
*** pwhalen has joined #zuul14:13
gtemaany conceptual reasons for not having security_groups in nodepool.openstack.pool? I have difficulties extending default SG due to sharing tenant of zuul with other teams.14:23
openstackgerritMerged openstack-infra/zuul-website master: Add favicon  https://review.openstack.org/56768514:25
corvusgtema: no, i think that would be okay.14:25
gtemacorvus: ok, will work on that the. Thanks14:26
corvusclarkb, Shrews, mordred: ^ in case you can think of any issues gtema may run into14:26
corvuspabelanger: did you get anywhere with the email issue?14:27
corvuspabelanger: i suspect it's because zuul's mailman is not running14:28
corvuspabelanger: let's switch to infra for this14:28
gundalowcorvus: Thanks for the suggestion of removing uuids. https://softwarefactory-project.io/r/#/c/12174/ Hopefully this is right14:38
gundalowoh, typo14:40
mordredcorvus, gtema: nope - I think having support for security groups is a fine idea - we just never did it ourselves because we don't really use security groups for things14:48
*** openstackgerrit has quit IRC14:49
clarkbcorvus: gtema if it is an older cloud then nodepools turning on and of of instances combined with group to group rules hammers the database. If cloud is newer (like last 2.5years or so) this has been fixed14:50
clarkbmostly just a warning to be careful with group to group rules14:51
gtemaclarkb: thanks. Generally it is a "custom" Hauwei public cloud (OpenTelekomCloud, currently on Newton level, but pending upgrade). Security groups are generally working fine. So far I have not seen problems here, but sure - might be the case.14:57
gtemaany idea why review 567864 might have failed on py36? There is no clear info in logs, just "ft2.17: tests.unit.test_gerrit_crd.TestGerritCRD.test_crd_check_reconfiguration_StringException"? In py35 everything is fine.14:59
rcarrillocruzwait, is https://docs.openstack.org/infra/nodepool/configuration.html the logo now?15:00
rcarrillocruznice!15:00
fungircarrillocruz: indeed it is15:01
fungias seen on https://zuul-ci.org/15:02
fungieven has a page icon as of a few minutes ago15:02
rcarrillocruz++15:03
*** pcaruana has quit IRC15:07
corvusgtema: that usually means the test timed out in some way.  it probably is not related to that change and you can recheck.  the py36 job is new, and may have slightly different timings than the py35 job, so we may encounter some more races in tests for the next while (until we can track them down and fix them)15:09
corvusgundalow: yay!  looks good15:10
gtemacorvus: thanks15:11
ShrewsDoes this seem right to anyone else?  http://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/driver/__init__.py#n68815:17
ShrewsI don't think comparing __dict__ between objects is doing what we might expect it to be doing15:17
corvusShrews: how so?15:18
Shrewssimple test shows me that contains method pointers and not the actual attributes15:18
ShrewsB.__dict__ is {'__module__': '__main__', '__init__': <function B.__init__ at 0x7f5175d18840>, '__eq__': <function B.__eq__ at 0x7f5175d188c8>, '__doc__': None, '__hash__': None}15:19
Shrewsthe attributes I defined in that class don't even appear there15:19
corvusShrews: B looks like a class object, not an instance15:20
Shrewsoh, ha15:20
Shrewsyep15:20
Shrewswell now I'm wondering why we keep redefining __eq__ in subclasses15:21
corvusShrews: like where?15:21
Shrewscorvus: http://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/driver/openstack/config.py#n5015:22
Shrewsdriver configs, for one15:22
corvusShrews: that's probably "better" :)15:23
Shrewscorvus: i think so (if they actually compared types before comparing the attrs).15:24
corvusif that's universal, we could probably drop __eq__ on the base class; or maybe make it abc.15:24
corvusShrews: heh, yeah they should do that too :)15:24
Shrewsi'm going to clean this up15:24
Shrewsotherwise my multi-label patch gets even messier15:25
*** rlandy is now known as rlandy|brb15:40
*** dkranz has quit IRC15:49
*** rlandy|brb is now known as rlandy16:01
*** acozine1 has joined #zuul16:09
*** dkranz has joined #zuul16:20
*** dkranz has quit IRC16:23
*** dkranz has joined #zuul16:25
*** dkranz has quit IRC16:27
*** dkranz has joined #zuul16:28
*** gtema has quit IRC16:53
*** openstackstatus has quit IRC17:00
*** openstack has joined #zuul17:04
*** ChanServ sets mode: +o openstack17:04
tobiashjust did a second try of ansible 2.5 and jobs look good now17:14
clarkbtobiash: was the fix for shell and command in the callback the only major fix you needed?17:19
tobiashclarkb: that was the only fix needed so far17:19
mnaserdoes zuul have a lot of binary dependencies overall (aka: would there be a lot of issues deploying it on arm?)17:20
corvusmnaser: off the top of my head: the zuul executor wants bubblewrap; and there's some crypto stuff in the scheduler17:21
clarkbmnaser: the bindep file should give you a good idea of what is needed. re2 is probably the only one I would worry about. bubblewrap too maybe?17:21
fungii think pyca/cryptography deploys fine on arm though17:21
mnaseri guess one way to find out17:21
fungias far as having its library dependencies met17:21
fungimnaser: multiple ways to find out, but one fairly easy one would probably be trying to add an arm64 integration test since we have nodes of that description17:22
*** jpena is now known as jpena|off17:22
mnaserfungi: oh that's true as well, i was going to try deploying it here17:22
clarkbfungi: ya cryptography should be fine as evidenced by our arm64 nodepool builder which uses cryptography for image uploads iirc17:23
corvusmnaser: pabelanger: could you throw a windmill job on an arm node?17:24
fungithat's sorta what i was wondering too17:24
mnasercorvus: i pinged pabelanger just now to see if i can get windmill docs to run on a vm but i'm looking at doing it upstream too17:24
mnaser:)17:24
pabelangerSure17:25
pabelanger1 minute17:25
mnaseri can fix problems as they appear17:25
corvusmnaser: i can make problems as you fix them.  that's teamwork!17:26
mnaser:D17:26
clarkbbubblewrap is built for arm64 on ubuntu bionic at least. as is libre217:26
mnasermaybe this could be the demo17:26
mnasercause i'm supposed to come up with some interesting one.. not sure how long windmill takes to deploy however17:26
corvushowever long it takes, it will take longer on arm17:27
mnaser:\ so my little 8 minutes i'm given wouldn't be enough17:27
mnaser(this is supposed to be some high end arm hardware but yeah)17:27
fungiif could be a non-live demo with the thumb-twiddling elided?17:27
mnasercould be, but that's no fun17:28
mnaser:p17:28
corvushttp://logs.openstack.org/98/564598/1/gate/windmill-ubuntu-bionic/38a6054/job-output.txt.gz  took 15 minutes.  i have no idea how that's divided up tho17:29
mnaserhttp://logs.openstack.org/83/567883/3/check/windmill-ubuntu-bionic/67cc8e5/logs/ara-report/17:29
mnaserlooks like 1m15s just installing zuul17:29
mnasermaybe prepopulating cache could help[17:29
clarkbprebuilt wheels likely would help17:30
clarkbthen hope nothing has a new release on pypi that day (or use a mirror, or tell it no new downloads)17:30
mnaseryeah a local mirror just for this17:30
mnaserhttp://zuul.openstack.org/stream.html?uuid=45449af46beb4234970c98e995a7b9c9&logfile=console.log time to watch, thanks for the patch pabelanger17:31
pabelangerthe test jobs in windill don't use the wheel mirrors in openstack-infra, mostly to confirm we can build things in the gate17:31
pabelangerI'm guess it will fail when we get to installing bwrap, unless we created an arm64 package17:33
pabelangerwe might want to add ubuntu-bionic to arm64 in that case17:33
clarkbthere is a bionic arm64 package17:33
*** smyers has quit IRC17:33
fungiof bwrap17:34
clarkbyes17:34
pabelangeryah, but we don't have that image on linaro17:34
clarkbah17:34
pabelangerjust xenial and stretch17:34
corvusfetch-zuul-cloner ?17:34
corvusis this still a legacy job?17:34
pabelangercorvus: still happens in our base jobs17:34
pabelangerno17:35
corvusi thought we only put that in the legacy base job17:35
pabelangerremoval has been up for some time, but haven't landed it yet17:35
corvusi apparently thought wrong.  hrm.17:35
clarkbI think monty expected much to break if it went in?17:35
pabelangerhttps://review.openstack.org/513506/17:36
mnaserwe should add a feature to have # of viewers in a zuul job console17:36
mnaseri think we'd see some good numbers on the arm job right now17:36
mnaser:P17:36
*** smyers has joined #zuul17:36
corvuspabelanger, clarkb: how about after summit we try to regroup on that?17:37
clarkbwfm17:37
pabelangersure17:37
corvusthat's several minutes we waste in every job :|17:37
fungisoon post-summit sounds like a great time17:38
clarkbcorvus: isn't it just copying one file?17:38
clarkboh no it has to make a venv right?17:38
corvusyep17:39
corvusbecause we removed it from images17:39
corvus(which was a step which was supposed to last for about 3 days :)17:39
clarkbif only pyyaml were part of stdlib17:40
fungiif pyyaml were part of stdlib, pep 518 would never have ended up using toml17:41
mnaserc/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory17:41
mnaserhmmm17:41
mnaseris it possible libffi-dev is not being installed?17:41
corvuslibffi-dev [platform:dpkg]17:41
corvusthat's from bindep17:41
mnaserhttps://packages.ubuntu.com/xenial/arm64/libffi-dev/download17:42
mnaserit also exists..17:42
pabelangerI think I see the issue17:42
mnaserbootstrap tries to install bindep17:43
corvuswhere do we run bindep?17:43
mnaserso it can run bindep17:43
mnaserbut it fails in that stage, i don't see bindep running17:43
pabelangertools/install_bindep.sh is what runs it, but that happens after the virtuelenv is created17:44
mnaserpabelanger: makes me curious how that is not an issue on other architectures17:47
pabelangermnaser: not sure, will need to debug that in a bit17:48
*** openstackgerrit has joined #zuul17:55
openstackgerritMonty Taylor proposed openstack-infra/nodepool master: Use openstacksdk instead of os-client-config  https://review.openstack.org/56615817:55
pabelangerI noticed when setting up a 3pci system, the tox jobs doesn't work out of box. This is because DIB images, don't include tox by default, and when run ensure-tox role, it will pip install tox into ~/.local/bin/tox but a user then needs to set tox_executable: ~/.local/bin/tox manually, as .local/bin isn't in the default path18:14
pabelangerI was thinking of proposing tox_executable: ~/.local/bin/tox as the default for the tox job because of that18:15
clarkbor maybe just add that to the path?18:15
pabelangerI am not sure best place to do that18:15
clarkbon the shell command running tox?18:16
pabelangerright now I have to shadow zuul-jobs to modify the job to just set tox_executable18:16
clarkbthough I guess if nothing else is installed there then they are equivalent in function18:16
pabelangerseems like a lot of work just for 3pci18:17
clarkbI would update zuul-jobs18:17
pabelangernot sure, we are saying tox_executable is the path to tox in our docs18:18
*** acozine1 has quit IRC18:19
pabelangerwill need to see if command task will accept PATH18:19
clarkboh the tox role isn't installing tox18:19
pabelangerright, we have ensure-tox that does it18:20
pabelangerthis isn't an issue in openstack-infra, because we include tox at build time18:20
clarkbcan ensure-tox register tox_executable to be ~/.local/bin/tox if it installs tox?18:21
clarkbexcept that runs in pre.yaml18:21
pabelangerright18:21
pabelangerand I think would make it more tricky if a user setup tox_executable themself18:21
pabelangerI'm thinking we set tox_environment: ~/.local/bin/tox by default and in the case of openstack-infra, we maybe set a site-variable (like we did with bindep)18:22
clarkbI would have tox check instead18:23
clarkbthen you shouldn't need to change any config anywhere18:23
pabelangeractually, what is we removed tox from infra-packages in openstack-infra, then just used ensure-tox role, like other zuul users would18:25
pabelangerthen we'd wouldn't need to check for more then 1 location18:25
clarkbthat assumes we are the only people preinstalling tox18:25
pabelangerright, give that we don't have docs yet in zuul to actually build a DIB, we could make ask on ML18:26
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Fix ConfigValue comparisons  https://review.openstack.org/56792718:26
Shrewsso that ^ was "fun"18:26
clarkbor just check if it is in path use `tox` else use ~/.local/bin/tox ?18:26
clarkbpabelanger: one hack that can be used is add a symlink to ~/.local/bin/tox to /usr/local/bin in ensure-tox18:28
clarkb(this is actually how I use tox locally)18:28
fungiyeah, works fine so long as the user invoking tox has read access to whichever homedir that got installed into18:30
clarkbShrews: forgive my ignorance but wouldn't the simplest fix to just be compare __dict__ and remove any __eq__ overrides??18:31
Shrewsclarkb: that doesn't take into account class types18:31
Shrewsso a ConfigValue could be equal to some random dict18:31
pabelangermaybe it is just easier to setup PATH=$PATH:$HOME/.local/bin:$HOME/bin for zuul user in dib element18:32
Shrewsclarkb: but also, there are some attributes we DONT want to compare18:32
Shrewsclarkb: see the NOTEs i left about recursion18:32
clarkbShrews: in that case maybe check the isinstance then just super to the base class __eq__ ?18:32
clarkboh if you don't want to compare some things then ya need to override18:32
Shrewsi missed the static driver changes for that  :/18:33
clarkbpabelanger: that doesn't fix the 3pci images does it? if so you could just install tox on those images18:33
pabelangerclarkb: right, same issue modifying a default image18:33
pabelangerokay, I'll look at tox role and see about doing double check18:34
clarkblrwxrwxrwx 1 clark users 24 Oct  3  2016 /home/clark/bin/tox -> /home/clark/venv/bin/tox is my hack18:36
clarkbpabelanger: something like register command: bash -c 'PATH=~/.local/bin:$PATH type -p {{tox_executable}}' if rc != 0 error18:49
corvusclarkb, pabelanger: yeah, the goals should be that ensure-tox works whether tox is pre-installed or not.  and that the tox role works in either case without special arguments (you should only need to set those if you, say, wanted to run tox from some really weird place).18:54
corvusclarkb, pabelanger: so it sounds like something like the check clarkb is proposing in the tox role should maybe do the trick18:54
clarkbthe neat thing about type is that if you give it a full path that is valid it just spits it back at you18:56
clarkbso it should be backward compatible with any full path as well as just executable name18:57
pabelangersure, I'll work on a patch for this afternoon19:01
*** acozine1 has joined #zuul19:07
*** gtema has joined #zuul19:07
SpamapSmmmmm website with logo and tagline19:09
* SpamapS just needs the t-shirt next.19:10
SpamapSand stiiickkarrss19:10
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Fix ConfigValue comparisons  https://review.openstack.org/56792719:17
*** gtema has quit IRC20:09
*** acozine1 has quit IRC21:26
*** rlandy has quit IRC22:03
*** ssbarnea_ has quit IRC22:16
gundalowzuul-jobs/roles/prepare-workspace/tasks/main.yaml's Synchronize src repos to workspace directory is failing for me, thought the task has no_log: true. Can I sub class that and remove the no_log? https://ansible.softwarefactory-project.io/logs/13/13/793d6e03a7af2d23cb6d333409c4079c63514471/check/ansible-test-network-integration-vyos-devel/0cedf45/ara/ hitting issues with a new node type I'm bring up22:39
corvusgundalow: you'll need a new base job that uses a new role instead of the standard prepare-workspace role22:50
corvusgundalow: where's your project config repo again?  (the repo that had that nodepool change earlier)22:52
corvusgundalow: nm, found it: github.com/ansible-network/zuul-config22:52
gundalowcorvus: https://github.com/ansible-network/ansible-zuul-jobs/pull/13/files#diff-dbd9cd0ce7e9a5628770143f1488ff59R38 defines the job that uses the ansible-network-vyos-1.1.822:52
gundalownode22:52
*** smyers has quit IRC22:53
gundalowansible-network-vyos-1.1.8 = Fedora 27 with zuul user & SF's public key22:53
corvusgundalow: want me to write a change real quick that should get you the logs?22:53
gundalowcorvus: That would be great :)22:53
corvusk, just a min22:54
gundalowAt a guess I think it may be auth problems, or maybe dest directory/permissions are wrong22:54
corvusyeah sounds likely22:54
*** smyers has joined #zuul22:55
corvusgundalow: okay here we go: https://github.com/ansible-network/zuul-config/pull/1823:08
corvusgundalow: now since zuul-config is a config repository, zuul's usual behavior of dynamically altering the config doesn't apply -- we have to actually land that change before we can use it23:09
corvusgundalow: but once you do, you can update https://github.com/ansible-network/ansible-zuul-jobs/pull/13  to add "parent: base-test" to the job definition, and it will use that one instead of the normal base job.23:10
gundalowMerged the PR23:10
gundalowAdded https://github.com/ansible-network/ansible-zuul-jobs/pull/13/commits/ce419f41d8b48bbb429a8d039136eea87926b3ac23:15
corvusrunning at https://ansible.softwarefactory-project.io/zuul/stream.html?uuid=6caf6f4ffab748eaa75ba04001b8706b&logfile=console.log23:15
gundalowcorvus: missing rsync23:16
corvusgundalow: yay!  it's an answer at least :)23:16
gundalowAnd something I can fix23:16
gundalowthough not today23:16
gundalowcorvus: nhicher Thank you again for all your help!23:17
gundalowcorvus: Not sure where in the world you are, though it should be weekend for you now23:18
corvusgundalow: np, have a good weekend :)23:18
gundalowThanks for this though, means I can solve more of my own problems now :)23:18
corvusi feel it it should have been weekend yesterday23:18
gundalow:D23:18
corvusgundalow: also, let me know if you have any questions about the pr i submitted23:19
corvusi mean, after weekending23:19
openstackgerritJames E. Blair proposed openstack-infra/zuul master: WIP replace use of asyncio with cherrypy  https://review.openstack.org/56795923:25
corvusmordred, tristanC, Shrews, tobiash: ^ i needed a break from working on slides, so i took a couple of hours to poke at that ^.  i have the unit tests for the basic gearman status check and the websocket console streaming working.23:27
corvusmordred, tristanC, Shrews, tobiash: the advantage of that is that we can drop our dependence on aiohttp (which has caused us a number of problems recently as they are providing a very unstable api), and cherrypy is a more traditional pythonic threaded system, so it should fit in better with how the rest of the system works.23:29
corvuswe can stop typing "await" all the time.23:29
corvusit supports a routes-based dispatcher, so the mapping to what we're doing is pretty straightforward.  i think porting the rest of the webapp will only take a couple hours.23:30
clarkbdoes it do websockets? that was the original aiohttp/asyncio reason iirc23:30
corvusclarkb: ws4py provides a cherrypy plugin to add websocket support23:31
clarkbcool23:31
corvuslocally, i have tests.unit.test_web.TestWeb.test_web_status and tests.unit.test_streaming.TestStreaming.test_websocket_streaming passing23:31
corvusthey probably won't pass in zuul because i forgot to add the packages to requirements.txt23:31
corvusbut there's code in that patch which implements the websockets console streaming23:32
corvus(another nice thing is that with one more minor refactor, i think we can achieve Shrews's dream of having a single helper class for all our streamers)23:33

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!