| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add git timeout https://review.openstack.org/509517 | 00:10 |
|---|---|---|
| jeblair | okay, that should work now. I wrote two tests that exercise it, and verified they work. unfortunately, i could not find a way to do that without actually using the network. i'm not sure if we want to add unit tests that hit our git server, so i've @skipped them for now. we should discuss that further. | 00:11 |
| jeblair | (well, to rephrase, i know i don't want to add unit tests that hit our git server.) | 00:12 |
| jeblair | (we might be able to do this with some semi-complex monkeypatching of popen) | 00:12 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Improve exception handling around lost requests https://review.openstack.org/510594 | 00:20 |
| pabelanger | jeblair: good to know, also okay with local fork | 00:29 |
| *** jkilpatr has quit IRC | 00:59 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: Add unbound role https://review.openstack.org/510725 | 03:21 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: Add unbound role https://review.openstack.org/510725 | 07:08 |
| *** electrofelix has joined #zuul | 07:39 | |
| *** jkilpatr has joined #zuul | 11:03 | |
| tobiash | tristanC: I just tried out your zuul-web patches in my shiny new v3 deployment | 11:23 |
| tobiash | tristanC: I noticed that it loads the status.json only once | 11:24 |
| tobiash | tristanC: I get this js error: http://paste.openstack.org/show/623208/ | 11:24 |
| tobiash | tristanC: do you know which version of jquery we depend on? | 11:25 |
| tobiash | tristanC: ah, 'updated' jquery from 3.2.1 to 1.11.1 and now it works :) | 11:36 |
| leifmadsen | mordred: OHAI | 12:36 |
| leifmadsen | back from AstriCon, so I'm going to be looking into Zuul again this week | 12:36 |
| leifmadsen | I'm going to work on automating what we already documented, and get that as something we can run as a basis to continue more documentation | 12:36 |
| *** dkranz has joined #zuul | 12:44 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Remove integration test playbooks from zuul-jobs https://review.openstack.org/510868 | 12:56 |
| *** dkranz has quit IRC | 13:02 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Remove infra-gate pipelines https://review.openstack.org/510885 | 13:33 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul master: Remove infra-gate pipeline https://review.openstack.org/510886 | 13:35 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul master: Remove infra-check and infra-gate pipelines https://review.openstack.org/510886 | 13:45 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Revert "Use new infra pipelines" https://review.openstack.org/510885 | 13:48 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul master: Revert "Use new infra pipelines" https://review.openstack.org/510886 | 13:49 |
| *** dkranz has joined #zuul | 14:19 | |
| jeblair | aha! i think i figured out how to test the git timeout stuff. next patchset should have working tests | 14:47 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add git timeout https://review.openstack.org/509517 | 15:02 |
| openstackgerrit | Ken Giusti proposed openstack-infra/zuul-jobs master: Parameterize the test-setup role https://review.openstack.org/510907 | 15:03 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add git timeout https://review.openstack.org/509517 | 15:10 |
| jeblair | okay that's ready for review | 15:10 |
| dmsimard | I'm not a frontend guy but we should probably figure out a way to load the status page progressively/asynchronously | 15:10 |
| dmsimard | With >1k changes, it takes >30 seconds for my browser to show anything | 15:11 |
| dmsimard | I mean, we can argue that there should never be >1k changes in the first place but it's probably a good benchmark :) | 15:11 |
| jeblair | dmsimard: yeah. after cutover mnaser is going to pitch in on modernizing it a bit and incorporating it into zuul-web. | 15:11 |
| dmsimard | mnaser is everywhere, EVERYWHERE! :D | 15:12 |
| jeblair | dmsimard: that will let him make some efficiency improvements | 15:12 |
| jeblair | dmsimard: after that -- yeah, i'd love it if we could have it use websockets to get updates to individual changes as they happen. that will take some internal changes. currently zuul can't produce anything like that. | 15:13 |
| jeblair | (as in, there is no internal event stream like "item foo changed") | 15:13 |
| mnaser | jeblair dmsimard i have a local branch i hacked over a few days ago redoing the UI in angular and it was rendering much faster | 15:16 |
| mnaser | the problem in the current UI is that the state/filtering is all done with the UI rather than an in-memory structure for most of things | 15:17 |
| Shrews | jeblair: qq on that change... for the test_clone_timeout() test, since the Repo() __init__ method also calls _ensure_cloned(), why isn't that throwing an exception? | 15:24 |
| dmsimard | jeblair, mnaser: oh, I think we can definitely allow ourselves to think outside the box for the UI. Websockets is a good idea, maybe we can get the UI to consume mqtt/firehose for updates too. | 15:24 |
| mnaser | that's much harder though, i dont know if it can justify the extra amount of logic to maintain a sync'd state | 15:25 |
| pabelanger | sleep 30 to the rescue | 15:25 |
| Shrews | jeblair: oh, maybe it is... we just log it and move on, then explicitly try it again | 15:26 |
| pabelanger | dmsimard: maybe even fedmsg from zuul POV. I admit, I am not familiar with status.json logic in zuul | 15:27 |
| Shrews | jeblair: k, nm. got it now | 15:27 |
| jeblair | Shrews: yeah. i'd like to remove some of those exception handlers, but maybe in a later change | 15:31 |
| kklimonda | can I use ansible variables when defining variables for zuul jobs? something like `variable_name: {{ ansible_var.HOME }}` that will ba later expanded by ansible when we execute playbooks | 15:47 |
| jeblair | kklimonda: does https://docs.openstack.org/infra/zuul/feature/zuulv3/user/jobs.html#variables and https://docs.openstack.org/infra/zuul/feature/zuulv3/user/config.html#attr-job.vars answer your question? | 15:48 |
| Shrews | | 0000180791 | rax-dfw | None | centos-7 | 37936dee-84bf-4d41-89c8-1f1ddadb1f72 | in-use | 00:21:03:32 | locked | | 15:59 |
| Shrews | jeblair: 21 hour lock... anything to worry about? | 15:59 |
| pabelanger | Shrews: ya, we had blocked executors this morning, 4 of them | 15:59 |
| pabelanger | so nodes should be running jobs now | 15:59 |
| Shrews | ah, k. also, i guess that doesn't mean locked for 21 hours | 16:00 |
| jeblair | Shrews: maybe run it down to a job just to make sure? | 16:00 |
| pabelanger | likey 19 hours :) | 16:00 |
| pabelanger | but they were locked during the time executors were blocked | 16:00 |
| jeblair | Shrews: er, doesn't it mean that it has been in-use and locked for 21 hours? | 16:00 |
| jeblair | presumably if it were ever unlocked during that time it would no longer exist | 16:01 |
| Shrews | Unlocked node 0000180791 for request 200-0000348315 | 16:01 |
| Shrews | jeblair: i can't remember if we update the time on status change or not | 16:01 |
| jeblair | Shrews: that is what the time field is supposed to be | 16:01 |
| jeblair | state_time is the time when the current state was set. i believe we have the object api do that for us automatically | 16:02 |
| Shrews | jeblair: ah yeah. we have a separate created_time that we don't output | 16:03 |
| kklimonda | @jeblair hum, it answers my question now that I've dug into ansible docs and found what I was looking for. I wonder if an explicit example, similar to https://gist.github.com/kklimonda/6983e0fa84a62dc051d5afbb78d3efea would make it easier to process what's being said, but maybe that's just me. | 16:03 |
| Shrews | storeNode() updates state_time for us | 16:03 |
| Shrews | (at least in nodepool code) | 16:04 |
| jeblair | kklimonda: it's not just you -- we need to add examples like that to the docs. | 16:05 |
| *** bhavik1 has joined #zuul | 16:06 | |
| jeblair | the neutron ci folks would like us to merge https://review.openstack.org/510580 before the next restart | 16:08 |
| jeblair | i'll add it to the etherpad | 16:08 |
| jeblair | mordred: 510237 has -1 | 16:11 |
| jeblair | mordred: also 510686 | 16:11 |
| mordred | jeblair: looking now | 16:11 |
| pabelanger | +2 | 16:12 |
| mordred | jeblair: gah. linters | 16:12 |
| jeblair | mordred: and other javascript changes too, may want to review the whole set | 16:12 |
| Shrews | jeblair: so, if I get this zuul log output correct (http://paste.openstack.org/show/623262/), that long locked node is for change 510283,1 (https://review.openstack.org/510283), but that is merged and zuul has posted its vote | 16:13 |
| mordred | jeblair: being able to spell javascript helps | 16:14 |
| Shrews | so either my analysis of the zuul logs is wrong, or there is something wrong with zuul. likely the former | 16:14 |
| mordred | jeblair: apparently the correct spelling is not "javscript' | 16:14 |
| jeblair | Shrews: where do you see it reported? | 16:19 |
| mordred | jeblair: https://review.openstack.org/#/c/510686/ POST_FAILURE doens't show anyting in logs that I can see as an error, nor do I see anythin gin the executor logs | 16:20 |
| jeblair | Shrews: i think that item is still in the pipeline | 16:20 |
| mordred | jeblair: I'm rechecking the change itself - I'm a little concerned that there is a post failure with nothing logged, but with nothing logged I can't really debug it further | 16:21 |
| jeblair | mordred: nothing on executor logs? | 16:21 |
| jeblair | mordred: you said that, sorry | 16:21 |
| jeblair | mordred: i got distracted by the word 'gin' | 16:21 |
| mordred | jeblair: :) | 16:22 |
| mordred | jeblair: I mean - there are executor logs - and they show the POST_FAILURE for that build | 16:22 |
| mordred | jeblair: but nothing in them provides any indication as to WHY | 16:22 |
| jeblair | mordred: so the answer is probably in the .json file after the upload | 16:22 |
| mordred | jeblair: we COULD add some code to look for errors in the json file if the final post playbook fails - but that seems like it's potentially crossing a line between executor and content | 16:24 |
| jeblair | mordred: i wonder if we could emit the json from the last post playbook on error.... | 16:24 |
| mordred | jeblair: heh | 16:24 |
| mordred | jinx | 16:24 |
| jeblair | :) | 16:24 |
| mordred | jeblair: we absolutely could do that - and maybe it's a good safety valve for these sorts of things even if it might be a mild layer violation | 16:25 |
| jeblair | mordred: ya. or, maybe we could have a place where zuul copies the json file on post failure. | 16:25 |
| Shrews | jeblair: oh, you're right. | 16:25 |
| jeblair | mordred: we'd need to make sure we don't let that grow too large | 16:25 |
| jeblair | but sort of a dead-letter place | 16:25 |
| jeblair | mordred: which do you think would be better? | 16:26 |
| mordred | jeblair: let's try reading the json and emitting it into the error message as a start - that way people without root might be more empower to help debug an issue | 16:27 |
| jeblair | mordred: how does that help people without root? | 16:28 |
| jeblair | i thought we were talking about having it emit into the executor log, since we're talking about issues during or after the log upload | 16:29 |
| mordred | jeblair: oh- I was talking about emitting it into the message zuul leaves on the change | 16:29 |
| mordred | jeblair: but we can totally just start with executor log and see what it gets us or if it would be useful to log it otherwise | 16:30 |
| jeblair | mordred: yeah, let's just focus on executor -- the output we're talking about could be quite large and gibberish :) | 16:30 |
| clarkb | so we do log something in the json we can pull out? | 16:32 |
| jeblair | mordred: do you have a handle on what we'd need to do to get the partial json into the log; you want to take that? | 16:32 |
| jeblair | clarkb: presumably, yes -- the json should have everything we need to identify the error | 16:33 |
| jeblair | i mean, the console text log may as well, but it's not as complete as the json | 16:33 |
| *** Guest66098 is now known as mgagne | 16:34 | |
| *** mgagne has quit IRC | 16:34 | |
| *** mgagne has joined #zuul | 16:34 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Update statsd output for tenants https://review.openstack.org/510580 | 16:35 |
| jeblair | added to etherpad | 16:35 |
| jeblair | fungi, clarkb: if you have a moment for 509517 i'd appreciate it | 16:39 |
| mordred | jeblair: yah - I'm on it - sorry, was looking at it in a different window | 16:41 |
| fungi | jeblair: sure thing | 16:41 |
| fungi | jeblair: i love the fake git ;) | 16:44 |
| jeblair | fungi: works almost as well as the real one | 16:47 |
| fungi | some might argue it works better! | 16:47 |
| SpamapS | like a broken clock | 16:50 |
| fungi | except that this git is probably right more than twice a day | 16:51 |
| fungi | jeblair: i left a few inline comments in case any of that is something you want to address in a new patchset or followup change | 16:59 |
| jeblair | fungi: ah thx | 17:01 |
| clarkb | jeblair: I'm trying to understand how the fake_git.sh would timeout in the tests. Is it just that python takes more than .001s to fork and exec the bash script? | 17:07 |
| jeblair | clarkb: yes | 17:08 |
| jeblair | clarkb: er, the script also sleeps 30s | 17:08 |
| jeblair | clarkb: which is >>>> .001s | 17:08 |
| clarkb | ugh I need caffeine clearly | 17:09 |
| clarkb | I say the exit in the version case and ocmpletely missed that | 17:09 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Grab json log contents for final post playbook failures https://review.openstack.org/510952 | 17:10 |
| mordred | jeblair, clarkb, pabelanger: ^^ there's a stab at getting us logs for things that fail in the final playbook | 17:10 |
| tobiash | mordred: cool :) | 17:13 |
| *** jkilpatr_ has joined #zuul | 17:13 | |
| clarkb | jeblair: I've +2'd with a comment for clarification as well. But generally lgtm | 17:13 |
| tobiash | mordred: do we want a test for this? | 17:13 |
| tobiash | (not sure if that's testable easily) | 17:14 |
| *** jkilpatr has quit IRC | 17:14 | |
| mordred | tobiash: hrm. it might not be super terrible to test - there's already a test that checks for the file itself - lemme take a stab | 17:14 |
| fungi | dhellmann and i were discussing the tag-releases job in #openstack-release earlier and it dawned on me, that job relies on fetching git notes from its origin remote but zuulv3 doesn't look like it pushes git notes into the prepared repos on nodes... in the short term i'm guessing we should just fix the script to add a remote from which it can pull git notes, but longer-term is this something zuul should provide | 17:15 |
| fungi | directly? | 17:15 |
| tobiash | mordred: do we want this only in executor log or maybe also in the review comment? | 17:15 |
| mordred | tobiash: I was originally thinking review comment - but then I think we came around to thinking putting it in the executor log first to see how it goes | 17:18 |
| mordred | fungi: oh - that's a fascinating question | 17:19 |
| tobiash | mordred: ok, that's probably also easier | 17:19 |
| clarkb | mordred: left comments on the debugging change | 17:19 |
| jeblair | tobiash, mordred: i don't think review comments should have giant blobs of json in them. we need to treat the final post playbooks as a system problem that the admin is responsible for. they need to, quite simply, always work. | 17:20 |
| fungi | mordred: turns out the reason we were discussing it is because we discovered today that it's actually been broken in v2 ever since we switched the job to zuul-cloner some time last year, but we didn't notice because the node we reuse for the job in v2 had an already cached git remote for something like git.o.o up to the point where i deleted all the workspaces yesterday and then zuul-cloner prepared a workspace | 17:21 |
| fungi | which the job was no longer able to use | 17:21 |
| jeblair | (so if they don't work, and the admin has to look at a log, that's okay -- it's the admin's problem) | 17:21 |
| fungi | mordred: because we also don't pull git notes onto the v2 mergers, it would seem | 17:22 |
| tobiash | jeblair: right, sounds good to me, just had this thought | 17:22 |
| tobiash | it might make sense to change this finger url reporting to something more user understandable like 'something went wrong, contact the admin' | 17:24 |
| jeblair | tobiash: as long as it still has the uuid in it. however, we haven't had a finger url get reported in a long time. it seems like our problems have been after the point where that gets set. | 17:27 |
| *** persia has quit IRC | 17:30 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: DNM: testing base-test changes https://review.openstack.org/510965 | 17:31 |
| tobiash | jeblair: ok, then that's probably not so important anymore :) | 17:32 |
| mordred | tobiash: btw - turns out adding a test for this was useful - it found a bug | 17:46 |
| SpamapS | TDD -- it's a thing. | 17:47 |
| SpamapS | I'm quite sad I wasn't able to take the time to start with a test for the load governor. | 17:47 |
| tobiash | Yay :) | 17:47 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Mask dependent config errors https://review.openstack.org/510968 | 17:48 |
| mordred | jeblair: ^^ ooh! yay! | 17:49 |
| mordred | jeblair: +2 with inline comment | 17:51 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Grab json log contents for final post playbook failures https://review.openstack.org/510952 | 17:51 |
| mordred | jeblair, tobiash, clarkb, pabelanger: ^^ | 17:51 |
| mordred | SpamapS: you too | 17:51 |
| SpamapS | can I just say how nice it is to be py3 _only_ ? | 17:54 |
| * SpamapS never wants to see six again | 17:54 | |
| clarkb | mordred: curious what you think about comments on the first patchset | 17:54 |
| jeblair | mordred: good questions; i think i want to leave that line as-is. but i could be wrong. :) | 17:57 |
| mordred | clarkb: I shall go read them | 17:58 |
| mordred | clarkb: ah - good point! | 17:58 |
| jeblair | clarkb: reposnded on 509517 | 18:02 |
| mordred | clarkb: actually - no, we always run all of the playbooks - and we only want to extract info from the log if we fail on the actual final one ... the only way we'll leave that loop before running the final playbook would be if runAnsiblePlaybook threw an exception and we poppoed out of the whole method | 18:02 |
| *** bhavik1 has quit IRC | 18:03 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Grab json log contents for final post playbook failures https://review.openstack.org/510952 | 18:05 |
| jeblair | mordred: lgtm with one suggestion i left on ps2 | 18:06 |
| *** persia has joined #zuul | 18:07 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add comment explaining gitpython requirement https://review.openstack.org/510980 | 18:15 |
| jeblair | fungi: ^ | 18:15 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Grab json log contents for final post playbook failures https://review.openstack.org/510952 | 18:16 |
| fungi | thanks jeblair! | 18:17 |
| mordred | jeblair, tobiash: ^^ ok - that should do it this time | 18:17 |
| fungi | also on the subject of git notes, it looks like we can maybe use the git.remote.FetchInfo.note class attribute? https://github.com/gitpython-developers/GitPython/blob/master/git/remote.py#L202 | 18:19 |
| fungi | i'll noodle on that a bit for post-rollout featureset, and just do a workaround in the tag-releases job for now | 18:20 |
| *** electrofelix has quit IRC | 18:21 | |
| jeblair | mordred: erm, one more note on that. | 18:24 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Grab json log contents for final post playbook failures https://review.openstack.org/510952 | 18:24 |
| mordred | jeblair: DOH | 18:24 |
| jeblair | mordred: i'm happy with it. i don't know if pep8 will be happy with the "as e". | 18:25 |
| jeblair | since it's unused now | 18:25 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Grab json log contents for final post playbook failures https://review.openstack.org/510952 | 18:25 |
| mordred | jeblair: I'm 100% certain it will not be happy with it | 18:26 |
| jeblair | w00t | 18:27 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add git timeout https://review.openstack.org/509517 | 18:35 |
| pabelanger | Yay | 18:40 |
| pabelanger | happy to see us roll that our today :) | 18:40 |
| clarkb | mordred: jeblair 510952 did end up failing test jobs | 18:55 |
| jeblair | mordred, clarkb: left comment about failure -- i think we just need to remove a bad line from the test, but otherwise everything is ok? | 18:59 |
| mordred | clarkb, jeblair: yah - that's me having left in a debugging line | 19:02 |
| mordred | clarkb, jeblair: https://review.openstack.org/510952 Grab json log contents for final post playbook failures | 19:02 |
| mordred | updated | 19:02 |
| jeblair | +3 | 19:03 |
| *** jkilpatr_ has quit IRC | 19:12 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Fix doc typo that missed important words https://review.openstack.org/509905 | 19:12 |
| *** jkilpatr has joined #zuul | 19:27 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Remove integration test playbooks from zuul-jobs https://review.openstack.org/510868 | 19:30 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Grab json log contents for final post playbook failures https://review.openstack.org/510952 | 19:35 |
| jeblair | fungi: can you +3 510968 ? | 19:48 |
| jeblair | would love to have it in upcoming restart | 19:48 |
| fungi | yup, looking | 19:48 |
| fungi | looks great, i agree that'll be a big help starting tomorrow especially | 19:51 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Mask dependent config errors https://review.openstack.org/510968 | 20:13 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add comment explaining gitpython requirement https://review.openstack.org/510980 | 20:13 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Switch statsd config to zuul.conf https://review.openstack.org/511017 | 20:23 |
| *** openstackgerrit has quit IRC | 20:33 | |
| pabelanger | +1 for restarts | 20:37 |
| *** openstackgerrit has joined #zuul | 20:41 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Specify duplicate project defn behavior https://review.openstack.org/511025 | 20:41 |
| *** dkranz has quit IRC | 20:56 | |
| mrhillsman | how does run retrigger zuul job via github | 21:36 |
| mrhillsman | when the issue is openstack failed to build node | 21:37 |
| mrhillsman | no code changes needed to the repo | 21:37 |
| mrhillsman | s/run/one | 21:38 |
| pabelanger | mrhillsman: if the job didn't run, because nodepool didn't bring a node online, nodepool will try again | 21:43 |
| mrhillsman | but it will eventually stop trying correct? | 21:43 |
| pabelanger | no, nodepool will keep trying until it is able to bring the node online to ready state | 21:43 |
| pabelanger | if it keeps failing, then it is possible something is wrong with the cloud | 21:44 |
| mrhillsman | got it | 21:44 |
| mrhillsman | also, after running a job node is destroyed/rebuilt correct? | 21:44 |
| pabelanger | Yah | 21:44 |
| mrhillsman | awesome | 21:44 |
| pabelanger | you can see in nodepool-launcher debug.log | 21:45 |
| mrhillsman | ok cool, will try to figure out what is going on | 21:46 |
| mrhillsman | node failure was the first problem | 21:47 |
| mrhillsman | now some error about finger | 21:47 |
| mrhillsman | simple finger://zuul.local/8595a30af40a4a6a9e8242ec2d041305 : RETRY_LIMIT in 0s | 21:47 |
| pabelanger | mrhillsman: if your zuul / nodepool public? | 21:47 |
| mrhillsman | unfortunately not, working on getting resources for public one | 21:48 |
| pabelanger | mrhillsman: usually means zuul tried to run playbook 3 times, but failed for some reason. You likely need to look in zuul-executor debug.log to see why | 21:48 |
| mrhillsman | just need this one dang job to run hehe | 21:48 |
| mrhillsman | ah, thx | 21:48 |
| mrhillsman | see it - FileNotFoundError: [Errno 2] No such file or directory: 'bwrap' | 21:50 |
| mrhillsman | meh | 21:50 |
| mrhillsman | python 3.5 | 21:50 |
| mrhillsman | ffs | 21:50 |
| mrhillsman | i'm just going to destroy all the things hehe | 21:50 |
| mrhillsman | would hate to 2.7 zuul only to run into another 3.5 related issue | 21:51 |
| mrhillsman | it was all pretty too :( | 21:51 |
| mrhillsman | at least i do not have to redo openstack | 21:52 |
| pabelanger | mrhillsman: Oh, ya. you need to install bubblewrap, we use that to run ansible-playbook in. | 21:53 |
| pabelanger | what OS are you using? | 21:53 |
| mrhillsman | ubuntu 16.04 | 21:53 |
| mrhillsman | i think i can just 2.7 all the things and be ok | 21:53 |
| mrhillsman | will try that since nodepool ain't complaining | 21:54 |
| mrhillsman | if it fails, then i'll just redo it | 21:54 |
| pabelanger | no, you'll need python3 :) | 21:54 |
| pabelanger | zuul-web needs it | 21:54 |
| mrhillsman | really? | 21:54 |
| mrhillsman | ah ok | 21:54 |
| pabelanger | ya | 21:54 |
| pabelanger | you'll also need the following PPA | 21:54 |
| pabelanger | https://launchpad.net/~openstack-ci-core/+archive/ubuntu/bubblewrap | 21:54 |
| pabelanger | that will get you bubblewrap | 21:54 |
| mrhillsman | what if i just bypass zuul-web for now? | 21:55 |
| clarkb | you also can't python3.4, has to be 3.5 or higher | 21:55 |
| clarkb | due to the type annotations being a syntax error otherwise | 21:55 |
| mrhillsman | jim said it probably still has some issues | 21:55 |
| pabelanger | mrhillsman: Hmm, not sure. You won't have console logs, but jobs should run | 21:55 |
| pabelanger | also won't have ability to use secrets | 21:55 |
| mrhillsman | boo | 21:56 |
| clarkb | ubuntu ships with python3.5 by default though | 21:56 |
| mrhillsman | ok, will go with your suggestions | 21:56 |
| clarkb | so not sure why that would be a problem | 21:56 |
| mrhillsman | yep, got 3.5 | 21:56 |
| clarkb | its the dfeault python too | 21:56 |
| pabelanger | ya, think it comes down to how pip was installed | 21:56 |
| mrhillsman | i just python <(curl -sk ...) for pip | 21:56 |
| mrhillsman | nodepool was complaining as well | 21:57 |
| mrhillsman | until i installed pip via 2.7 and reinstalled | 21:57 |
| pabelanger | Ya, safest would be python3.5 all around | 21:57 |
| mrhillsman | ok, will stick with 3.5 | 21:58 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: Add unbound role https://review.openstack.org/510725 | 22:36 |
| Shrews | pabelanger: mrhillsman: technically, nodepool could stop trying to launch nodes for a request. if a node fails 3 times for a provider, the provider will decline the request. if all providers decline the request, the request will be marked as failed | 22:39 |
| mrhillsman | danke | 22:39 |
| Shrews | mrhillsman: also, as others have mentioned, we test nodepool on at least python 3.5. although it *may* work on 2.7 (we recently removed tests for that), we don't guarantee it will continue to do so | 22:41 |
| mrhillsman | interesting, i will try to move it back to 3.5 | 22:42 |
| mrhillsman | i was getting errors, do not recall them | 22:42 |
| mrhillsman | unfortunately | 22:42 |
| mrhillsman | pretty sure it was a module missing | 22:42 |
| mrhillsman | scrolled up to look, it was dib failing | 22:44 |
| mrhillsman | n00b here so at this point i'm just charging through | 22:44 |
| mrhillsman | it is VM environment so i'll go back to init snapshot and do over in a day or so probably | 22:45 |
| mrhillsman | has been good learning so far | 22:46 |
| mrhillsman | i did not setup the repos correctly so working on that, i see ansible attempting to do its thing | 22:47 |
| mrhillsman | it may be easier for a dev to grab things earlier than me as some docs stuff was too cryptic until my brain started piecing it together | 22:48 |
| mrhillsman | hands-on learner unfortunately | 22:48 |
| mordred | mrhillsman: I learn best by breaking and fixing in a loop | 22:49 |
| mrhillsman | ^ | 22:49 |
| mordred | mrhillsman: althugh Shrews might say I learn best by brekaing and having other people fix ;) | 22:49 |
| mrhillsman | hehe | 22:49 |
| Shrews | mordred: i would never say that... | 22:49 |
| Shrews | to your face | 22:49 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Switch statsd config to zuul.conf https://review.openstack.org/511017 | 23:01 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Emit some stats from executor https://review.openstack.org/511073 | 23:01 |
| mrhillsman | great success :) | 23:38 |
| mrhillsman | had to re-read some things | 23:38 |
| mrhillsman | will destroy and rebuild and see about only python 3.5, thx for the assists | 23:39 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!