| pabelanger | for now, I've disabled zuul_log_partition using a single container | 00:32 |
|---|---|---|
| *** rcarrillocruz has quit IRC | 01:11 | |
| *** rlandy has quit IRC | 01:31 | |
| *** openstackgerrit has joined #zuul | 02:13 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: Uncap cherrypy https://review.openstack.org/601136 | 02:13 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: rewrite interface in react https://review.openstack.org/591604 | 02:14 |
| *** rfolco has quit IRC | 02:54 | |
| *** jiapei has joined #zuul | 03:42 | |
| *** quique|off is now known as quiquell | 05:31 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Fix unreachable nodes detection https://review.openstack.org/602829 | 05:39 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Also retry the job if a post job failed with unreachable https://review.openstack.org/602830 | 05:39 |
| *** AJaeger has quit IRC | 06:15 | |
| *** AJaeger has joined #zuul | 06:18 | |
| *** pcaruana has joined #zuul | 06:33 | |
| *** quiquell is now known as quiquell|brb | 06:55 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add build page https://review.openstack.org/597024 | 07:10 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add job page https://review.openstack.org/597048 | 07:10 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add config-errors notifications drawer https://review.openstack.org/597147 | 07:10 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add change status page https://review.openstack.org/599472 | 07:11 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add project page https://review.openstack.org/604266 | 07:11 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add labels page https://review.openstack.org/604682 | 07:11 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add nodes page https://review.openstack.org/604683 | 07:11 |
| *** chkumar|off is now known as chandankumar | 07:18 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool master: Implement a Kubernetes driver https://review.openstack.org/535557 | 07:19 |
| *** quiquell|brb is now known as quiquell | 07:38 | |
| *** jpena|off is now known as jpena | 08:01 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: web: add tenant and project scoped, JWT-protected actions https://review.openstack.org/576907 | 08:04 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: CLI: add create-web-token command https://review.openstack.org/605386 | 08:09 |
| *** jiapei has quit IRC | 08:31 | |
| *** panda|off is now known as panda | 09:00 | |
| *** electrofelix has joined #zuul | 09:05 | |
| *** jpena is now known as jpena|lunch | 11:06 | |
| *** quiquell is now known as quiquell|lunch | 11:27 | |
| *** ssbarnea|bkp has joined #zuul | 11:37 | |
| *** quiquell|lunch is now known as quiquell | 11:43 | |
| *** jimi_|ansible is now known as jimi|ansible | 11:43 | |
| *** pcaruana has quit IRC | 11:50 | |
| *** rfolco has joined #zuul | 11:56 | |
| openstackgerrit | Simon Westphahl proposed openstack-infra/zuul master: Use merger to get list of files for pull-request https://review.openstack.org/603287 | 12:00 |
| openstackgerrit | Simon Westphahl proposed openstack-infra/zuul master: Use merger to get list of files for pull-request https://review.openstack.org/603287 | 12:13 |
| *** rlandy has joined #zuul | 12:22 | |
| *** jpena|lunch is now known as jpena | 12:34 | |
| *** pcaruana has joined #zuul | 12:39 | |
| *** samccann has joined #zuul | 13:17 | |
| corvus | tobiash: are you planning to look at https://review.openstack.org/591604 again? | 14:37 |
| tobiash | corvus: I'm not finished with my local tests yet, but I'm fine with fixing any findings afterwards | 14:38 |
| tobiash | E.g. opening live logs in a new tab didn't work | 14:39 |
| tobiash | But no idea yet if that's because of the deployment or the rewrite | 14:40 |
| corvus | i take it the test_streaming failures are due to the cherrypy 17.x release mentioned in 601136? | 14:44 |
| *** annabelleB has joined #zuul | 14:58 | |
| *** annabelleB has quit IRC | 15:21 | |
| *** quiquell is now known as quiquell|off | 15:28 | |
| *** annabelleB has joined #zuul | 15:39 | |
| *** chandankumar is now known as chkumar|off | 15:46 | |
| *** annabelleB has quit IRC | 15:47 | |
| openstackgerrit | Merged openstack-infra/zuul master: Uncap cherrypy https://review.openstack.org/601136 | 16:33 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: WIP: Playing with k8s install https://review.openstack.org/605803 | 16:34 |
| tobiash | corvus: yes I think so | 16:35 |
| Shrews | tristanC: 605803 ^^ is me playing around with the idea I floated in your k8s driver change. Not sure how far I can get with it | 16:36 |
| corvus | Shrews: cool -- when we get that going, we should make it a role in zuul-jobs | 16:38 |
| corvus | it's going to be a pretty swell thing when we can say "roles: [devstack, kubernetes]" in a playbook and just have those things for jobs to use :) | 16:39 |
| Shrews | corvus: yeah, i was disappointed to not find one there already :( | 16:39 |
| Shrews | corvus: not sure i'm qualified to do the k8s config portion of it. tristanC can help there i think | 16:40 |
| corvus | we should move the current nodepool deployment out of being a devstack plugin into ansible role/playbook | 16:41 |
| corvus | then we can use the same "deploy nodepool" bits in the devstack functional job, and the k8s functional job | 16:41 |
| corvus | clarkb: ^ cc | 16:41 |
| AJaeger | corvus, clarkb, should we recommend ZUUL as envvar for the kata job - or give them a better guidance? See https://review.openstack.org/605773 | 16:42 |
| clarkb | AJaeger: I think that is fine for individual jobs to do | 16:43 |
| corvus | AJaeger: yeah totally up to them. | 16:43 |
| pabelanger | jlk: with github3.py, have you used it with a netrc file for github credentials? Just looking into it now and don't see much info | 16:44 |
| clarkb | corvus: ya that should work re refactoring deploy nodepool bits | 16:45 |
| tobiash | mordred: update on the segfault issue: the kernel upgrade didn't help so we will switch to ubuntu based executors and see if that fixes it | 16:45 |
| jlk | pabelanger: I don't believe I have | 16:47 |
| jlk | pabelanger: there is a test case covering it which generates a test netrc file | 16:48 |
| pabelanger | jlk: okay, cool. let me look at that | 16:49 |
| jlk | the test is a bit odd. It's testing that IF a token exists, that it DOESNT use netrc file | 16:49 |
| jlk | I'm gathering netrc is a python requests level thing that should Just Work(tm) | 16:49 |
| pabelanger | yah, I believe so | 16:50 |
| *** jpena is now known as jpena|off | 17:11 | |
| pabelanger | jlk: okay, I think I'd need to refactor github3.py login a bit to support it. I'll use username / password for now to move forward and work on that in the future | 17:15 |
| jlk | cool | 17:15 |
| *** panda is now known as panda|off | 17:46 | |
| corvus | Shrews, tristanC: left a few comments on 535557 -- most are doc nits (i'm happy to help with that but had a few questions), but i do think we want to work through what data we're passing back to zuul and how it gets used a little more. | 17:56 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul-jobs master: WIP: Add role to install kubernetes https://review.openstack.org/605823 | 17:59 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Implement a Kubernetes driver https://review.openstack.org/535557 | 18:00 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: WIP: Playing with k8s install https://review.openstack.org/605803 | 18:00 |
| *** sshnaidm is now known as sshnaidm|off | 18:03 | |
| Shrews | corvus: tristanC: i think we need something like minikube installed (https://kubernetes.io/docs/tasks/tools/install-minikube/) for us to have a nodepool k8s functional test. That seems to be a devstack equivalent. Is that the route we should take? | 18:05 |
| Shrews | because my role to install k8s works from me locally, but complains because there is no cluster (provided by minikube) | 18:06 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Fix unreachable nodes detection https://review.openstack.org/602829 | 18:07 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Also retry the job if a post job failed with unreachable https://review.openstack.org/602830 | 18:07 |
| corvus | clarkb, pabelanger: ^ you've mentioned minikube in the past | 18:10 |
| pabelanger | corvus: I've used kubespray in the past | 18:10 |
| clarkb | minikube is the upstream k8s devstack like thing | 18:11 |
| pabelanger | I think that is more production then dev | 18:11 |
| clarkb | it would probably work well for this use case | 18:11 |
| pabelanger | https://review.openstack.org/#/c/549768/ | 18:11 |
| pabelanger | was a WIP I did a while back | 18:11 |
| pabelanger | but, don't have real preference personally | 18:11 |
| Shrews | would we want minikube installation to be a part of the install-kubernetes role? Installation of that perhaps controlled by a variable? | 18:13 |
| Shrews | or an entirely separate role? | 18:14 |
| pabelanger | if we want a separate role, maybe we should just use kubespray, and contribute to that | 18:16 |
| clarkb | I would expect install kubernetes to install a cluster | 18:16 |
| corvus | pabelanger: your kubespray playbook runs ansible; is it runnable by zuul directly? | 18:20 |
| pabelanger | corvus: I think I tried, but had issues with ansible 2.3 version | 18:22 |
| pabelanger | but we should try again to have zuul do it | 18:22 |
| pabelanger | the easy part, is it did just work | 18:22 |
| corvus | may be worth a quick test to see if a role with include_playbook would work | 18:23 |
| corvus | otherwise, minikube sounds simple :) | 18:24 |
| pabelanger | ansible 2.4 was min requirement | 18:24 |
| pabelanger | I can try again later this evening and report back | 18:24 |
| pabelanger | but yah, don't want to block if minikube is working now | 18:25 |
| Shrews | after a cursory glance, seems like maybe minikube is a bit more self-contained? | 18:25 |
| Shrews | less setup and whatnot | 18:25 |
| pabelanger | yah, kubespray supports a lot of different backends (networking) | 18:26 |
| corvus | simple is better than complete for this i think | 18:28 |
| Shrews | i guess the most flexible way to handle this would be the install-kubernetes role having 2 vars: an "install-cluster" boolean, and a "cluster-type" that defaults to "minikube", but could later add support for "kubespray" | 18:29 |
| Shrews | but there i go thinking too far ahead again :) | 18:30 |
| pabelanger | Shrews: yah, I think if minikube gets us testing now for nodepool, and we can merge the driver. I think that is a bigger win, then how we install k8s | 18:43 |
| pabelanger | +1 to minikube | 18:43 |
| Shrews | hrm, i just noticed how zuul-jobs roles are very debian OS specific | 19:42 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Provide some accounting of node usage in logs https://review.openstack.org/605856 | 19:52 |
| *** pcaruana has quit IRC | 19:54 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Provide some accounting of node usage in logs https://review.openstack.org/605856 | 19:57 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Indicate whether a build is voting in the logs https://review.openstack.org/605857 | 19:59 |
| mordred | Shrews: yah - they tend to get written for ubuntu until someone decides to add support for something else | 20:07 |
| *** goern has joined #zuul | 20:08 | |
| mnaser | can i throw an idea over the fence and run away? :> | 20:15 |
| mordred | mnaser: I don't know that we can stop you :) | 20:28 |
| mnaser | adding scheduling to nodepool in terms of time | 20:28 |
| mnaser | aka max-servers: 300 at night but max-servers: 50 at the day or so | 20:29 |
| mnaser | i mean.. sometimes i think i can do this at our side just by playing with quotas.. but yeah | 20:29 |
| pabelanger | mnaser: I'd use the feature, we are stilling how we could do that now for billing reasons | 20:30 |
| mnaser | i think in openstack-infra, providers might be able to be more flexible | 20:31 |
| pabelanger | for now, I'm thinking of some sort of automation to change max-server via cfgmt | 20:31 |
| mordred | mnaser: that's an interesting thought - I'm guessing that some of the providers (such as you) might be more willing/able to donate more things at some times vs. others? | 20:31 |
| mnaser | yeah, weekends are slow, nights are slow | 20:31 |
| mnaser | it'd help with flush out the queues at night | 20:31 |
| mordred | yah - for times like this when we're just super behind | 20:31 |
| mnaser | yep | 20:31 |
| *** samccann has quit IRC | 20:32 | |
| pabelanger | on the cloud side, some sort of automated quota increase could also work | 20:32 |
| pabelanger | max-server: 300, but when quota lowered, nodepool only launcher what it can | 20:33 |
| mnaser | yeah i figured we could do that but i dunno, feels hacky. | 20:37 |
| Shrews | mnaser: tobiash does something similar in his environment. might want to chat with him | 20:41 |
| Shrews | but opposite... ramps down at night. maybe using max-ready-age? | 20:42 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-sphinx master: Add attr_overview directive https://review.openstack.org/604980 | 20:43 |
| mordred | Shrews: yah - I believe that's also about controlling min-ready | 20:45 |
| corvus | i think either approach would be useful -- nodepool groks quotas as reported by providers now, so a provider changing quota behind the scenes seems like a legitimate way to accomplish that. but also, doing the same in nodepool could be useful for folks who don't control the environment. | 20:45 |
| mordred | corvus: ++ | 20:45 |
| mordred | it sounds like between mnaser's suggestion and what Shrews reminds us tobiash is doing - the theme would be about attaching temporality to all of the quota-related settings - max-ready-age, min-ready max-servers | 20:46 |
| corvus | yeah, though we may want to support setting max cpu/mem/etc as well, since that may be a more important limiting factor in some cases (and we've moved beyond just "servers" being the only quota metric) | 20:49 |
| pabelanger | and volumes! | 20:50 |
| pabelanger | totally agree | 20:50 |
| mordred | ++ | 20:51 |
| *** rlandy is now known as rlandy|biab | 22:03 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Add nodesets API route https://review.openstack.org/605877 | 22:34 |
| corvus | mordred: you based that on a series of changes which uses the zuul api to export nodepool information... maybe we'll do that, but we might need to talk about it a bit more and it's an added complication your patch doesn't need. | 23:01 |
| corvus | though i've just +2d them. but before approval, i think i'd like to ask mordred, clarkb, and Shrews whether that sounds like a good idea, or a bridge too far. | 23:06 |
| *** rlandy|biab is now known as rlandy | 23:57 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!