| SpamapS | Cool I think that one works for me. | 00:03 |
|---|---|---|
| SpamapS | clarkb: added myself | 00:04 |
| *** zhuli_ has joined #zuul | 00:15 | |
| *** ssbarnea_ has quit IRC | 00:28 | |
| *** gouthamr has quit IRC | 00:30 | |
| *** dmellado has quit IRC | 00:31 | |
| *** gouthamr has joined #zuul | 00:38 | |
| *** dmellado has joined #zuul | 00:38 | |
| *** gouthamr has quit IRC | 00:58 | |
| *** dmellado has quit IRC | 00:59 | |
| *** elyezer has quit IRC | 01:12 | |
| *** spsurya has joined #zuul | 01:14 | |
| *** zhuli_ is now known as zhuli | 01:21 | |
| *** elyezer has joined #zuul | 01:22 | |
| *** gouthamr has joined #zuul | 01:23 | |
| *** dmellado has joined #zuul | 01:26 | |
| *** gouthamr has quit IRC | 01:32 | |
| *** dmellado has quit IRC | 01:32 | |
| *** rlandy|bbl is now known as rlandy | 01:45 | |
| *** gouthamr has joined #zuul | 01:56 | |
| *** dmellado has joined #zuul | 01:56 | |
| *** dmellado has quit IRC | 02:09 | |
| *** dmellado has joined #zuul | 02:13 | |
| *** gouthamr has quit IRC | 02:22 | |
| *** gouthamr has joined #zuul | 02:23 | |
| *** gouthamr has quit IRC | 02:27 | |
| *** gouthamr has joined #zuul | 02:32 | |
| *** gouthamr has quit IRC | 02:37 | |
| *** dmellado has quit IRC | 02:38 | |
| *** gouthamr has joined #zuul | 03:38 | |
| *** dmellado has joined #zuul | 03:39 | |
| *** gouthamr has quit IRC | 03:52 | |
| *** dmellado has quit IRC | 04:04 | |
| *** dmellado has joined #zuul | 04:39 | |
| *** gouthamr has joined #zuul | 04:40 | |
| *** gouthamr has quit IRC | 04:49 | |
| *** gouthamr has joined #zuul | 04:50 | |
| *** gouthamr has quit IRC | 04:56 | |
| *** ianw has quit IRC | 05:06 | |
| *** patrickeast has quit IRC | 05:15 | |
| *** sdake has quit IRC | 05:15 | |
| *** TheJulia has quit IRC | 05:15 | |
| *** abelur has quit IRC | 05:15 | |
| *** hogepodge has quit IRC | 05:15 | |
| *** sdake has joined #zuul | 05:15 | |
| *** sdake has joined #zuul | 05:15 | |
| *** abelur has joined #zuul | 05:15 | |
| *** TheJulia has joined #zuul | 05:16 | |
| *** hogepodge has joined #zuul | 05:16 | |
| *** threestrands has quit IRC | 05:47 | |
| *** ianw has joined #zuul | 05:55 | |
| *** nguyenhai has joined #zuul | 06:12 | |
| *** nguyenhai_ has quit IRC | 06:16 | |
| *** hashar has joined #zuul | 06:28 | |
| *** dmsimard has quit IRC | 06:39 | |
| *** dmsimard has joined #zuul | 06:40 | |
| *** yolanda_ has joined #zuul | 06:42 | |
| *** yolanda has quit IRC | 06:45 | |
| *** dmellado has quit IRC | 07:01 | |
| *** dmellado_ has joined #zuul | 07:24 | |
| *** ssbarnea_ has joined #zuul | 07:25 | |
| *** dmellado_ has quit IRC | 07:34 | |
| *** jpena|off is now known as jpena | 07:48 | |
| *** gtema has joined #zuul | 07:59 | |
| *** ssbarnea_ has quit IRC | 07:59 | |
| *** dmellado_ has joined #zuul | 08:01 | |
| *** dmellado_ has quit IRC | 08:11 | |
| *** jimi|ansible has quit IRC | 08:36 | |
| *** dmellado_ has joined #zuul | 08:50 | |
| *** nguyenhai has quit IRC | 08:54 | |
| *** ssbarnea_ has joined #zuul | 09:18 | |
| gtema | during zuul job execution I restarted the zuul-scheduler. This resulted in "lost" job and github branch/PR stucked in the pending state. Is it intended or have I configured something wrong? | 09:34 |
| tobiash | gtema: that's expected as zuul reports pending at start and reports success/failure at end | 10:16 |
| tobiash | but you can just rerun your job and everything should be ok | 10:16 |
| gtema | tobiash: it is more about usability. I.e. as an admin of a big installation I do not know which jobs are lost, but the user does not have nice interface to restart job without sending "empty commit" | 10:18 |
| gtema | tobiash: with PR you can of course make comment "recheck", but what with regular branch check job? | 10:19 |
| tobiash | gtema: you can also add a comment trigger so the user can retrigger the job via a comment | 10:19 |
| tobiash | however branch triggered pipelines are a problem in that regard | 10:19 |
| gtema | tobiash: yes, exactly | 10:19 |
| tobiash | gtema: what's on the long term roadmap is that zuul will store its state in zookeeper in order to allow running multiple schedulers at the same time | 10:21 |
| gtema | tobiash: it is nice in travis and co, that you can restart job from UI, but in zuul it is not possible. And even as an admin it is not that user-friendly | 10:21 |
| gtema | tobiash: nice to hear | 10:21 |
| gtema | tobiash: I basically faced similar problem, when job fails due to the config problems - no easy way to restart | 10:22 |
| tristanC | fwiw, "restart job from UI" is something mhu proposed a spec for: https://review.openstack.org/#/c/562321 | 10:39 |
| gtema | tristianC: thanks - fits | 10:39 |
| tristanC | gtema: also there is a zuul-changes.py script you can use to dump the current queue before restarting the scheduler | 10:42 |
| gtema | tristianC: ok, this will however mean I need to rethink deployment. With pip (and ansible-role-zuul) tools are not available on the server, which actually should be, as even encrypt is not available this way | 10:45 |
| *** spsurya has quit IRC | 10:56 | |
| *** jpena is now known as jpena|lunch | 12:03 | |
| *** jimi|ansible has joined #zuul | 12:14 | |
| *** ssbarnea_ has quit IRC | 12:16 | |
| *** electrofelix has quit IRC | 12:26 | |
| *** ssbarnea_ has joined #zuul | 12:33 | |
| *** rlandy has joined #zuul | 12:34 | |
| *** elyezer has quit IRC | 12:40 | |
| *** dkranz has joined #zuul | 12:43 | |
| *** ssbarnea_ has quit IRC | 12:49 | |
| *** elyezer has joined #zuul | 12:52 | |
| *** jpena|lunch is now known as jpena | 12:54 | |
| *** ssbarnea_ has joined #zuul | 12:57 | |
| *** pwhalen has quit IRC | 13:06 | |
| *** pwhalen has joined #zuul | 13:10 | |
| *** pwhalen has joined #zuul | 13:10 | |
| *** pcaruana has joined #zuul | 13:17 | |
| *** gouthamr has joined #zuul | 13:30 | |
| pabelanger | tobiash: clarkb: mordred: looks like there is another breakage with os-client-config: http://logs.openstack.org/58/566158/1/check/nodepool-functional-py35/45d52ed/controller/logs/screen-nodepool-launcher.txt.gz#_May_04_05_56_27_378446 | 13:49 |
| pabelanger | cloud_config.get_cache_expiration() | 13:49 |
| pabelanger | ah, was reported in openstack-infra: https://bugs.launchpad.net/os-client-config/+bug/1768813 | 13:49 |
| openstack | Launchpad bug 1768813 in os-client-config "config.get_cache_expiration_time() function missing in 1.31.0 release" [Undecided,New] | 13:49 |
| pabelanger | Shrews: thanks! | 13:49 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: WIP: Add fedora mirror to nodepool dsvm jobs https://review.openstack.org/566102 | 13:58 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Add fedora mirror to nodepool dsvm jobs https://review.openstack.org/566102 | 13:59 |
| openstackgerrit | Merged openstack-infra/nodepool master: Fix test patching of clouds.yaml file locations https://review.openstack.org/566138 | 14:07 |
| *** Guest46098 has quit IRC | 14:13 | |
| *** Guest46098 has joined #zuul | 14:14 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Remove debian-jessie from nodepool dsvm testing https://review.openstack.org/566327 | 14:26 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Add fedora mirror to nodepool dsvm jobs https://review.openstack.org/566102 | 14:28 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Add fedora-28 to nodepool dsvm https://review.openstack.org/559211 | 14:28 |
| *** ssbarnea_ has quit IRC | 14:35 | |
| *** trishnag has quit IRC | 14:43 | |
| *** trishnag has joined #zuul | 14:47 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul master: Add tox-py36 jobs https://review.openstack.org/565881 | 14:48 |
| pabelanger | clarkb: okay, ^ should fix up bindep for bubblewrap now | 14:49 |
| *** ssbarnea_ has joined #zuul | 14:51 | |
| *** dmellado_ is now known as dmellado | 14:57 | |
| *** pcaruana has quit IRC | 15:13 | |
| *** acozine has joined #zuul | 15:33 | |
| *** hashar is now known as hasharAway | 15:36 | |
| corvus | fdegir: when you have a sec, can i ask you more about your static node situation? | 15:38 |
| fdegir | corvus: yes - I'm at the airport and will try to respond | 15:39 |
| fdegir | corvus: was typing an answer to your mail but got interrupted | 15:40 |
| fdegir | corvus: if you want, i can complete my response and send the mail which we can continue chatting here if you want | 15:40 |
| corvus | fdegir: that sounds great, and no rush :) | 15:41 |
| corvus | don't miss your flight :) | 15:41 |
| fdegir | corvus: sent the mail | 15:55 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Add fedora-28 to nodepool dsvm https://review.openstack.org/559211 | 15:57 |
| corvus | fdegir: what's a POD? | 16:01 |
| fdegir | corvus: pool of devices | 16:02 |
| fdegir | corvus: in OPNFV, we have an official spec for the PODs we use | 16:03 |
| fdegir | corvus: 1 + 5 baremetal nodes | 16:03 |
| clarkb | pabelanger: +2 thanks. I like that simplification | 16:03 |
| fdegir | corvus: the 1 is the jumphost which serves as the entry point to a certain POD | 16:03 |
| fdegir | corvus: where we drive the deployment towards the remaining 5 nodes | 16:03 |
| fdegir | corvus: the 3 of 5 nodes act as controllers | 16:04 |
| fdegir | corvus: and the last 2 are compute nodes | 16:04 |
| corvus | fdegir: are those 1+5 always used together? | 16:04 |
| fdegir | corvus: yes | 16:04 |
| fdegir | corvus: I mean the jumphost gets connected to jenkins as slave | 16:04 |
| fdegir | corvus: and the remaining 5 has nothing to do with jenkins | 16:04 |
| corvus | fdegir: so, using nodepool, you might only register the jumphost | 16:04 |
| fdegir | corvus: yes | 16:04 |
| fdegir | corvus: when you initiate the deployment, the installers pxe boot the remaining 5 nodes and provision them with whatever OS they support | 16:05 |
| fdegir | which could be ubuntu, centos, or opensuse | 16:05 |
| fdegir | corvus: after the 5 nodes are provisioned, installation procedure starts with installing packages (if needed), doing network configuration, etc. etc. | 16:05 |
| fdegir | corvus: after that, OpenStack gets deployed on the nodes and the services are installed on them depending on what role they have | 16:06 |
| fdegir | corvus: the important thing for nodepool is the jumphost - that's the OS I was talking about in my response | 16:06 |
| fdegir | corvus: for example, we have tripleo which only supports Centos | 16:06 |
| fdegir | corvus: we have juju, it works with ubuntu | 16:07 |
| fdegir | corvus: openstack ansible works with all 3 | 16:07 |
| corvus | fdegir: so you might register a "centos" jumphost, and an "ubuntu" jumphost. a tripelo job would request a centos jumphost, and for ansible, you'd have two jobs: centos job and an ubuntu job, to make sure you tested both? | 16:08 |
| fdegir | corvus: yes | 16:08 |
| fdegir | corvus: for the distro part | 16:09 |
| fdegir | corvus: and then we have sriov for tripleo and joid for example | 16:09 |
| *** EmilienM is now known as EvilienM | 16:09 | |
| fdegir | corvus: in this case, tripleo needs a node with centos + sriov and joid needs ubuntu + sriov | 16:10 |
| fdegir | joid=juju | 16:10 |
| corvus | fdegir: you said sriov on both of those nodes, is that a mistake? | 16:11 |
| fdegir | corvus: no, you could have 2 PODs supporting sriov | 16:11 |
| fdegir | corvus: but their jumphosts my have different OS | 16:11 |
| corvus | fdegir: will you have any pods withou sriov? | 16:11 |
| fdegir | corvus: we have that too | 16:11 |
| fdegir | corvus: i didn't even mention other networking configuration or traffic generators etc. etc. | 16:12 |
| fdegir | I mean I don't expect all the different use cases to be covered | 16:12 |
| corvus | fdegir: so you will have some jobs that will request "ubuntu+sriov", and other jobs which will request "ubuntu" and they don't care whether sriov is there or not? | 16:12 |
| fdegir | corvus: that is true | 16:12 |
| fdegir | but | 16:12 |
| fdegir | corvus: we really don't want to waste resources with special hw/capabilities for basic deployments | 16:13 |
| fdegir | because they are very few in number | 16:13 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Fix typo with _logConsole function https://review.openstack.org/566356 | 16:13 |
| fdegir | so this is the first part - the deployment | 16:14 |
| fdegir | the other thing i mentioned above - traffic generators as well | 16:14 |
| fdegir | the testing we do also contribute to number of combinations we might end up | 16:14 |
| pabelanger | clarkb: Shrews: tristanC: ^fixes a typo with recent refactor of nodepool-launcher | 16:15 |
| corvus | fdegir: are the traffic generators part of the pod as well, or can they be a separate resource? | 16:15 |
| fdegir | liek we shouldn't really run the test job that tests specific hw on a deployment that doesn't use any of the specific stuff | 16:15 |
| fdegir | corvus: they are not visible from jenkins/nodepool | 16:15 |
| fdegir | corvus: we just know they are attached to pod network | 16:15 |
| fdegir | corvus: which might be included in pod descriptor file | 16:16 |
| fdegir | i don't remember this part exactly | 16:16 |
| corvus | fdegir: what's an example of a set of labels that you might want to give a single node registered in nodepool? | 16:16 |
| fdegir | it could be like ubuntu + sriov, centos + ovsdpdk | 16:16 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Remove debian-jessie from nodepool dsvm testing https://review.openstack.org/566327 | 16:17 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Add fedora mirror to nodepool dsvm jobs https://review.openstack.org/566102 | 16:17 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Add fedora-28 to nodepool dsvm https://review.openstack.org/559211 | 16:17 |
| fdegir | when it comes to testing, i can think of something like centos + sriov + trafficgen | 16:17 |
| corvus | fdegir: well, the same node wouldn't be both ubuntu and centos? | 16:17 |
| fdegir | corvus: not at the moment | 16:17 |
| fdegir | corvus: the jumphosts are static - we don't (re)provision them | 16:17 |
| fdegir | this is in our roadmap - make everything dynamic so we remove the OS dimension and provision jumphosts on demand | 16:18 |
| fdegir | or | 16:18 |
| fdegir | isolate installers from jumphosts by moving into VMs or containers | 16:18 |
| corvus | fdegir: i think i understand a lot about what you're saying, but i still don't understand enough to construct a set of labels for a single node. like, i know that you are saying you'll have a job that requires centos+sriov+trafficgen, so therefore, you probably want a node with a label "centos_sriov_trafficgen". but what other labels would it be useful for the same node to have? | 16:19 |
| fdegir | but since all the installers we have come from upstream openstack, that's not an easy thing to achieve | 16:19 |
| fdegir | those 3 are separate labels - not combined | 16:20 |
| fdegir | another thing we do is, not all the pods are ci pods | 16:20 |
| corvus | fdegir: zuul only knows how to request a single label for a given node. | 16:20 |
| fdegir | meaning that some member companies donate pods but they are donated for development work | 16:21 |
| fdegir | which we can't run ci on them and need to ensure jobs don't end them | 16:21 |
| fdegir | so these don't get ci-pod labels | 16:21 |
| fdegir | corvus: ok | 16:21 |
| fdegir | corvus: but then what was the nodepool multi-label feature about? | 16:21 |
| corvus | fdegir: so when writing a job in zuul, if you need centos+sriov+trafficgen, you'd have to have a label which means "this node has centos and sriov and trafficgen" which is why i speculated about the combined label centos_sriov_trafficgen | 16:21 |
| fdegir | corvus: ok - i thought having nodepool multi-label means zuul can request nodes using this feature as well | 16:22 |
| corvus | fdegir: it's about having a single node in nodepool with multiple labels, indicating it can satisfy different kinds of requests | 16:22 |
| fdegir | corvus: ok - now i think i realize the mistake I made | 16:23 |
| corvus | fdegir: unfortunately, part of the problem is that this feature snuck in without very much thought and doesn't really fit the architecture of the system. that's part of why we're trying to remove it and/or redesign it to fit | 16:23 |
| fdegir | corvus: since we don't have dynamic pod assignment at the moment, we haven't really had the need to assign nodes using multiple labels | 16:23 |
| fdegir | corvus: i was talking about a future need but thinking a bit more about jenkins, i believe it doesn't support it either | 16:24 |
| corvus | fdegir: jenkins does let you give nodes multiple labels, then lets you specify things like "foo && bar", so you'd get a slave with both the foo and bar labels | 16:25 |
| fdegir | corvus: ok - i didn't know that because we haven't tried it yet | 16:25 |
| fdegir | corvus: so that's what we will need :) | 16:25 |
| corvus | fdegir: i think you would need that if you need to speficy centos+sriov+trafficgen as well as centos+sriov | 16:25 |
| corvus | fdegir: but if you don't have any jobs which want centos+sriov and don't care about trafficgen, then it's not necessary | 16:26 |
| fdegir | corvus: we have those | 16:26 |
| fdegir | corvus: we have different test levels - functional and benchmarking for example | 16:26 |
| fdegir | corvus: these have test cases for sriov | 16:26 |
| fdegir | corvus: on top of that, we have long duration testing which makes use of traffic gen | 16:27 |
| fdegir | corvus: so same pod can serve both functional+benchmarking and long duration testing on different times | 16:27 |
| corvus | fdegir: and you also have centos+sriov pods without trafficgen? | 16:28 |
| corvus | fdegir: (i ask because if all the centos+sriov pods do have trafficgen, then they're basically the same and trafficgen doesn't need to be a label) | 16:30 |
| fdegir | corvus: i think we do | 16:31 |
| fdegir | the hw we have not homogenous | 16:32 |
| corvus | fdegir: okay. basically, in nodepool, a label uniquely identifies a resource; so if a resource only needs to be addresed by one set of criteria (A+B+C) then it's okay for it to only have one label (A+B+C). it really only needs more than one label if you need to address it by multiple criteria (A+B+C in some cases, just A in others, A+B in others, etc) | 16:33 |
| clarkb | jumping in with an idea maybe nodepool continues to uniquely label resources but jobs can specify either or | 16:34 |
| clarkb | so resource A has label X+Y+Z and resource B has X+Y label. Then in the job you say labels: - X+Y+Z - X+Y ? | 16:35 |
| corvus | fdegir: it sounds like most of the time, specifying a node by A+B+C is sufficient, but maybe sometimes you'll want to use A+B or just A; in those cases, you could still request A+B+C, you would just be overspecifying. | 16:35 |
| clarkb | I don't know if that is easier to accomodate | 16:35 |
| fdegir | corvus: right | 16:35 |
| corvus | clarkb: that may work; we should think about it. | 16:36 |
| fdegir | clarkb: but if you have a label to uniquely identify a resource | 16:36 |
| fdegir | clarkb: we qlready have that with node names that are unique | 16:37 |
| mordred | corvus: a similar usecase - based on what clarkb said - are the openstack tox jobs. they're currently designed to use ubuntu-xenial - but there is nothing about them that would not work on fedora-28 - if nodepool can provide both ubuntu-xenail and fedora-28, we COULD (not saying we should) say that tox jobs can run on ubuntu-xenial or fedora-28 and who cares | 16:37 |
| fdegir | that’s a case we have too | 16:38 |
| fdegir | general pool of PODs | 16:38 |
| mordred | but I don't know if that would make the system better or worse to use overall | 16:38 |
| corvus | fdegir: well, you could still have multiple A+B+C pods, so even though they have unique hostnames, specifying them by label lets you use all of them. | 16:38 |
| fdegir | and in openstack ansible case, it doesn’t care about the OS of the junphost | 16:38 |
| fdegir | just to reiterate again - i just wanted to highlight something we will need but it doesn’t mean it has to be supported | 16:39 |
| fdegir | but we might bot be the only ones with similar needs | 16:39 |
| fdegir | not* | 16:39 |
| corvus | fdegir: this is good, i like working with actual use cases :) | 16:39 |
| corvus | fdegir: i want to explore a different line of thought for a moment though | 16:40 |
| mordred | yah. actually use cases are better than theoretical ones | 16:40 |
| corvus | we've been thinking about this in jenkins terms | 16:40 |
| fdegir | if you want more, i can bring more | 16:40 |
| fdegir | you know, telco... | 16:40 |
| corvus | nodepool actually behaves a little differently, and it's interesting that you use the term "pool" | 16:40 |
| corvus | because nodepool does have an idea of a pool, and it's not that different from your POD | 16:40 |
| corvus | instead of only registering the jumphost in nodepool, you could consider registering all of the hosts/resources | 16:41 |
| fdegir | yes - when i say pool, it is group of nodes that are pooled together with the use of labels on jenkins | 16:41 |
| corvus | then instead of requesting a "centos+sriov+traficgen" node, which happens to come with 5 other nodes, you can request: 1 centos jumphost, 5 nodes, 1 traffic generator | 16:42 |
| fdegir | that wouldn’t work... | 16:42 |
| corvus | nodepool will fulfill that request only if there is a pool that has at least 1 centos jumphost, 5 nodes, and 1 traffic generator | 16:43 |
| corvus | which means that if you have another pool with a centos jumphost and no traffic generator, it won't handle the request. | 16:43 |
| fdegir | that might work - sorry for junpinh | 16:43 |
| * Shrews catches up on scrollback from lunch | 16:43 | |
| fdegir | i think this might work | 16:43 |
| mordred | corvus: ooh. I like where you're going with that | 16:43 |
| fdegir | corvus: would it be possible to ensure zuul runs things on jumphost still? | 16:44 |
| fdegir | not on any others randomly | 16:45 |
| corvus | fdegir: yes, all the nodes will be added to the ansible inventory, but you write playbooks which specify which nodes things run on, so you can just say "host: jumphost" in a playbook and it will only run there | 16:45 |
| fdegir | right | 16:46 |
| corvus | fdegir: (this is actually how some of our new v3 multinode jobs work in openstack) | 16:46 |
| clarkb | and if you want to be extra careful you can have an early task that breaks ssh from executor to those nodes and only allow it from the jumphost | 16:46 |
| fdegir | corvus: so the example you’ve given | 16:46 |
| clarkb | (not necessary though) | 16:46 |
| fdegir | corvus: does the nodepool need to connect to all of those? | 16:46 |
| clarkb | fdegir: as currently implemented it needs to be able to do an ssh keyscan but I think pabelanger has added in an option to not do that? | 16:47 |
| fdegir | corvus: or just some kind of pool template and only try to access to the one we specify as host: jumphost | 16:47 |
| corvus | fdegir: hrm, currently i think it does; and also the zuul executor. | 16:47 |
| fdegir | clarkb: corvus: the thing is, we don’t realy deal with and guarantee the target nodes will have os on them | 16:48 |
| fdegir | only the jumphost is guaranteed to have an os and confugured | 16:48 |
| mordred | if we set the node connection type to something other that ssh | 16:48 |
| fdegir | the rest of them are simply handled by installers | 16:48 |
| mordred | then neither the executor nor nodepool will try to ssh to them | 16:48 |
| fdegir | that works | 16:49 |
| corvus | mordred: i think there's a whitelist, so we'd have to add "null" or something | 16:49 |
| pabelanger | host-key-checking is the setting in nodepool | 16:49 |
| mordred | yah | 16:49 |
| pabelanger | https://zuul-ci.org/docs/nodepool/configuration.html#pools | 16:49 |
| corvus | we could do that, or, it's possible the "only register the jumphost" is the better model for some circumstances | 16:49 |
| mordred | corvus: which would be pretty easy to do ... and incidentally would prevent accidental use of the nodes from the executor | 16:49 |
| mordred | corvus: beause it would put ansible_connection_plugin: null in the inventory - and if you tried to use that, given that such a plugin doesn't exist, you'd get boom | 16:50 |
| corvus | mordred: well, we need to not break the setup task | 16:50 |
| mordred | yah. that's where we'd need the whitelist I think - because we already preventit from running setup if connection type is not supported | 16:51 |
| corvus | mordred: heh, the setup task has it's own inventory | 16:51 |
| mordred | that too | 16:51 |
| corvus | mordred: so we're good, actually. we just add 'null' to the blacklist | 16:51 |
| mordred | because ansible-network needs us to not run setup | 16:51 |
| mordred | corvus: ++ | 16:51 |
| corvus | BLACKLISTED_ANSIBLE_CONNECTION_TYPES = ['network_cli'] | 16:51 |
| corvus | that's the current value | 16:51 |
| mordred | yup | 16:51 |
| clarkb | blacklist or whitelist? | 16:51 |
| mordred | I like 'null' | 16:52 |
| mordred | clarkb: it's a blacklist - because ansible -msetup works for most things ... but not for network_cli | 16:52 |
| clarkb | ah | 16:52 |
| clarkb | we also have to separately whitelist it as an allowed connection type right? | 16:52 |
| mordred | so adding null to that would allow registering nodes in nodepool that you don't want ansible to try to talk to | 16:52 |
| mordred | clarkb: we prevent people from setting connection type | 16:52 |
| mordred | in playbooks | 16:53 |
| clarkb | I think we whitelist them as part of tobiash security updates | 16:53 |
| corvus | right, but in nodepool, we allow the admin to specify the connection type, and i think it is a whitelist | 16:53 |
| mordred | ah | 16:53 |
| fdegir | sorry, boarding | 16:54 |
| fdegir | before i lose the connection | 16:54 |
| corvus | fdegir: thanks! | 16:54 |
| fdegir | we have something else which i don’t remember i talked about in dublin during ptg | 16:55 |
| fdegir | i think i mentioned this to rcarrillocruz | 16:55 |
| gtema | btw, about nodepool + shade. What is the expectation from interface_ip in a public cloud without attached floating_ip? In my case it is empty and nodepool-launcher fails | 16:55 |
| *** jpena is now known as jpena|off | 16:55 | |
| fdegir | so, not all the nodes are accessible via ssh | 16:55 |
| fdegir | on jenkins, we use jnlp so the nodes call jenkins | 16:56 |
| fdegir | i don’t know if you heard about this | 16:56 |
| corvus | in summary: i think there's some interesting new features in nodepool that may help with fdegir's case, but also, we should look seriously at supporting either multiple labels on the nodepool side, or multiple label requests on the zuul side. or both. | 16:56 |
| fdegir | that was all from me for the timebeing so thanks for listening | 16:57 |
| fdegir | i’ll be back and look at the backlog later but just to mention jnlp-like thingy is something i want to have a discussion about later | 16:58 |
| corvus | fdegir: the zuul executor needs to be able to contact the nodes (or at least one node in the case of a jumphost). however, if the nodes aren't reachable, you may be able to put the executor near the nodes, and connect it to zuul via a vpn or similar. | 16:58 |
| fdegir | corvus: yes, i remember this conversation from ptg | 16:58 |
| fdegir | now they ask me to turn off my phone so talk to you later | 16:59 |
| corvus | fdegir: bon voyage! | 16:59 |
| fdegir | thx | 16:59 |
| corvus | Shrews, clarkb, mordred: what i'm getting is that because of the way nodepool works, we have some flexibility and some things that otherwise would have needed multiple label support may not because of the ability to request individual resources. however, it does seem like there are some likely use cases for multiple label support. fdegir's is one example, mordred's is another. | 17:01 |
| corvus | Shrews, clarkb, mordred: so i'm inclined to think we should either handle multiple labels on nodes, or multiple label requests. | 17:02 |
| corvus | either way -- the hard stuff is in nodepool. if we do multiple label requests, the nodepool pool manager has to exhaustively search the combinations to figure out if it can satisfy a request. | 17:03 |
| mordred | corvus: I agree with all of those words | 17:04 |
| corvus | if we handle multiple labels on nodes, we probably have to change how we report things | 17:04 |
| corvus | we could also just bite the bullet and do both | 17:04 |
| mordred | gtema: when you say "without attached floating_ip" - does the node have a fixed ip that's 'public' or does it just have a private fixed ip and your nodepool can talk on that private subnet? | 17:05 |
| Shrews | corvus: i'm not quite clear how a "multiple label request" would function | 17:05 |
| mordred | corvus: I can see value in both things - the hard logic in nodepool of matching multiple requests could likely be used to satisfy both UI mechanisms | 17:06 |
| corvus | Shrews: clarkb suggested a list of labels which should be considered as OR'd together. | 17:06 |
| corvus | Shrews: so a zuul job says "ubuntu OR fedora" and each nodepool manager says "do i have an ubuntu node? yes! i can handle this" | 17:06 |
| corvus | Shrews: or another pool manager says "do i have an ubuntu node? no. do i have a fedora node? yes!" | 17:07 |
| clarkb | and if it doesn't have either of them but has capacity could boot a random one of the two | 17:07 |
| Shrews | corvus: ah, gotcha | 17:07 |
| gtema | mordred: private fixed ip in the same subnet | 17:07 |
| Shrews | either way, requires a nodepool change then | 17:07 |
| corvus | i'm being loose with terminology, i should really say "a pool manager says do i have an ubuntu label configured" | 17:07 |
| corvus | Shrews: yep. zuul gets off easy :| | 17:08 |
| gtema | mordred: issue is that nodepool fails (raise exception) if interface_ip is empty | 17:08 |
| Shrews | All I wanted to do was register static nodes :( *sad trombone* | 17:08 |
| corvus | Shrews: it just has to worry about how the yaml is formatted. maybe that's not so easy after all. :) | 17:08 |
| corvus | Shrews, clarkb, mordred: how about we let our subconsious minds mull this over the weekend and regroup for a decision next week? | 17:09 |
| Shrews | ++ | 17:09 |
| mordred | gtema: cool. so - there is an option you can put into your clouds.yaml for that cloud in question | 17:09 |
| mordred | gtema: "private: true" | 17:09 |
| mordred | gtema: that will tell shade that it should use the private address for interface_ip | 17:09 |
| clarkb | Shrews: corvus wfm | 17:10 |
| gtema | mordred: yes, I have found this part in the sdk, but it is somehow weird. The cloud is public in reality. It is your loved OTC | 17:11 |
| mordred | \o/ | 17:11 |
| mordred | gtema: yah - that's ... maybe not the *best* named config option - but what it's saying is not that the cloud is private or public, but that shade should consider the private interface to be how you want to connect to the nodes from the location you're using shade | 17:12 |
| mordred | "connect_using_private_interface" would probably be a better option name | 17:12 |
| gtema | morder: :-/ ok | 17:12 |
| gtema | mordred: should I make a patch for renaming? | 17:13 |
| *** myoung|ruck is now known as myoung|ruck|biab | 17:13 | |
| mordred | no - not just yet - I'm not crazy about that long name either - let's see if we can come up with a better name - or possibly what we might want to do is make this a flag in the networks: config | 17:14 |
| mordred | gtema: but this is a thing that comes up from time to time - so privatE: true is clearly not working for peopl | 17:14 |
| gtema | mordred: ok. got it. | 17:15 |
| gtema | mordred: story on storyboard for discussion? | 17:15 |
| mordred | gtema: that's a great idea | 17:17 |
| gtema | mordred: pl | 17:17 |
| gtema | ok | 17:17 |
| *** acozine has quit IRC | 17:19 | |
| gtema | mordred: done at https://storyboard.openstack.org/#!/story/2001966 | 17:22 |
| openstackgerrit | Merged openstack-infra/nodepool master: Fix typo with _logConsole function https://review.openstack.org/566356 | 17:31 |
| mordred | gtema: thanks! | 17:45 |
| gtema | welcome | 17:45 |
| *** pcaruana has joined #zuul | 17:54 | |
| *** acozine has joined #zuul | 17:57 | |
| *** dtruong2 has joined #zuul | 18:22 | |
| *** dtruong2 has quit IRC | 18:22 | |
| *** dtruong2 has joined #zuul | 18:22 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool master: Add openstacksdk to nodepool-functional-py35-src https://review.openstack.org/566387 | 18:50 |
| *** dtruong2 has quit IRC | 18:59 | |
| *** gtema has quit IRC | 19:06 | |
| *** pcaruana has quit IRC | 19:10 | |
| *** hasharAway is now known as hashar | 20:36 | |
| openstackgerrit | Merged openstack-infra/zuul master: Add zuul systemd drop-in files for CentOS 7 https://review.openstack.org/565074 | 21:13 |
| openstackgerrit | Merged openstack-infra/zuul master: Skip attempting python3-devel installation on CentOS 7 https://review.openstack.org/565080 | 21:13 |
| openstackgerrit | Merged openstack-infra/zuul master: Add start and end timestamp to task and play result in zuul_json callback https://review.openstack.org/563888 | 21:17 |
| *** myoung|ruck|biab is now known as myoung|ruck | 21:18 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add logo to docs https://review.openstack.org/566406 | 21:22 |
| pabelanger | if anybody is up for some nodepool-dsvm testing reviews, removes debian-jessie and adds fedora with mirrors: https://review.openstack.org/#/q/topic:nodepool-dsvm+status:open | 21:26 |
| corvus | pabelanger, clarkb: i think it's time to look at moving that stuff out of nodepool. when i went to make the release, nodepool changes are basically just dib testing changes. | 21:39 |
| corvus | it was hard to find out if anything in nodepool itself had actually changed | 21:40 |
| corvus | i think inside of nodepool itself, we should have enough testing to build a single OS image with dib and an ssh key | 21:40 |
| corvus | the rest we should either move to dib, or to project-config | 21:41 |
| *** hashar has quit IRC | 21:42 | |
| clarkb | wfm, dib is probably preferable so that changes are self testing | 21:45 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add logo to docs https://review.openstack.org/566406 | 21:47 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool master: Remove initial stub release note https://review.openstack.org/566409 | 21:48 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool master: Add logo to docs https://review.openstack.org/566410 | 21:48 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Add logo to docs https://review.openstack.org/566411 | 21:50 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-base-jobs master: Add logo to docs https://review.openstack.org/566412 | 21:51 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-sphinx master: Add logo to docs https://review.openstack.org/566413 | 21:52 |
| corvus | fungi: where did you find that mailman was trying to run list_lists? | 21:55 |
| corvus | fungi: er, that the debian scripts were trying to run that, rather | 21:55 |
| corvus | fungi: ah i think i found it | 21:58 |
| corvus | also, sorry, wrong channel :) | 22:01 |
| *** acozine has quit IRC | 22:22 | |
| *** rlandy has quit IRC | 22:30 | |
| *** pwhalen has quit IRC | 22:34 | |
| *** pwhalen has joined #zuul | 22:36 | |
| fungi | no problem, just found my way back from the beach bar | 22:43 |
| *** ssbarnea_ has quit IRC | 22:43 | |
| *** trishnag has quit IRC | 23:24 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!