| *** mattw4 has quit IRC | 00:01 | |
| *** igordc has quit IRC | 00:12 | |
| *** sgw has quit IRC | 00:25 | |
| *** tobberydberg has quit IRC | 00:35 | |
| *** johanssone has quit IRC | 00:36 | |
| *** tobberydberg has joined #zuul | 00:37 | |
| *** fdegir has quit IRC | 00:37 | |
| *** fdegir3 has joined #zuul | 00:37 | |
| *** johanssone has joined #zuul | 00:39 | |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Add container functional test https://review.opendev.org/693464 | 00:45 |
|---|---|---|
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Dockerfile: install nodepool-builder dependencies https://review.opendev.org/693306 | 00:45 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Add a releases container job https://review.opendev.org/698818 | 00:45 |
| *** saneax has quit IRC | 00:51 | |
| *** sgw has joined #zuul | 00:52 | |
| *** jamesmcarthur has joined #zuul | 00:55 | |
| *** jamesmcarthur has quit IRC | 01:28 | |
| *** jamesmcarthur has joined #zuul | 02:09 | |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Add container with siblings functional test https://review.opendev.org/693464 | 02:29 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Dockerfile: install nodepool-builder dependencies https://review.opendev.org/693306 | 02:29 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Add a releases container job https://review.opendev.org/698818 | 02:29 |
| *** jamesmcarthur has quit IRC | 02:44 | |
| *** swest has quit IRC | 02:45 | |
| *** jamesmcarthur has joined #zuul | 02:47 | |
| *** jamesmcarthur has quit IRC | 02:48 | |
| *** swest has joined #zuul | 03:01 | |
| *** bhavikdbavishi has joined #zuul | 03:09 | |
| *** jamesmcarthur has joined #zuul | 03:10 | |
| *** bhavikdbavishi has quit IRC | 03:13 | |
| *** jamesmcarthur has quit IRC | 03:34 | |
| *** jamesmcarthur has joined #zuul | 03:37 | |
| *** fdegir3 has quit IRC | 03:37 | |
| *** fdegir3 has joined #zuul | 03:38 | |
| *** jamesmcarthur has quit IRC | 03:53 | |
| *** rlandy has quit IRC | 04:07 | |
| *** mhu has quit IRC | 04:46 | |
| *** raukadah is now known as chkumar|rover | 05:08 | |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Add container-with-siblings functional test https://review.opendev.org/693464 | 05:11 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Dockerfile: install nodepool-builder dependencies https://review.opendev.org/693306 | 05:11 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Add a container-with-releases functional test https://review.opendev.org/698818 | 05:11 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Functional tests - use common verification script https://review.opendev.org/698834 | 05:28 |
| *** swest has quit IRC | 05:32 | |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Add a container-with-releases functional test https://review.opendev.org/698818 | 05:47 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Functional tests - use common verification script https://review.opendev.org/698834 | 05:47 |
| *** pcaruana has joined #zuul | 06:02 | |
| *** swest has joined #zuul | 06:55 | |
| *** logan- has quit IRC | 07:09 | |
| *** logan_ has joined #zuul | 07:10 | |
| *** logan_ is now known as logan- | 07:10 | |
| *** saneax has joined #zuul | 07:44 | |
| *** hashar has joined #zuul | 07:48 | |
| *** jcapitao has joined #zuul | 08:01 | |
| *** fdegir3 is now known as fdegir | 08:18 | |
| *** jangutter has joined #zuul | 08:33 | |
| *** tosky has joined #zuul | 08:40 | |
| *** jpena|off is now known as jpena | 08:54 | |
| *** themroc has joined #zuul | 09:21 | |
| tobiash | mnaser: sorry for off topic here, do you know what's the process how we could revive this spec: https://review.opendev.org/572805 ? | 09:33 |
| tobiash | we found that this (and the according implementation) is crucual for our cloud with higher load when combining ceph storage with local nvme disks | 09:34 |
| tobiash | it might be interesting for you as well | 09:34 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Dockerfile: install nodepool-builder dependencies https://review.opendev.org/693306 | 09:41 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Add a container-with-releases functional test https://review.opendev.org/698818 | 09:41 |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Functional tests - use common verification script https://review.opendev.org/698834 | 09:41 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Log apscheduler triggers https://review.opendev.org/698858 | 09:51 |
| *** mhu has joined #zuul | 09:52 | |
| tobiash | zuul-maint: we're trying to hunt down occasionally missed timer triggers. This should aid with debugging ^ | 09:59 |
| tobiash | it might be some sort of race wrt reconfigurations but we want to rule out missing triggers from apscheduler | 10:01 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Don't set ansible_python_interpreter if in vars https://review.opendev.org/698774 | 10:48 |
| *** yolanda has quit IRC | 10:53 | |
| openstackgerrit | Merged zuul/zuul master: Fix exceptions on reconfig with changing window https://review.opendev.org/697023 | 11:06 |
| openstackgerrit | Merged zuul/zuul master: Fix data return child jobs not skipped on failure https://review.opendev.org/698712 | 11:07 |
| *** yolanda has joined #zuul | 11:25 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Support podman for test setup https://review.opendev.org/698884 | 11:45 |
| *** jangutter has quit IRC | 11:47 | |
| *** jangutter_ has joined #zuul | 11:47 | |
| *** mauved has quit IRC | 11:50 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Fix canceling builds in starting phase https://review.opendev.org/684414 | 11:52 |
| *** mauved has joined #zuul | 11:52 | |
| *** jcapitao is now known as jcapitao|afk | 11:54 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Enforce sql connections for scheduler and web https://review.opendev.org/630472 | 11:57 |
| *** rfolco has joined #zuul | 11:58 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Defer setting build result to event queue https://review.opendev.org/666643 | 12:03 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add support for smart reconfigurations https://review.opendev.org/652114 | 12:06 |
| openstackgerrit | Merged zuul/zuul master: Don't process builds not longer in job graph https://review.opendev.org/697420 | 12:12 |
| *** EmilienM has quit IRC | 12:12 | |
| *** EmilienM has joined #zuul | 12:13 | |
| *** EmilienM is now known as EvilienM | 12:17 | |
| *** jpena is now known as jpena|lunch | 12:29 | |
| *** mgoddard has quit IRC | 12:30 | |
| *** reiterative has joined #zuul | 12:31 | |
| *** hashar has quit IRC | 12:50 | |
| mnaser | tobiash: i think someone would have to work on that spec, i think it's very useful tbh. there has been talks of adding precaching images in nova | 13:02 |
| mnaser | which would indirectly solve this too | 13:02 |
| tobiash | mnaser: I guess I'll start to work on that spec next year or at least support my collegues | 13:04 |
| mnaser | tobiash: awesome! yeah, this would have a huge impact in off-loading the glance server in that scenario you describe | 13:04 |
| tobiash | yes, we have a version of that running in one of our clouds and the impact is dramatic | 13:05 |
| tobiash | mnaser: so to start reviving it I need to restore that change and move the file to specs/ussuri/approved? | 13:07 |
| mnaser | tobiash: yep, that's step 0 :) | 13:08 |
| *** jcapitao|afk is now known as jcapitao | 13:08 | |
| tobiash | mnaser: I don't have the restore buttin, shall I upload it as a new change? | 13:11 |
| mnaser | tobiash: good question.. yes i would repropose (like https://review.opendev.org/#/c/683002/ ) | 13:13 |
| AJaeger | tobiash: that's in nova-specs? Any nova-specs core has the restore button, check https://review.opendev.org/#/admin/groups/302,members | 13:13 |
| AJaeger | tobiash: so, any nova-core plus a few more... | 13:14 |
| mnaser | i think the thing is you're supposed to repropose it (even nova-spec cores are doing that) | 13:14 |
| tobiash | ok, then I'll repropose and add a link back to the original one for earlier discussion? | 13:15 |
| *** mgoddard has joined #zuul | 13:21 | |
| mnaser | tobiash: yep seems fair (and use the same topic so it can be referenced there too) | 13:22 |
| tobiash | I asked in #openstack-nova about the preferred way | 13:23 |
| tobiash | a nova core already restored it | 13:23 |
| *** jamesmcarthur has joined #zuul | 13:26 | |
| *** jangutter_ is now known as jangutter | 13:30 | |
| *** jamesmcarthur has quit IRC | 13:34 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: [WIP] Docker compose example: add keycloak authentication https://review.opendev.org/664813 | 13:34 |
| *** rlandy has joined #zuul | 13:35 | |
| *** jpena|lunch is now known as jpena | 13:37 | |
| *** jamesmcarthur has joined #zuul | 13:44 | |
| *** Goneri has joined #zuul | 13:45 | |
| pabelanger | is zuul.artifacts passed down to all child jobs? eg: I have a use case for a buildset with JobA->JobB->JobC, where both A and B build artifacts and zuul_return. I'd like JobC to have access to both. | 13:47 |
| pabelanger | I plan to test, but figured I ask first | 13:47 |
| pabelanger | tobiash: re: glance, I always though torrents was a good idea too, then compute nodes could share them | 13:50 |
| tobiash | pabelanger: interesting idea, but ceph is already a distributed storage system so I guess that's a layer too much (but might be useful in a non-ceph glance store) | 13:53 |
| pabelanger | tobiash: yah, this was mostly in infra-cloud that we ran a few years ago. no ceph, single glance | 13:55 |
| pabelanger | but this idea sounds good too | 13:55 |
| Shrews | corvus: your gce change lgtm. my initial review was lacking coffee as i was laser focused on one class and how it interacted with the common code, so please ignore it. :) | 13:57 |
| *** mhu has quit IRC | 14:04 | |
| *** mhu has joined #zuul | 14:05 | |
| mnaser | tobiash: a cool idea btw if you are looking to implement it is https://github.com/dragonflyoss/Dragonfly | 14:07 |
| *** jamesmcarthur has quit IRC | 14:16 | |
| *** jamesmcarthur has joined #zuul | 14:17 | |
| mordred | mnaser: you know - back in the day we were pitching the idea of doing something like back behind glance/nova | 14:21 |
| *** jamesmcarthur has quit IRC | 14:21 | |
| *** jamesmcarthur has joined #zuul | 14:21 | |
| mordred | but I haven't read backscroll yet | 14:21 |
| mnaser | i think there is like actual bittorrent code tha twas recently removed from nova | 14:21 |
| mordred | yeah - it's the sort of thing that needs users | 14:23 |
| mordred | we were looking at it when I had the team of folks inventing tripleo and working on how to make hp public cloud better ... bittorrent image distribution seemed like a good idea :) | 14:23 |
| mordred | mnaser: re: the above - I though if you had ceph drivers for nova and glance that nova was already doing zero-copy ceph operations to boot an image - is that only if you're doing BFV? | 14:24 |
| pabelanger | ++ bittorrent | 14:25 |
| mnaser | mordred: we do, thats all good, but this is the case where your nova-compute is using local storage (something like fast nvme's for CI?) but glance is your image storage. in order to download an image from glance, you need to hit the HTTP API | 14:25 |
| mnaser | that spec proposes that if nova-compute sees that the image lives on ceph, and it has access to that cluster, it will download it via ceph instead of hitting the glance API | 14:26 |
| mnaser | sorry, *ceph is your image storage* | 14:26 |
| mnaser | so short-circuiting the download via glance api if the image is on ceph and nova-compute is using local storage | 14:27 |
| tobiash | yes, fast local nvme for ci is our use case :) | 14:27 |
| mordred | mnaser: ahhh. cool | 14:28 |
| mnaser | tobiash: i figured ;) | 14:34 |
| mordred | Shrews, tobiash, corvus: patch looked good to me - and with 3x+2 I went ahead and landed it. I figured that means it's good enough and we can always improve things as we move forward | 14:34 |
| *** dmellado has quit IRC | 14:36 | |
| *** dmellado has joined #zuul | 14:39 | |
| *** igordc has joined #zuul | 14:40 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Ensure logs dir exists https://review.opendev.org/698924 | 14:53 |
| *** jamesmcarthur has quit IRC | 15:02 | |
| *** jamesmcarthur has joined #zuul | 15:02 | |
| *** themroc has quit IRC | 15:06 | |
| *** jamesmcarthur has quit IRC | 15:09 | |
| openstackgerrit | Merged zuul/nodepool master: Add Google Cloud provider https://review.opendev.org/698342 | 15:12 |
| *** jamesmcarthur has joined #zuul | 15:17 | |
| *** rfolco is now known as rfolco|doctor | 15:27 | |
| *** dtroyer has quit IRC | 15:52 | |
| *** chkumar|rover is now known as raukadah | 15:53 | |
| *** igordc has quit IRC | 16:15 | |
| *** mattw4 has joined #zuul | 16:21 | |
| *** sgw has quit IRC | 16:24 | |
| *** sgw has joined #zuul | 16:39 | |
| clarkb | tristanC: for https://review.opendev.org/#/c/695594/1 that only updates the test playbook. Did a file get missed on git add? | 16:46 |
| clarkb | AJaeger: ^ fyi on that? | 16:47 |
| clarkb | er s/\?// | 16:47 |
| *** armstrongs has joined #zuul | 16:51 | |
| AJaeger | clarkb: I should have +2, the other files don't have this hardcoded, this is obvious once I digged deeper. The commit message could be clearer ;( | 16:55 |
| AJaeger | I left a message | 16:55 |
| clarkb | ok so just a test cleanup then | 16:55 |
| clarkb | wfm | 16:55 |
| AJaeger | clarkb: hope so ;) We can wait for tristanC to confirm... | 16:56 |
| clarkb | nah I approved. | 16:56 |
| clarkb | if there was a missed file a followup change is fine too | 16:56 |
| AJaeger | That works as well... | 16:57 |
| *** jcapitao is now known as jcapitao|off | 17:02 | |
| clarkb | mordred: you probably know the answer to my question in https://review.opendev.org/#/c/691114/10 does go test imply go build? | 17:03 |
| *** jcapitao|off has quit IRC | 17:07 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Gitlab - Implement the note event and the comment trigger action https://review.opendev.org/698964 | 17:07 |
| *** michael-beaver has joined #zuul | 17:09 | |
| mordred | clarkb: I believe so | 17:10 |
| clarkb | cool does anyone else want to review the go job and role additions to zuul-jobs or should I approve it? | 17:12 |
| clarkb | corvus: ^ possible you are interested but I'm happy to approve as is | 17:12 |
| corvus | clarkb: yeah, i'll give it a quick pass and +3 since it's a new group; thanks to you and mordred for doing the hard work :) | 17:13 |
| openstackgerrit | Merged zuul/zuul-jobs master: fetch-zuul-cloner: replace /home/zuul by ansible_user_dir https://review.opendev.org/695594 | 17:18 |
| *** jangutter has quit IRC | 17:25 | |
| *** openstackgerrit has quit IRC | 17:29 | |
| corvus | clarkb: did you mean to +2 691114? | 17:32 |
| clarkb | corvus: yes I did | 17:37 |
| corvus | clarkb: +2 from me, feel free to +# | 17:38 |
| clarkb | in fact I completely failed at posting my question there when I asked mordred | 17:38 |
| clarkb | done | 17:38 |
| clarkb | there were a couple other minor docs things that we can address in a followup | 17:38 |
| mordred | clarkb: I was wondering about your reference to "your question" earlier | 17:40 |
| *** hashar has joined #zuul | 17:49 | |
| *** openstackgerrit has joined #zuul | 17:50 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Adds go jobs and roles. https://review.opendev.org/691114 | 17:50 |
| *** jpena is now known as jpena|off | 17:53 | |
| Shrews | ianw: i left you a question on 694845. i cannot figure out what that ARG option is supposed to do | 17:57 |
| Shrews | tobiash: should https://review.opendev.org/695001 have a release note? seems like a new feature maybe? | 18:10 |
| Shrews | hrm, at least needs a doc update for the new endpoint | 18:14 |
| tobiash | Shrews: yes, I'll add that | 18:16 |
| openstackgerrit | Tobias Henkel proposed zuul/nodepool master: Add ready endpoint to webapp https://review.opendev.org/695001 | 18:27 |
| openstackgerrit | Tobias Henkel proposed zuul/nodepool master: Remove uneeded shebang and exec bit on some files https://review.opendev.org/692100 | 18:30 |
| *** hashar has quit IRC | 18:33 | |
| tristanC | clarkb: no file got missed, i was chasing another issue and thought this was used by the job. the commit message could have been clearer | 18:35 |
| *** yolanda has quit IRC | 18:43 | |
| openstackgerrit | Merged zuul/nodepool master: Ensure both kubernetes and openshift token are b64decoded https://review.opendev.org/687435 | 18:48 |
| *** saneax has quit IRC | 18:48 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Keep task stdout/stderr separate in result object https://review.opendev.org/650276 | 18:49 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Optionally support mitogen for job execution https://review.opendev.org/657024 | 18:53 |
| clarkb | tobiash: ^ I believe pabelanger uses mitogen and it works | 18:54 |
| pabelanger | yes, we do on bastion host to deploy zuul.a.c., we get much faster deploys | 18:55 |
| pabelanger | I think it would be worth while to test it for zuul jobs | 18:55 |
| *** tosky has quit IRC | 18:57 | |
| *** igordc has joined #zuul | 19:05 | |
| openstackgerrit | Merged zuul/nodepool master: Remove uneeded shebang and exec bit on some files https://review.opendev.org/692100 | 19:07 |
| corvus | i thought someone *did* test it for zuul jobs and found it didn't help? | 19:10 |
| corvus | iirc, the description of what mitogen does doesn't actually suggest that it would be very beneficial for zuul | 19:11 |
| corvus | but i'd love to have a current real-world report | 19:12 |
| corvus | (with zuul) | 19:13 |
| tobiash | in theory it reduces the resources consumed by ansible and improves job runtime | 19:15 |
| pabelanger | yes, we see that | 19:15 |
| mnaser | is there any docs when y'all went through the pain of dealing with setting up a swift backed docker registry? | 19:15 |
| tobiash | pabelanger: are you using mitogen in zuul? | 19:16 |
| mnaser | the authurl it's listing is: "https://storage.myprovider.com/v3/auth" which makes no sense, and /v3/auth is not even a valid endpoint in keystone? | 19:16 |
| clarkb | corvus: tobiash pabelanger mitogen claims to reuse processes rather than forking a new one for each playbook task. I would expect that bit would make jobs run quicker | 19:16 |
| clarkb | mnaser: isn't your keystone auth endpoint arbitrary? | 19:16 |
| clarkb | the cloud can make it whatever they like. I'm sure someone just typed in an example off the top of their head | 19:17 |
| * clarkb looks for our configs | 19:17 | |
| pabelanger | tobiash: no, we have zuul run a promote job, on our bastion host, the nested ansible does use it | 19:17 |
| tobiash | clarkb: yes that's true as well although the benefit in my first tests one and a half years ago was less impressive than expected | 19:17 |
| mnaser | right, but the docs list: URL for obtaining an auth token. https://storage.myprovider.com/v2.0 or https://storage.myprovider.com/v3/auth | 19:17 |
| mnaser | so that seems to imply "please put the explicit version name" | 19:17 |
| tobiash | clarkb: but it might be that our executor itself was too much bottlenecked because it was on ceph back then | 19:17 |
| *** jamesmcarthur has quit IRC | 19:18 | |
| clarkb | mnaser: we use an explict v2.0 url | 19:19 |
| clarkb | mnaser: similar to https://storage.myprovider.com/v2.0 | 19:19 |
| mnaser | welp that won't work cause we don't even have v2.0 anymore \o/ | 19:19 |
| mnaser | i found the upstream library though https://github.com/ncw/swift | 19:20 |
| clarkb | mnaser: and you've tried using an explicit v3 url? | 19:20 |
| mnaser | i tried a couple of combos and figured i'd ask now | 19:20 |
| *** jamesmcarthur has joined #zuul | 19:20 | |
| corvus | +registry_swift_authurl: https://identity.api.rackspacecloud.com/v2.0/ | 19:20 |
| mnaser | hmm ill try /v3 | 19:21 |
| corvus | mnaser: ^ that's from our git history for when we used swift backed docker registry2.0 | 19:21 |
| corvus | mnaser: that was the value we passed to "storage.swift.authurl" in config.yml | 19:21 |
| mnaser | "Swift authentication failed: HTTP Error: 404: 404 Not Found" -- ill just dig some more, i think there might be something else im missing | 19:21 |
| corvus | mnaser: https://opendev.org/opendev/system-config/src/commit/f357e5cdab3f54f4a2cbf8e327dcff943a3ac7b3/playbooks/roles/registry/templates/config.yml.j2 | 19:22 |
| mnaser | authversion | 19:22 |
| corvus | mnaser: that's what that file looked like at the time | 19:22 |
| mnaser | there we go we went into "panic: Swift authentication failed: Bad Request" with /v3 | 19:23 |
| mnaser | and authversion: 3 | 19:23 |
| tobiash | in real live zuul_console seems to be broken with mitogen in https://review.opendev.org/657024 | 19:23 |
| clarkb | I'm betting the zuul registry would just work with keystone v3 auth | 19:24 |
| corvus | mnaser: nb that we had problems with that in production; it mostly worked, but occasionally blobs went missing. we never triaged whether it was the docker registry swift driver or in interaction with the non-standard "swift" that rax runs | 19:24 |
| clarkb | depending on what your use cases are | 19:24 |
| mnaser | yeah.. i'm checking out harbor which has a whole UI, oidc for auth, etc.. and it uses the docker registry | 19:25 |
| *** jamesmcarthur has quit IRC | 19:25 | |
| clarkb | and ya we would get empty objects | 19:26 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Optionally support mitogen for job execution https://review.opendev.org/657024 | 19:36 |
| *** michael-beaver has quit IRC | 19:37 | |
| *** sgw has quit IRC | 19:38 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Optionally support mitogen for job execution https://review.opendev.org/657024 | 19:51 |
| *** gmann is now known as gmann_afk | 20:13 | |
| tobiash | clarkb: do you want the tenant setting in 657024 or would a followup be ok? | 20:16 |
| *** hashar has joined #zuul | 20:18 | |
| clarkb | followup should be ok | 20:24 |
| tobiash | kkkkkkkkkkkkkkkkkkkk | 20:25 |
| tobiash | whoops | 20:25 |
| tobiash | keyboard hung | 20:25 |
| *** Goneri has quit IRC | 20:27 | |
| *** rfolco|doctor is now known as rfolco | 20:31 | |
| *** michael-beaver has joined #zuul | 20:38 | |
| * tobiash is benchmarking a new openstack cloud using a check loop pipeline with short running multinode jobs. It's saturating at 150 instances per minute :) | 20:50 | |
| mordred | tobiash: one of nodepool's most successful uses is in destroying clouds | 20:55 |
| mordred | I mean, it's good at CI - but it's REALY good at load-testing openstack apis | 20:56 |
| *** EvilienM is now known as EmilienM | 20:57 | |
| *** hashar has quit IRC | 21:07 | |
| corvus | mordred, tobiash, Shrews: i just created a GKE cluster, made a deployment with a pod with nodepool-launcher and zk, dropped in a nodepool.yaml and watched it spin up a GCE vm. here's the config and output: http://paste.openstack.org/show/787573/ | 21:10 |
| *** rlandy has quit IRC | 21:10 | |
| corvus | note there is no auth info there; it's using a service account which is automatically installed and picked up by the client lib | 21:11 |
| corvus | (i redacted nothing from that paste) | 21:11 |
| tobiash | corvus: awesome :) | 21:11 |
| corvus | (and, incidentally, if you're curious like i was, it looks like gce is about 60 seconds from create to ready) | 21:12 |
| clarkb | gce hasreally consistebt boot times. I think they use the google version of ceph | 21:13 |
| tobiash | 60s is plausible, we have that in our clouds as well most of the time | 21:13 |
| corvus | i'm exploring options for how to set up zuul for the upstream gerrit project. and it looks like "run zuul in k8s and use it to run workloads on gce" might be a really good fit, and the service account should make that even easier | 21:13 |
| corvus | yeah, 60s i think is on par with the fastest openstack clouds opendev has | 21:13 |
| Shrews | neat | 21:16 |
| Shrews | corvus: looking at that example, I think we should make the config ‘driver’ value match the driver directory name: gce | 21:20 |
| Shrews | to be consistent | 21:20 |
| corvus | Shrews: oh whoops, it is. i accidentally copied rev-1 of the config to the paste | 21:21 |
| mordred | I thought we did? | 21:21 |
| mordred | ah | 21:21 |
| corvus | Shrews: "host: localhost" and "driver: gce" are the changes | 21:21 |
| mordred | corvus: that's pretty awesome | 21:21 |
| Shrews | oh good | 21:21 |
| corvus | here's the real config: http://paste.openstack.org/show/787574/ | 21:21 |
| corvus | sorry about that | 21:21 |
| clarkb | corvus: out if curiousity is there areason a gce driver would be preferable to the gerrit project over running jobs in k8s? | 21:26 |
| clarkb | (since it soundslike they have k8s to run zuul in anyway) | 21:26 |
| corvus | clarkb: i'd like to do both. but i have little experience with jobs in pods, and we know that image building will be important, so i'd like to start with gce for the greatest chance of initial success. | 21:28 |
| mordred | yah | 21:29 |
| mordred | same here - although corvus is doing all the work | 21:29 |
| *** igordc has quit IRC | 21:30 | |
| corvus | well, i'm thinking we're about at a place where it doesn't have to just be me :) | 21:31 |
| clarkb | container image building you mean? | 21:32 |
| corvus | next week i'll confirm the plan with han-wen at google, send an email to the list, and ask the gerrit maintainers for some repos. | 21:32 |
| corvus | clarkb: yeah; sometime's that's squirrely in containers | 21:32 |
| clarkb | ya and its squirrely all the time with docker and buildah right? need a special builder like img to do it iirc | 21:32 |
| mordred | or special permissions | 21:33 |
| mordred | like - it can be done and all ... but it's _really_ easy to do it in a vm | 21:33 |
| corvus | the perms are probably okay? i think it's maybe the same as for zuul-exec? but yeah | 21:33 |
| mordred | it's the poster child for tasks that are _not_ easier in a container | 21:33 |
| fungi | sorry for being on silent running all day, i've been here but trying to get some stuff done. stepping out for an early dinner now, but if anyone needs me for something i should be back in an hour-ish | 21:33 |
| corvus | i just don't want to be making excuses for why something is hard or isn't working with zuul the first time out | 21:34 |
| clarkb | mordred: in my head its still "run iscsid" in fact this came up today in #openstack-qa :) | 21:34 |
| fungi | er, i guess i meant to say that in #openstack-infra, but it's reasonably applicable here ;) | 21:34 |
| corvus | but i'm very keen to add gke as a second nodepool launcher and start experimenting with it there. | 21:34 |
| corvus | fungi: nice to hear from you :) | 21:34 |
| clarkb | corvus: on the positive side of things gce/gke don't ipv6 so you'll avoid that entire class of problems | 21:34 |
| fungi | :D | 21:35 |
| clarkb | whcih is funny becaus aiui all of googles infrastructure runs on ipv6 only. So its ipv4 on ipv6 and you can't ipv6 as a user :/ | 21:35 |
| corvus | clarkb: yes, "positive". apparently ipv6 in gcloud is "you can terminate ipv6 with the load balancer product and connect it to your service running on ipv4" | 21:35 |
| clarkb | corvus: yup | 21:35 |
| corvus | clarkb: naturally they run on ipv6 -- so they can give all their ipv4 addrs to cloud users :) | 21:36 |
| corvus | but that reminds me, i was thinking in this configuration, we could probably be nice netizens and only request internal addrs. | 21:36 |
| corvus | huh, well it's looking like there's no pnat. either 1:1 nat or no connection to outside network at all. | 21:52 |
| corvus | do any of the nodepool drivers have a "prefer internal address" option? | 21:57 |
| corvus | i want to say that for openstack we pushed that out to clouds.yaml? | 21:57 |
| mordred | corvus: yeah - we havea . private: true option in clouds.yaml | 22:01 |
| corvus | ok, i think i'll add a 'use-internal-ip' option to the gce provider | 22:02 |
| mordred | corvus: which is intended to be used in the similar use case - you're running in the same network context as the vms are coming from | 22:02 |
| corvus | it's looking like launching a vm without an external ip is not practical (i mean, you could do it, but it would either access zero network resources, or you would need to set up you own network/proxy). that's not exciting enough for me to implement support for right now. but i think it would be a benefit to have zuul use the internal ip for its part, to route traffic better (and save money) | 22:03 |
| *** logan- has quit IRC | 22:04 | |
| *** logan- has joined #zuul | 22:06 | |
| mordred | corvus: ++ | 22:06 |
| mordred | corvus: is there a way for the driver to detect if it's running ,.. wait - that's a dumb question | 22:07 |
| corvus | mordred: probably, actually. | 22:07 |
| mordred | corvus: I was GOING to say "is there a way for the driver to detect if it's running local to the vms and not need the option" | 22:07 |
| corvus | yeah | 22:07 |
| mordred | corvus: but - it's irrelevant - becasue what you want to know is if the executor is running there | 22:08 |
| corvus | i mean, maybe something about the network, or the project, or... | 22:08 |
| corvus | mordred: that's true | 22:08 |
| mordred | so I think "use-internal-ip" is more corrector | 22:08 |
| mordred | and maybe we could something something autodetect something in the future as an enhancement something | 22:08 |
| corvus | i think you're right, and this should just be expressed by the user | 22:09 |
| corvus | huh, i just noticed, 'region' isn't actually used | 22:10 |
| corvus | (zone implies region) | 22:10 |
| *** jamesmcarthur has joined #zuul | 22:17 | |
| *** rfolco has quit IRC | 22:19 | |
| *** jamesmcarthur has quit IRC | 22:28 | |
| *** jamesmcarthur has joined #zuul | 22:28 | |
| *** Goneri has joined #zuul | 22:34 | |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: GCE: add use-internal-ip option https://review.opendev.org/699041 | 22:47 |
| corvus | apparently i have used $0.20 worth of google cloud to develop that driver | 22:47 |
| *** michael-beaver has quit IRC | 22:48 | |
| *** pcaruana has quit IRC | 22:48 | |
| *** rfolco has joined #zuul | 22:50 | |
| *** rfolco has quit IRC | 22:55 | |
| *** jamesmcarthur has quit IRC | 22:57 | |
| *** mattw4 has quit IRC | 22:58 | |
| *** jamesmcarthur has joined #zuul | 23:15 | |
| *** jamesmcarthur has quit IRC | 23:18 | |
| *** jamesmcarthur has joined #zuul | 23:37 | |
| *** jamesmcarthur has quit IRC | 23:41 | |
| *** rfolco has joined #zuul | 23:49 | |
| clarkb | corvus: for ^ that implies the test environment can get everything locally right? If so should that be added to the documentation? | 23:53 |
| clarkb | or maybe I've misunderstood and the publci IP is still there but zuul and nodepool will rpefer the private ip for their comms | 23:54 |
| corvus | clarkb: it's for the second thing | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!