| *** armstrongs has quit IRC | 00:01 | |
| *** holser has quit IRC | 00:10 | |
| *** tosky has quit IRC | 00:14 | |
| *** holser has joined #zuul | 01:08 | |
| *** holser has quit IRC | 01:31 | |
| *** rlandy|bbl is now known as rlandy | 01:53 | |
| *** wuchunyang has joined #zuul | 01:57 | |
| *** smyers has quit IRC | 01:59 | |
| *** smyers has joined #zuul | 02:17 | |
| *** bhavikdbavishi has joined #zuul | 03:17 | |
| *** hamalq has quit IRC | 03:21 | |
| *** wuchunyang has quit IRC | 04:01 | |
| *** vishalmanchanda has joined #zuul | 04:31 | |
| *** saneax has joined #zuul | 05:03 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #zuul | 05:33 | |
| *** reiterative has joined #zuul | 05:42 | |
| *** mach1na has joined #zuul | 06:59 | |
| *** mach1na has quit IRC | 07:03 | |
| *** mach1na has joined #zuul | 07:05 | |
| *** bhavikdbavishi has quit IRC | 07:05 | |
| *** mach1na has quit IRC | 07:09 | |
| openstackgerrit | Daniel Pawlik proposed zuul/zuul master: Improve Elasticsearch reporter doc and driver, changed index name https://review.opendev.org/761441 | 07:15 |
|---|---|---|
| *** zenkuro has quit IRC | 07:27 | |
| *** mach1na has joined #zuul | 07:29 | |
| *** jpena|off is now known as jpena | 07:35 | |
| *** mach1na has quit IRC | 07:58 | |
| tobiash | corvus: do you want more reviews on 728118 (rest api tenant scoping)? | 07:58 |
| *** jcapitao has joined #zuul | 07:59 | |
| *** saneax has quit IRC | 08:09 | |
| *** bhavikdbavishi has joined #zuul | 08:10 | |
| *** tosky has joined #zuul | 08:13 | |
| *** bhavikdbavishi1 has joined #zuul | 08:21 | |
| *** bhavikdbavishi has quit IRC | 08:22 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 08:22 | |
| openstackgerrit | zbr proposed zuul/zuul-jobs master: More E208 https://review.opendev.org/761293 | 08:33 |
| *** mach1na has joined #zuul | 08:36 | |
| *** sshnaidm|rover has quit IRC | 08:36 | |
| *** sshnaidm|rover has joined #zuul | 08:38 | |
| *** rpittau|afk is now known as rpittau | 08:39 | |
| *** sshnaidm|rover has quit IRC | 08:43 | |
| *** saneax has joined #zuul | 09:04 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Only report dequeue if we have reported start https://review.opendev.org/761520 | 09:11 |
| *** nils has joined #zuul | 09:19 | |
| *** sugaar has joined #zuul | 09:47 | |
| *** sshnaidm has joined #zuul | 09:49 | |
| *** mach1na has quit IRC | 09:52 | |
| *** mach1na has joined #zuul | 09:55 | |
| *** bhavikdbavishi has quit IRC | 09:55 | |
| *** arxcruz has quit IRC | 10:04 | |
| *** holser has joined #zuul | 10:07 | |
| *** arxcruz has joined #zuul | 10:09 | |
| bschanzel | tristanC: as we discussed some days ago, I've prepared a change for the k8s/OKD provider to remove the default workingDir from their pod specs. Could you take a look at https://review.opendev.org/#/c/758965/ ? | 10:13 |
| *** hashar has joined #zuul | 10:15 | |
| *** noonedeadpunk has quit IRC | 10:32 | |
| *** noonedeadpunk has joined #zuul | 10:32 | |
| *** mach1na has quit IRC | 10:45 | |
| *** jfoufas1 has joined #zuul | 11:21 | |
| *** noonedeadpunk has quit IRC | 11:21 | |
| *** mach1na has joined #zuul | 11:24 | |
| *** noonedeadpunk has joined #zuul | 11:26 | |
| *** mach1na has quit IRC | 11:28 | |
| *** bhavikdbavishi has joined #zuul | 11:35 | |
| *** bhavikdbavishi1 has joined #zuul | 11:38 | |
| *** bhavikdbavishi has quit IRC | 11:40 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 11:40 | |
| *** jcapitao is now known as jcapitao_lunch | 11:52 | |
| *** mach1na has joined #zuul | 11:54 | |
| *** rfolco has joined #zuul | 12:02 | |
| *** wuchunyang has joined #zuul | 12:09 | |
| *** mach1na has quit IRC | 12:10 | |
| *** wuchunyang has quit IRC | 12:14 | |
| *** armstrongs has joined #zuul | 12:28 | |
| *** jpena is now known as jpena|lunch | 12:32 | |
| *** mach1na has joined #zuul | 12:57 | |
| *** rfolco has quit IRC | 13:01 | |
| *** jcapitao_lunch is now known as jcapitao | 13:01 | |
| *** rfolco has joined #zuul | 13:01 | |
| *** Goneri has joined #zuul | 13:11 | |
| *** jpena|lunch is now known as jpena | 13:31 | |
| *** saneax has quit IRC | 13:37 | |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Component Registry in ZooKeeper https://review.opendev.org/759187 | 13:43 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Move management and result events to model https://review.opendev.org/761163 | 13:43 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Allow (de-)serialization of management events https://review.opendev.org/761164 | 13:43 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Allow (de-)serialization of result events https://review.opendev.org/761165 | 13:43 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Add and fix fields in driver trigger event models https://review.opendev.org/761166 | 13:43 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Allow (de-)serialization of trigger events https://review.opendev.org/761167 | 13:43 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Interface to get a driver's trigger event class https://review.opendev.org/761168 | 13:43 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Clear list of Zookeeper connections after tests https://review.opendev.org/761169 | 13:43 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Implementation of Zookeeper backed event queues https://review.opendev.org/761170 | 13:43 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Implementation of Zookeeper event watcher https://review.opendev.org/761171 | 13:43 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Switch to Zookeeper backed trigger event queues https://review.opendev.org/761172 | 13:43 |
| *** bhavikdbavishi has quit IRC | 13:52 | |
| *** armstrongs has quit IRC | 14:09 | |
| *** Goneri has quit IRC | 14:22 | |
| *** vishalmanchanda has quit IRC | 14:30 | |
| *** Goneri has joined #zuul | 14:38 | |
| *** zenkuro has joined #zuul | 14:38 | |
| tristanC | bschanzel: thanks! | 14:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Configure json-server as mock API for development https://review.opendev.org/760933 | 14:54 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: WIP UI: Remove stretchable link from builds and buildset table https://review.opendev.org/761621 | 14:54 |
| tobiash | zuul-maint: what do you think about making this a regex? https://zuul-ci.org/docs/nodepool/configuration.html#attr-providers.[openstack].cloud-images.image-name | 14:59 |
| tobiash | this would make it possible/easier to have a snapshot based image update approach that is driven by zuul | 14:59 |
| tobiash | we'd like to have something like that for the really large images where image building and uploading to glance is not feasable anymore | 15:00 |
| tristanC | tobiash: sounds good to me, should we keep on using re2 or stdlib would be enough? | 15:02 |
| tobiash | since that's nodepool without user config stdlib for that would be enough (nodepool has no re2 anyway) | 15:03 |
| tobiash | the idea would be that nodepool takes the last match of the alphabetically sorted list | 15:04 |
| avass | tobiash: I believe that's how the image filter works for ec2 as well | 15:04 |
| tobiash | oh that one: https://zuul-ci.org/docs/nodepool/configuration.html#attr-providers.[aws].cloud-images.image-filters | 15:05 |
| tobiash | yeah so that would be feature wise then on par with that | 15:05 |
| tristanC | tobiash: i'm confused with re2 usage, shouldn't all new regexp attribute use re2 to be future proof? | 15:05 |
| avass | but except that it's some sort of globbing and not a regex | 15:06 |
| tobiash | jepp | 15:06 |
| corvus | tobiash: ++ i'm interested in 2 ideas: 1) replacing nodepool-builder with some zuul jobs; 2) creating a second nodepool-builder that's snapshot based [that was in the original spec] | 15:08 |
| corvus | tobiash: i think both are viable solutions and worth exploring. #1 gets a lot more visibility to the whole system. | 15:09 |
| tobiash | corvus: we have a running version of a zuul driven image build that uploads to s3 combined with a fake diskimage build that downloads it in the nodepool builder | 15:09 |
| avass | we're doing #1 but using packer. and I think the CI jobs are not completely finished | 15:09 |
| tobiash | corvus: for the really large images we need to get rid of the upload/download and directly use snapshots instead | 15:10 |
| corvus | tobiash: separately -- i'm curious about why uploading to glance isn't feasible. what's the bottleneck? is there some way to improve that? | 15:10 |
| tobiash | corvus: the images are ~700gb which takes >24h to upload via glance | 15:11 |
| corvus | tobiash: but uploading via s3 is shorter? | 15:11 |
| tobiash | well, we upload it to s3 which is shorter, download it to the builders, then upload it to multiple regions via glance | 15:11 |
| tobiash | and we have multiple of those images and glance is not really able to handle this amount of data | 15:12 |
| corvus | right, from a first principles perspective it sounds like there ought to be room for improvement in glance. i wonder if it's worth talking with the glance authors about it, or do you think that even if glance worked as well as its peers (s3, etc) you would still want to avoid it? | 15:13 |
| tobiash | we still want to avoid it, best case would be to get glance upload down to maybe ~6h which would still make overall round trip time of like 18h due to s3 upload, then download, then glance upload | 15:15 |
| tobiash | so even if glance can be tuned (that would need a non-python fast direct stream to ceph probably) we have still a huge penalty for copying data around | 15:16 |
| corvus | tobiash: but if glance were faster, you could use the builder as designed and not copy to s3? | 15:17 |
| corvus | or maybe i don't know why you're copyng to s3 in the first place instead of using dib | 15:18 |
| tobiash | corvus: we don't use the builder there because we need a ci job producing the cache data in this image which can only be done within zuul (and as a side effect makes it debuggable by the project) | 15:19 |
| corvus | tobiash: ok. could you push the data from the job to the builder directly instead of using s3? (or even run the job on the builder as a static node?) | 15:19 |
| *** hashar is now known as hasharKids | 15:20 | |
| tobiash | that's not really possible since we have 12 builders, all running in openshift | 15:20 |
| *** bhavikdbavishi has joined #zuul | 15:33 | |
| *** jfoufas1 has quit IRC | 15:37 | |
| *** flaper87 has quit IRC | 15:42 | |
| *** bhavikdbavishi1 has joined #zuul | 15:56 | |
| *** bhavikdbavishi has quit IRC | 15:59 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 15:59 | |
| openstackgerrit | Merged zuul/nodepool master: k8s/OpenShift Provider: Remove workingDir Attribute https://review.opendev.org/758965 | 15:59 |
| *** zenkuro has quit IRC | 16:03 | |
| *** zenkuro has joined #zuul | 16:04 | |
| *** bhavikdbavishi has quit IRC | 16:06 | |
| *** bhavikdbavishi has joined #zuul | 16:06 | |
| *** wuchunyang has joined #zuul | 16:11 | |
| *** bhavikdbavishi1 has joined #zuul | 16:13 | |
| *** bhavikdbavishi has quit IRC | 16:15 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 16:15 | |
| *** wuchunyang has quit IRC | 16:16 | |
| *** mach1na has quit IRC | 16:25 | |
| clarkb | tristanC: re2 use isn't about future proofing but about performance and avoiding the possibility of slow regex processing in places where regexes are user supplied | 16:32 |
| clarkb | tristanC: in the case of admin supplied configs like for nodepool I think normal regexes are probably fine. It is more ofa concern when they are coming in unvetted zuul configs | 16:32 |
| tristanC | clarkb: what about having this config user supplied in the future? | 16:38 |
| clarkb | then re2 may make sense at that point. The reason for not defaulting to re2 unless it is user facing is you lose a lot of features like negative lookaheads | 16:40 |
| fungi | and also it's yet another non-stdlib dependency | 16:43 |
| *** jpena is now known as jpena|off | 16:43 | |
| fungi | (and which relies on compiled c extensions) | 16:44 |
| tristanC | fungi: we already depending on it for zuul | 16:44 |
| fungi | ahh | 16:45 |
| tristanC | clarkb: it's also for consistency, isn't it odd that some zuul configuration regex support negative lookaheads while some other don't | 16:45 |
| fungi | was trying to install it locally but currently i seem to not be able to build its extensions | 16:46 |
| tristanC | clarkb: iiuc, we didn't fully switch to re2 to avoid a breaking existing regexp | 16:46 |
| fungi | src/re2.cpp:14805:13: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘exc_value’; did you mean ‘curexc_value’? | 16:46 |
| fungi | that's fun | 16:46 |
| clarkb | tristanC: I don't find that odd because negative lookaheads are a very valuable feature in many cases | 16:46 |
| clarkb | its not ideal, but functionality like that is often very useful | 16:46 |
| clarkb | the building problem I often run into is they don't release new versions of re2 often so I'll end up with a cached wheel for an old python that breaks | 16:47 |
| clarkb | then I have to find that in my wheel cache and delete it to force a rebuild. Not really re2's fault but that hits me every 6 months or so | 16:47 |
| fungi | aha, it ships cython-generated files which need to be regenerated with newer cython for use with newer python | 16:48 |
| fungi | and `pip install re2` on its own won't take care of that | 16:48 |
| corvus | we haven't switched to re2 everywhere because in some places negative lookaheads are very important, so we need new features to accomodate that | 16:49 |
| tristanC | corvus: oh, then why are we using re2? | 16:50 |
| corvus | because work was started on that | 16:50 |
| corvus | tristanC: https://review.opendev.org/552809 is the continuation | 16:51 |
| corvus | if someone wants to continue working on that, i think we can eventually switch everything to re2 | 16:52 |
| *** jcapitao is now known as jcapitao_afk | 17:19 | |
| *** rpittau is now known as rpittau|afk | 17:21 | |
| tristanC | alright, then if we don't need negative lookaheads for nodepool image-name, i think it would be better to use re2 there as well | 17:22 |
| *** bhavikdbavishi has quit IRC | 17:25 | |
| corvus | tristanC: let's make sure to change it for the existing (aws?) drivers as well so it's consistent | 17:25 |
| *** bhavikdbavishi has joined #zuul | 17:25 | |
| *** hasharKids is now known as hashar | 17:30 | |
| avass | corvus: aws doesn't use regex afaik | 17:30 |
| avass | unless it does in nodepool somewhere | 17:30 |
| corvus | avass: oh i misunderstood your earlier comment. i see now you indicated it uses a glob. | 17:32 |
| avass | if you're thinking of the image filter i mentioned that's the aws api allowing you to glob and not boto | 17:32 |
| corvus | perhaps we should glob everywhere | 17:32 |
| avass | i guess globbing wouldn't have any perfomance issues | 17:37 |
| *** hamalq has joined #zuul | 17:38 | |
| corvus | i'm just brainstorming. globbing might be easy/simple/consistent with aws driver. but re2 would be more consistent with rest of zuul. i don't have a strong preference. | 17:41 |
| *** bhavikdbavishi has quit IRC | 17:49 | |
| *** jcapitao_afk has quit IRC | 18:11 | |
| *** jcapitao has joined #zuul | 18:18 | |
| *** hashar is now known as hashardinner | 18:39 | |
| *** jcapitao has quit IRC | 18:41 | |
| *** sshnaidm is now known as sshnaidm|afk | 18:44 | |
| *** wuchunyang has joined #zuul | 20:13 | |
| *** wuchunyang has quit IRC | 20:17 | |
| *** hashardinner is now known as hashar | 20:38 | |
| *** nils has quit IRC | 21:21 | |
| *** hashar has quit IRC | 21:27 | |
| *** hamalq has quit IRC | 22:11 | |
| *** rlandy has quit IRC | 22:25 | |
| *** tosky has quit IRC | 23:55 | |
| *** rfolco has quit IRC | 23:55 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!