| *** mattw4 has quit IRC | 00:07 | |
| *** jamesmcarthur has joined #zuul | 00:10 | |
| *** jamesmcarthur has quit IRC | 00:13 | |
| *** jamesmcarthur has joined #zuul | 00:14 | |
| *** jamesmcarthur has quit IRC | 00:19 | |
| *** armstrongs has joined #zuul | 00:39 | |
| *** jamesmcarthur has joined #zuul | 00:44 | |
| *** armstrongs has quit IRC | 00:49 | |
| *** jamesmcarthur has quit IRC | 00:50 | |
| *** jamesmcarthur has joined #zuul | 01:14 | |
| *** rlandy has quit IRC | 01:34 | |
| *** jamesmcarthur has quit IRC | 01:38 | |
| *** jamesmcarthur has joined #zuul | 01:38 | |
| *** jamesmcarthur has quit IRC | 01:44 | |
| *** jamesmcarthur has joined #zuul | 02:08 | |
| *** jamesmcarthur has quit IRC | 02:15 | |
| *** jamesmcarthur has joined #zuul | 02:41 | |
| *** jamesmcarthur has quit IRC | 02:47 | |
| *** rfolco has quit IRC | 02:53 | |
| *** mauved has quit IRC | 03:12 | |
| *** nborg1 has quit IRC | 03:23 | |
| *** jamesmcarthur has joined #zuul | 03:23 | |
| *** jamesmcarthur has quit IRC | 03:29 | |
| *** jamesmcarthur has joined #zuul | 03:36 | |
| *** jamesmcarthur has quit IRC | 03:41 | |
| *** jamesmcarthur has joined #zuul | 03:44 | |
| *** clarkb has quit IRC | 03:45 | |
| *** clarkb has joined #zuul | 03:45 | |
| *** timburke has quit IRC | 03:50 | |
| *** decimuscorvinus_ has quit IRC | 03:50 | |
| *** decimuscorvinus has joined #zuul | 03:51 | |
| *** jamesmcarthur_ has joined #zuul | 03:51 | |
| *** jamesmcarthur has quit IRC | 03:52 | |
| *** shanemcd has quit IRC | 03:52 | |
| *** johanssone has quit IRC | 03:52 | |
| *** mhu has quit IRC | 03:52 | |
| *** johanssone has joined #zuul | 03:53 | |
| *** timburke has joined #zuul | 03:53 | |
| *** shanemcd has joined #zuul | 03:54 | |
| *** johanssone has quit IRC | 04:00 | |
| *** johanssone has joined #zuul | 04:01 | |
| *** jamesmcarthur_ has quit IRC | 05:01 | |
| *** jamesmcarthur has joined #zuul | 05:03 | |
| *** jamesmcarthur has quit IRC | 05:08 | |
| *** bolg has joined #zuul | 05:10 | |
| *** jamesmcarthur has joined #zuul | 05:32 | |
| *** evrardjp has quit IRC | 05:34 | |
| *** evrardjp has joined #zuul | 05:35 | |
| openstackgerrit | Merged zuul/zuul-helm master: Drop args from container spec https://review.opendev.org/709424 | 05:39 |
|---|---|---|
| *** raukadah is now known as chandankumar | 06:03 | |
| *** jamesmcarthur has quit IRC | 06:12 | |
| *** AJaeger has quit IRC | 06:44 | |
| *** AJaeger has joined #zuul | 06:51 | |
| *** saneax has joined #zuul | 07:00 | |
| *** dpawlik has joined #zuul | 07:26 | |
| *** jamesmcarthur has joined #zuul | 07:41 | |
| *** Defolos has joined #zuul | 07:42 | |
| *** avass has joined #zuul | 07:42 | |
| avass | fungi: A regular expression (or list of regular expressions) which describe on what branches a job should run (or in the case of variants, to alter the behavior of a job for a certain branch). | 07:43 |
| avass | aaaah bad paste | 07:43 |
| avass | fungi: regarding https://review.opendev.org/#/c/578557/ we were thinking of implementing something like that. Nice to know someone is already working on it :) | 07:44 |
| *** jamesmcarthur has quit IRC | 07:47 | |
| avass | Ah, I'll take a look at it. :) | 07:52 |
| *** tosky has joined #zuul | 08:04 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Scheduler test app manager https://review.opendev.org/708812 | 08:05 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Use scheduler factory consistently in tests https://review.opendev.org/709542 | 08:17 |
| *** jpena|off is now known as jpena | 08:32 | |
| *** jamesmcarthur has joined #zuul | 08:43 | |
| *** jcapitao has joined #zuul | 08:47 | |
| *** jamesmcarthur has quit IRC | 08:48 | |
| *** yolanda__ is now known as yolanda | 09:00 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Use scheduler manager consistently in tests https://review.opendev.org/709542 | 09:24 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Refactor executor_client in tests https://review.opendev.org/709672 | 09:24 |
| *** jamesmcarthur has joined #zuul | 09:29 | |
| avass | Is there any way to configure how long a job can be 'queued', I guess waiting for a node, until it automatically fails/skips/succeeds? | 09:30 |
| *** jamesmcarthur has quit IRC | 09:33 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Refactor merge_client in tests https://review.opendev.org/709676 | 09:47 |
| zbr | is there are way to disable the retry support for pre on specific jobs? | 10:09 |
| *** mhu has joined #zuul | 10:49 | |
| fbo | tobiash: hi, I would like to know if you use zubbi in production https://github.com/bmwcarit/zubbi and if there is a demo instance we could play with ? | 10:50 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Fix install-docker on centos-8 https://review.opendev.org/709184 | 10:57 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Tests ensure-tox on all-platforms https://review.opendev.org/708642 | 11:00 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Tests bindep role on all-platforms https://review.opendev.org/708704 | 11:01 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Tests bindep role on all-platforms https://review.opendev.org/708704 | 11:02 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Tests ensure-tox on all-platforms https://review.opendev.org/708642 | 11:02 |
| pabelanger | zbr: yes, set job.attempts to 1 | 11:26 |
| pabelanger | https://zuul-ci.org/docs/zuul/reference/job_def.html#attr-job.attempts | 11:26 |
| zbr | thanks | 11:36 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Refactor merge_client in tests https://review.opendev.org/709676 | 11:42 |
| *** jcapitao is now known as jcapitao_lunch | 11:54 | |
| pabelanger | clarkb: over the last few days, I have seen an increase of email bouncing from you email address: http://paste.openstack.org/show/789969/ | 11:58 |
| *** rfolco has joined #zuul | 11:58 | |
| *** jpena is now known as jpena|lunch | 12:01 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Refactor nodepool in tests https://review.opendev.org/709703 | 12:07 |
| *** AJaeger has quit IRC | 12:12 | |
| *** AJaeger has joined #zuul | 12:17 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Refactor zookeeper in tests https://review.opendev.org/709709 | 12:23 |
| openstackgerrit | Merged zuul/zuul master: Ensure job.queued is correct after reconfig https://review.opendev.org/709559 | 12:28 |
| *** bolg has quit IRC | 12:31 | |
| *** bolg has joined #zuul | 12:32 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: WIP: Store unparsed branch config in Zookeeper https://review.opendev.org/705716 | 12:36 |
| *** rishabhhpe has joined #zuul | 12:57 | |
| rishabhhpe | Hi All , my SSH is working from zuul-nodepool server to spawned nova instance .. but ansible is not taking it and saying unreachable but manually i am able to ssh in to the instance | 12:58 |
| rishabhhpe | for error: http://paste.openstack.org/show/789977/ | 12:59 |
| *** sshnaidm is now known as sshnaidm|bbl | 12:59 | |
| *** rlandy has joined #zuul | 13:00 | |
| pabelanger | from you zuul-executor, can you manually SSH into the node from nodepool? | 13:02 |
| pabelanger | your* | 13:02 |
| rishabhhpe | pabelanger: yes i am able to do it | 13:07 |
| rishabhhpe | it was working fine till today morning but now suddenly it is throwing this error | 13:07 |
| pabelanger | rishabhhpe: possible it isn't at routing or network issue? | 13:08 |
| pabelanger | or, maybe SSH hostkey isn't setup properly | 13:08 |
| rishabhhpe | but it was working fine till morning i did not touched anything yet | 13:09 |
| pabelanger | sounds like network issue | 13:11 |
| pabelanger | or new image build was uploaded an something changed | 13:11 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Consolidate scheduler pause/exit as hibernation https://review.opendev.org/709723 | 13:12 |
| rishabhhpe | pabelanger: yes i changed the image but only the proxy except that everythong is same | 13:15 |
| *** jpena|lunch is now known as jpena | 13:15 | |
| *** jcapitao_lunch is now known as jcapitao | 13:15 | |
| *** jamesmcarthur has joined #zuul | 13:27 | |
| *** jamesmcarthur has quit IRC | 13:35 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: config: add tenant.toDict() method and REST endpoint https://review.opendev.org/621344 | 13:49 |
| tristanC | zuul-maint: could you please add https://review.opendev.org/621344 to your review stack. it's a much needed feature that would help us deprecate and safely remove labels. | 13:55 |
| *** rishabhhpe has quit IRC | 14:00 | |
| tristanC | one year ago, corvus said we shouldn't keep such feature in the long term, but we regurlarly receive request about such feature, e.g. to know which jobs use which labels | 14:04 |
| tristanC | thus please let me know if i should abandon that change and implement the feature in a separate project instead | 14:07 |
| *** avass has quit IRC | 14:23 | |
| *** Goneri has joined #zuul | 14:24 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Scheduler's pause/resume functionality https://review.opendev.org/709735 | 14:27 |
| tobiash | tristanC: corvus' concern was about response size. Was there a discussion about that? | 14:39 |
| tobiash | fbo: yes, we're using it in production (with some deployment specific additions). We once had a running demo instance but I don't know the current state of it. I have to ask my collegues about it | 14:40 |
| swest | fbo: I had to check, but the test instance is still up: http://ad43b5505180211e99b0002e3f6a3815-573606705.eu-west-1.elb.amazonaws.com/ | 14:42 |
| fbo | tobiash: swest thanks I'll have a look ! | 14:44 |
| corvus | fbo, swest: we were working on moving more of that functionality into zuul too... i think that would still be welcome | 14:44 |
| tristanC | tobiash: the change implements a zuul-web cache to prevent gearman bus saturation. i don't think the size is an issue. I think the real problem is that the schemas may not be stable and we are not commited to support such interface | 14:45 |
| *** harrymichal has joined #zuul | 14:46 | |
| tobiash | corvus: yes, there are a few changes in review that probably need updates (and reviews): https://review.opendev.org/#/q/status:open+project:zuul/zuul+branch:master+topic:zubbi-integration | 14:47 |
| *** michael-beaver has joined #zuul | 14:57 | |
| swest | corvus: tobiash: there was not much feedback on those changes so it came to nothing | 14:59 |
| clarkb | pabelanger: it is failing because DNS isnt fully configured for that name. I'velet mordred know in -infra | 15:00 |
| corvus | swest: i'm sorry i neglected them, i'll be happy to take a look if you still have time to work on it | 15:00 |
| swest | corvus: to be fair we also did not push for a review :) I'll talk to Felix about it | 15:03 |
| corvus | tristanC, tobiash: i just ran 621344 on the repl for the openstack tenant; it took 19.66 seconds to generate the dictionary data structure; unfortunately there were mapping proxies involved, so it doesn't render to json without further changes, but i ran "repr" on it to get an order of magnitude, it's 137,348,018 bytes. | 15:05 |
| tobiash | corvus, tristanC: what do you think about deferring this to after ha scheduler when zuul-web could pull this directly from zk (of course cached)? | 15:06 |
| corvus | tobiash: yeah, i think that's a good idea | 15:07 |
| corvus | i'm not convinced we're going to want to send a 137MB json response to rest api users even then, but if we do decide to do that, we will at least have the technical ability to do so without impacting the scheduler. | 15:08 |
| *** saneax has quit IRC | 15:09 | |
| corvus | (openstack's status json tops out around 5MB i think? and we thought that was pretty big) | 15:10 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Make test setup_config more pure https://review.opendev.org/709084 | 15:19 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Scheduler test app manager https://review.opendev.org/708812 | 15:19 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Use scheduler manager consistently in tests https://review.opendev.org/709542 | 15:19 |
| tristanC | corvus: when running this change in the rdo tenant, the config fitted in less than 10MB . | 15:21 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Refactor executor_client in tests https://review.opendev.org/709672 | 15:22 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Refactor merge_client in tests https://review.opendev.org/709676 | 15:22 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Refactor nodepool in tests https://review.opendev.org/709703 | 15:22 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Refactor zookeeper in tests https://review.opendev.org/709709 | 15:22 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Consolidate scheduler pause/exit as hibernation https://review.opendev.org/709723 | 15:22 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Scheduler's pause/resume functionality https://review.opendev.org/709735 | 15:22 |
| tristanC | though we dropped that patch in software-factory zuul package, and now we are not able to tell what jobs uses a specific label | 15:23 |
| corvus | tristanC: it's possible that was inflated by a bunch of "mappingproxy()" strings. if so, i agree 10MB seems plausible. | 15:23 |
| corvus | tristanC: after HA scheduler, we won't have to worry about the time to generate the structure, and if it does end up being 10MB for something like openstack, i think that sounds like somethig we should be able to handle | 15:24 |
| corvus | tristanC: how about a narrow solution to that problem? have the scheduler do a reverse mapping of label -> jobs, then provide that on an endpoint for labels? it could cache the results until the next reconfig. | 15:26 |
| *** irclogbot_3 has quit IRC | 15:27 | |
| *** johanssone has quit IRC | 15:27 | |
| corvus | tobiash: fyi we just used full-reconfigure to recover from an inconsistent state in opendev (from a period where a repo was broken in gerrit). so apparently that can happen. :) | 15:29 |
| tristanC | corvus: that would work, but looking at the zubbi stack, we also need a tenant -> jobs list, and i think pipeline -> projects is also a useful info. thus we might end up with multiple endpoint for each problems | 15:29 |
| *** johanssone has joined #zuul | 15:29 | |
| *** irclogbot_2 has joined #zuul | 15:29 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: WIP: Store unparsed branch config in Zookeeper https://review.opendev.org/705716 | 15:30 |
| tristanC | corvus: to know the jobs <-> labels relationship, it involves quite a bit of processing (e.g. look at earch ppc, then each pipeline, then each job variants, ...) | 15:31 |
| tristanC | thus, i'd rather implement https://review.opendev.org/621344 as a separate service to provide global config introspection | 15:32 |
| tobiash | tristanC, corvus: sounds like we want a graphql endpoint after ha scheduler ;) | 15:32 |
| corvus | tristanC, swest: yes, i think we need a roadmap for the zubbi work (either an email or a spec). because if it's fully integrated into zuul, we may not need to ship so much data. zubbi needs all of that now to perform processing, but if we just integrated it into the zuul web ui, it may be simpler.... | 15:32 |
| *** rfolco has quit IRC | 15:32 | |
| corvus | tobiash: :( | 15:32 |
| corvus | i don't think we need an external service that takes all of zuul's config, processes it, mutates it, and presents it to the user for viewing. that's *zuul's* job. | 15:33 |
| *** harrymichal has quit IRC | 15:33 | |
| tristanC | tobiash: i think a graphql endpoint would still custom function (e.g. as corvus suggested) to crunch the config data | 15:33 |
| corvus | so let's look at what things we want to show the user and figure out the best way to do that. | 15:34 |
| corvus | take labels: it's pretty intuitive to show a list of labels to the user, and in each label, show which jobs/nodesets use it. so why not have zuul calculate that once, cache the result, and return it? | 15:35 |
| corvus | same thing for roles (the killer feature for zubbi) -- move that into zuul itself and provide a roles endpoint that maps roles to jobs | 15:35 |
| tobiash | makes sense | 15:36 |
| tristanC | corvus: http://lists.zuul-ci.org/pipermail/zuul-discuss/2018-November/000640.html lists some of those things | 15:36 |
| tristanC | and the thread continued http://lists.zuul-ci.org/pipermail/zuul-discuss/2018-December/000643.html | 15:37 |
| corvus | at least i'm consistent :) | 15:38 |
| tristanC | corvus: indeed :) | 15:38 |
| tristanC | so, for labels, we would have /label/{name}/projects to return the project list, and /label/{name}/jobs to return the list of job | 15:39 |
| corvus | maybe start with /jobs -- we know the use case for that. /projects might be interesting too, but it's also one-click away from /label/{name}/jobs, so we might be able to live without it (but can add it if we find it would be helpful)? | 15:41 |
| *** bolg has quit IRC | 15:48 | |
| *** jamesmcarthur has joined #zuul | 15:55 | |
| *** jamesmcarthur has quit IRC | 15:55 | |
| *** jamesmcarthur_ has joined #zuul | 15:55 | |
| *** harrymichal has joined #zuul | 15:58 | |
| *** rishabhhpe has joined #zuul | 16:18 | |
| mordred | corvus: https://review.opendev.org/#/c/709306/ is a cleanup patch from the zuul-jobs stuff from the other day that might otherwise just fall through the cracks | 16:20 |
| corvus | mordred: thanks! | 16:25 |
| *** rfolco has joined #zuul | 16:27 | |
| *** Defolos has quit IRC | 16:41 | |
| *** mattw4 has joined #zuul | 16:44 | |
| *** harrymichal has quit IRC | 16:45 | |
| *** harrymichal has joined #zuul | 16:45 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Fix cleanup of symlink fixtures https://review.opendev.org/709306 | 16:46 |
| Shrews | corvus: qq on 709261 for you | 17:09 |
| *** jamesmcarthur has joined #zuul | 17:10 | |
| corvus | Shrews: thx, replied | 17:10 |
| Shrews | corvus: cool, so no stream is a hard fail. | 17:12 |
| corvus | yep | 17:12 |
| corvus | well, at least, no port forward | 17:12 |
| corvus | the port forward happens before anything on the remote node. so all we're asserting here is that kubectl is installed and has access to the cluster to establish a port-forward | 17:13 |
| corvus | the container can still completely fail to actually stream any output and that's fine | 17:13 |
| *** jamesmcarthur_ has quit IRC | 17:13 | |
| corvus | that's not the executor's problem though :) | 17:13 |
| Shrews | corvus: lgtm! | 17:17 |
| *** harrymichal has quit IRC | 17:24 | |
| rishabhhpe | Hi All , can u please help me in resolving this error .. i tried installing openssh with the disk builder but still it did not helped out for error please see here http://paste.openstack.org/show/789977/ | 17:27 |
| *** tjgresha has quit IRC | 17:30 | |
| zenkuro | hello, I was making zuul from scratch guide. And I have few questions: how to properly intdgrate mysql into zuul | 17:33 |
| zenkuro | Ive installled mysql, created zuul db and granted access writes to a zuul user along side with puting ot all including a password in zuul config | 17:35 |
| *** chandankumar is now known as raukadah | 17:35 | |
| zenkuro | but fro some reson db where not started to operate. creation of tables(like in software factory) didnt help as well | 17:36 |
| corvus | zenkuro: can you run the scheduler with the "-d" argument to see debug output, then see if you see a line like "Current migration revision" in the output? | 17:39 |
| *** evrardjp has quit IRC | 17:39 | |
| *** evrardjp has joined #zuul | 17:39 | |
| corvus | rishabhhpe: if i understand correctly, you are using diskimage builder (with nodepool-builder?) to build cloud images with zuul's ssh public key already installed, but the zuul executor is unable to connect to instances launched from that image? | 17:41 |
| rishabhhpe | corvus: yes i passed the pub and private key both for disk-image-builder able to ssh from zuul executor machine but when the same is happened with the help zuul playbook it is not going getting unreachable message | 17:42 |
| rishabhhpe | it was working till morning but from evening onwards i am stuck with this error ... not sure but from my end no changes are done | 17:43 |
| corvus | rishabhhpe: where on the executor is the private key installed? | 17:44 |
| corvus | (what's the full path) | 17:44 |
| rishabhhpe | corvus: yes on executor it is there | 17:44 |
| corvus | rishabhhpe: what is the full path to the private key file? | 17:44 |
| rishabhhpe | i have it in my service user ~/.ssh/ path | 17:45 |
| corvus | full path. no ~ | 17:45 |
| *** rishabhhpe has left #zuul | 17:46 | |
| *** rishabhhpe has joined #zuul | 17:46 | |
| rishabhhpe | corvus: /home/openstack-ci/.ssh/id_rsa | 17:46 |
| rishabhhpe | but my job got triggered today morning although it failed in between but i was able to pass this step . | 17:47 |
| corvus | rishabhhpe: and you run zuul-executor as openstack-ci, and "echo ~openstack-ci" is "/home/openstack-ci" ? | 17:50 |
| rishabhhpe | Yes .. | 17:50 |
| rishabhhpe | i am running my services as sudo | 17:50 |
| rishabhhpe | not as openstack-ci | 17:50 |
| corvus | rishabhhpe: is the effective user id of the zuul-executor process root or openstack-ci ? | 17:51 |
| openstackgerrit | Merged zuul/zuul-jobs master: Fix install-docker on centos-8 https://review.opendev.org/709184 | 17:52 |
| Shrews | might also want to check the value of https://zuul-ci.org/docs/nodepool/configuration.html#attr-diskimages.username | 17:52 |
| rishabhhpe | shrews: i am passing the user details in nodepool.yaml file | 17:53 |
| *** jpena is now known as jpena|off | 17:55 | |
| rishabhhpe | corvus: its effective user id is root only as i am starrting it with sudo | 17:55 |
| corvus | rishabhhpe: later, you should consider not running it as root; it should not be necessary and it will improve security. but we can continue to debug the current situtation. | 17:56 |
| corvus | rishabhhpe: can you check the zuul-executor logs and search for the line "Added SSH Key" and paste it here? | 17:57 |
| rishabhhpe | corvus: please see it here : http://paste.openstack.org/show/789998/ | 17:59 |
| rishabhhpe | i can see my key got replaced with root from today onwards | 17:59 |
| rishabhhpe | but how i can make sure that it should take the same key again | 17:59 |
| corvus | rishabhhpe: by default it uses ~/.ssh/id_ | 18:00 |
| corvus | oops | 18:00 |
| corvus | rishabhhpe: by default it uses ~/.ssh/id_rsa. so it looks like you may have switched users | 18:00 |
| corvus | rishabhhpe: so maybe the easiest and best solution is to go ahead and get it running as the openstack-ci user; that should solve this problem and is better security | 18:00 |
| rishabhhpe | corvus: from my end i am using the same configuration only | 18:01 |
| rishabhhpe | let me try that | 18:01 |
| corvus | rishabhhpe: (however, if you need to, you can specify the key path in zuul.conf with this option: https://zuul-ci.org/docs/zuul/discussion/components.html#attr-executor.private_key_file ) | 18:02 |
| rishabhhpe | and if i am trying to start it with the openstack-ci user i am landing in to other troubles like module pip not found | 18:02 |
| zenkuro | corvus: hm... cant connect to mysqk server name or server not known | 18:03 |
| rishabhhpe | corvus: in the above link you mentioned in that case no need to switch the user ? | 18:03 |
| *** sshnaidm|bbl is now known as sshnaidm | 18:04 | |
| corvus | rishabhhpe: correct. but you should really work on fixing the install so it doesn't require root. | 18:04 |
| rishabhhpe | OK Sure | 18:04 |
| *** harrymichal_ has joined #zuul | 18:05 | |
| corvus | zenkuro: maybe check the format of the dburi -- if you're using mysql, it should look something like this: dburi=mysql+pymysql://user:pass@hostname/database | 18:05 |
| corvus | zenkuro: and of course, check that you can use the mysql client to connect to that hostname from the zuul scheduler host | 18:06 |
| *** igordc has joined #zuul | 18:10 | |
| rishabhhpe | corvus: i had changes the zuul.conf file like this but still it is taking the same /root/.ssh/id_rsa file only .. please check if i had set it wrong | 18:18 |
| rishabhhpe | http://paste.openstack.org/show/789999/ | 18:20 |
| corvus | rishabhhpe: the name of the option is "private_key_file" not "sshkey" see https://zuul-ci.org/docs/zuul/discussion/components.html#attr-executor.private_key_file | 18:25 |
| rishabhhpe | ohh my bad | 18:25 |
| *** jamesmcarthur has quit IRC | 18:26 | |
| *** michael-beaver has quit IRC | 18:27 | |
| *** jcapitao has quit IRC | 18:29 | |
| tristanC | corvus: iiuc https://review.opendev.org/708860 would fix https://zuul.opendev.org/t/zuul/builds?job_name=zuul-operator-promote-image . in that case, could you please have a look | 18:32 |
| tristanC | corvus: i think the current zuul-operator repo is mostly usable and i'd like to give it a try and document it's current usage (e.g. run this kubectl command to get a zuul) | 18:33 |
| *** jamesmcarthur has joined #zuul | 18:38 | |
| *** jamesmcarthur has quit IRC | 18:39 | |
| *** jamesmcarthur has joined #zuul | 18:42 | |
| *** jamesmcarthur has quit IRC | 18:43 | |
| *** rishabhhpe has quit IRC | 18:46 | |
| *** jamesmcarthur has joined #zuul | 18:56 | |
| *** harrymichal_ has quit IRC | 18:56 | |
| corvus | tristanC: thanks, i'll look after the infra meeting; i just did a pass of changes in the operator repo and noticed this https://review.opendev.org/705589 is blocking https://review.opendev.org/702758 | 19:03 |
| *** Defolos has joined #zuul | 19:03 | |
| tristanC | corvus: trying the port-forward change locally, it seems like kubectl requires socat on the executor host too | 19:16 |
| corvus | tristanC: yes, i think i read that | 19:16 |
| corvus | is that missing from our images? | 19:17 |
| tristanC | i don't know, but that is a new requirement | 19:17 |
| *** igordc has quit IRC | 19:17 | |
| corvus | if we want to approach this more conservatively, we can probably update the change to bulk-output the stdout/stderr at the end of the command if kubectl didn't work, and start logging warnings. | 19:18 |
| corvus | (it's unclear to me whether there are many users that are both using pods and don't have kubectl+socat on their executor images (or are not using upstream container images where we can ensure that this change lands with those)) | 19:19 |
| *** sshnaidm has quit IRC | 19:28 | |
| *** jamesmcarthur has quit IRC | 19:28 | |
| *** sshnaidm has joined #zuul | 19:28 | |
| *** jamesmcarthur has joined #zuul | 19:32 | |
| zenkuro | corvus: thanks! error was in config. And Yes looks like mysql creates all necessary tables in db by itself | 19:34 |
| *** harrymichal has joined #zuul | 19:34 | |
| *** harrymichal has quit IRC | 19:42 | |
| corvus | zenkuro: yay! | 19:45 |
| *** igordc has joined #zuul | 19:49 | |
| *** jamesmcarthur has quit IRC | 19:56 | |
| *** jamesmcarthur has joined #zuul | 20:03 | |
| *** jamesmcarthur has quit IRC | 20:08 | |
| *** jamesmcarthur has joined #zuul | 20:12 | |
| *** rfolco has quit IRC | 20:54 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Evaluate CODEOWNERS settings during canMerge check https://review.opendev.org/644557 | 20:59 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Refactor branch protection test infrastructure https://review.opendev.org/709834 | 20:59 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Test canMerge also with checks api https://review.opendev.org/709835 | 20:59 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Optimize canMerge using graphql https://review.opendev.org/709836 | 20:59 |
| zenkuro | corvus: but for some reason build and buld set tabs are showing nothing besides 'Loading...' | 21:00 |
| zenkuro | If I want to commit somerhing to documentation should I create separate brunch or just push it to the mastef? | 21:02 |
| mordred | zenkuro: either is fine. when you run git review it will submit it to gerrit as a change aimed at the master branch | 21:03 |
| *** jamesmcarthur has quit IRC | 21:03 | |
| zenkuro | mordred: thanks | 21:04 |
| *** igordc has quit IRC | 21:08 | |
| corvus | zenkuro: check to make sure that zuul-web has the same sql configuration information in zuul.conf as zuul-scheduler does (you can use the same conf file for both), and that you have restarted zuul-web after you got everything working. | 21:10 |
| corvus | zenkuro: check the zuul-web logs for any errors. i'm also not positive what happens when you visit those tabs with a working database but no data in it; it's possible that case may not be handled well | 21:11 |
| *** harrymichal has joined #zuul | 21:14 | |
| *** igordc has joined #zuul | 21:16 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Optimize canMerge using graphql https://review.opendev.org/709836 | 21:17 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Evaluate CODEOWNERS settings during canMerge check https://review.opendev.org/644557 | 21:17 |
| *** igordc has quit IRC | 21:18 | |
| *** igordc has joined #zuul | 21:18 | |
| *** jamesmcarthur has joined #zuul | 21:18 | |
| clarkb | corvus: tobiash: out of curiousity why store teh times database in sql and not in zk? (I'm reading the ha scheduler spec now) | 21:20 |
| clarkb | is that due to the content size and znode size limits/ | 21:20 |
| corvus | clarkb: yeah, i mean, it's essentially there anyway; we can drop it as a separate thing once builds in sql are implied anyway | 21:21 |
| tobiash | clarkb: the idea was that the info is already there and we save complexity managing our own binary times data format in zk | 21:21 |
| clarkb | oh right the data is already there. Got it | 21:21 |
| *** igordc has quit IRC | 21:26 | |
| clarkb | tobiash: corvus ok I left a couple commments but mostly about thinking ahead and I think the spec itself is fine | 21:30 |
| *** zenkuro24 has joined #zuul | 21:39 | |
| *** zenkuro24 has quit IRC | 21:40 | |
| *** zenkuro13 has joined #zuul | 21:40 | |
| *** harrymichal has quit IRC | 21:41 | |
| *** mattw4 has quit IRC | 21:43 | |
| corvus | tobiash: do you know what the changes between patchset 17 and 18 are? | 21:43 |
| tobiash | corvus: that should be only a rebase due to the index conflict | 21:44 |
| corvus | ok cool, thx | 21:44 |
| corvus | i think i'll carry over the +2s and +W it now | 21:45 |
| tobiash | it conflicted with the circular dep merge | 21:45 |
| tobiash | \o/ | 21:45 |
| zenkuro13 | corvus: you where right there is an issue with the web https://softwarefactory-project.io/paste/show/1743/. and is there any separate zuul.conf for web besides general zuul conf? | 21:51 |
| corvus | zenkuro13: what was the issue? they are designed to be able to use the same config file, but you can use a different one with the "-c" argument. but if you find that's necessary, i'm curious why | 21:53 |
| zenkuro13 | corvus: I think Im using same config(since I havent specified opposit). | 21:55 |
| corvus | zenkuro13: the error you pasted just looks like a messy stop -- was there anything before that? | 21:55 |
| zenkuro13 | manual service termination and start | 21:56 |
| *** rfolco has joined #zuul | 22:07 | |
| *** jamesmcarthur has quit IRC | 22:08 | |
| *** mattw4 has joined #zuul | 22:18 | |
| *** jamesmcarthur has joined #zuul | 22:27 | |
| corvus | tobiash: implied sql reporters is going to be a drag on tests; we may need to do something clever there like mock it out, or create the tables in a test fixture so we don't have to run all the migrations. | 22:30 |
| *** jamesmcarthur has quit IRC | 22:32 | |
| tobiash | yes, or maybe we find an in memory sql backend for the tests | 22:33 |
| tristanC | iirc sqlite mostly works for zuul | 22:33 |
| tobiash | seems like sqlite supports an in-memory mode | 22:36 |
| tobiash | so maybe it's possible to create that as s fixture with pre created tables | 22:37 |
| corvus | my biggest concern is the migrations (that's slow even in sqlite), but secondarily, yeah, maybe sqlite in-memory for most tests and real dbs for the current db-specific tests | 22:37 |
| mordred | corvus: I like the idea of having a $something that just creates the tables | 22:38 |
| *** mattw4 has quit IRC | 22:38 | |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Store build.error_detail in SQL https://review.opendev.org/709857 | 22:39 |
| mordred | corvus: it could even be the result of "run migrations on empty database, run mysqldump to capture the simple schema creation commands, run the dump as the test fixture" | 22:39 |
| corvus | mordred: yeah | 22:39 |
| mordred | corvus: which we could either figure out how to run/generate at the top of a test run - or we could have a script to generate it when we add new db migrations ... or something else smarter | 22:40 |
| *** michael-beaver has joined #zuul | 22:40 | |
| corvus | ++ | 22:40 |
| openstackgerrit | Merged zuul/zuul master: Add spec for scale out scheduler https://review.opendev.org/621479 | 22:41 |
| tristanC | maybe there is a way to re-use a copy of the sqlite memory after the migration is performed? | 22:43 |
| mordred | tobiash: \o/ scale out scheduler! | 22:46 |
| mordred | corvus: question I didn't think to ask in the spec ... | 22:47 |
| tobiash | \o/ | 22:47 |
| mordred | tobiash, corvus: it calls for doing things in a series of discreet steps ... whichis great. step #1 is "required db" - do we land that patch and tag a 4.0.0 and just have the other steps be followup point releases? | 22:47 |
| *** mattw4 has joined #zuul | 22:47 | |
| tobiash | corvus: thinking about that I think the easiest would be to just mock the db away as most tests don't need that data | 22:49 |
| corvus | mordred: that's a good question... | 22:49 |
| corvus | tobiash: i'd be ok with that | 22:49 |
| tobiash | mordred: the order doesn't need to be strict, but I think starting with a release with mandatory sql is a good start | 22:50 |
| corvus | mordred, tobiash: i'd be okay with a feature branch if we want. i know we hate them, but this might be appropriate. | 22:50 |
| mordred | corvus: weirdly the only issue I can see with just doing a 4.0 with requred-db but none of the rest is one of marketing | 22:51 |
| mordred | which I'm not convinced is important ... just may be a thing we want to consider | 22:51 |
| corvus | as long as there's only one breaking change | 22:51 |
| corvus | so if we did that, i think we'd probably want to at least have all the components connect to zk even if they idle | 22:52 |
| tobiash | corvus: I'd like to avoid a feature branch as long as we don't need it ( e.g. sql and test refactorings could land independently) however at the end we might end up either with a feature branch or a stack of changes doing most of the zk stuff | 22:52 |
| mordred | ++ | 22:52 |
| mordred | I think zk connections - and also forcing ssl/auth on the zk connections is the other deployer facing breaking change right? but we could do ssl/auth, zk connections and required-ssl in the 4.0 - even with idle zk connections - and that would still be a pretty small change? | 22:53 |
| corvus | mordred: agreed | 22:54 |
| tobiash | yes, makes sense | 22:54 |
| mordred | maybe add the ability to configure zk connections for all components - that don't get used ... cut a release with that - and then send out a warning "you should go update your deployments to enable this" | 22:54 |
| mordred | so that people can take the update, then roll out any new config that's needed, then take the 4.0 and never actually be broken | 22:55 |
| corvus | https://storyboard.openstack.org/#!/story/2007192 doesn't seem very responsive to me | 22:55 |
| corvus | https://etherpad.openstack.org/p/zuulv4 | 22:56 |
| corvus | mordred: i think all components have the ability to have zk configuration | 22:56 |
| corvus | (the zk config is not component-specific) | 22:56 |
| mordred | ah - good point | 22:56 |
| corvus | so that's done! :) | 22:56 |
| corvus | but let's say that idea applies to zk auth though | 22:57 |
| mordred | yeah | 22:57 |
| mordred | and any intra-node firewalls people have | 22:57 |
| mordred | corvus: I feel like there was one more smallish breaking change we talked about bundling into 4.0 but I can't remember what it was | 23:01 |
| corvus | mordred: maybe old-style depends-on? | 23:02 |
| mordred | maybe? | 23:02 |
| mordred | corvus: also maybe dropping python 3.5 support (although that's still what opendev is using) | 23:03 |
| mordred | I don't think either of those were what I'm trying to remember though | 23:03 |
| corvus | i think that's a pretty cool plan | 23:04 |
| mordred | I agree | 23:04 |
| mordred | SpamapS, pabelanger: if you're around, could you look at the etherpad above? | 23:04 |
| corvus | that should mean that if you follow the recommendations to get the zk connections set up before upgrading to 4.0, everyone can more or less CD along together into zuul v4 without any major breaking changes/downtime (and, by the end, zero downtime at all ;) | 23:06 |
| mordred | \o/ | 23:07 |
| *** Defolos has quit IRC | 23:09 | |
| tobiash | mordred: ++ for dropping 3.5 :) | 23:09 |
| *** Defolos has joined #zuul | 23:10 | |
| mordred | we'd need to get opendev deploying from the container images to do that ... I think pretty much everyone else is running >3.5 already | 23:10 |
| *** armstrongs has joined #zuul | 23:12 | |
| clarkb | fwiw I don't think dropping python3.5 requires a major version bump | 23:17 |
| clarkb | (so maybe decoupling it from this process is worthwhile) | 23:17 |
| *** armstrongs has quit IRC | 23:22 | |
| mordred | clarkb: no? I mean - it is a breaking change if you're running on 3.5 | 23:25 |
| mordred | clarkb: I think we've been defining "breaking change" as "something that an operator of zuul needs to take positive action on" | 23:26 |
| *** Defolos has quit IRC | 23:26 | |
| clarkb | mordred: I think I see it more like the ansible bunps | 23:26 |
| clarkb | it is following up stream cadence and while it does at times require action to be taken there are multiple available paths forward | 23:26 |
| corvus | maybe it's "something that most/all operators need to take action on?" i dunno, it's a good question :) | 23:26 |
| mordred | clarkb: yah - but we haven't removed support for old ansibles either | 23:26 |
| clarkb | mordred: we have we removed 2.5 | 23:26 |
| mordred | oh we did? cool | 23:27 |
| mordred | but still -that's mostly reflecting a reality that's happening outside of the zuul context - the ansible community is ceasing to support that and we're following suit | 23:27 |
| mordred | anywho - I could go either way | 23:27 |
| clarkb | I also dont 'really see the urgency | 23:28 |
| clarkb | if zuul was using asyncio I would want to push forward with python versions but zuul has done the opposite | 23:28 |
| mordred | maybe let's say "if we've gotten opendev onto the containers, we could consider dropping 3.5 when we cut 4.0" | 23:28 |
| mordred | there's a couple of nice things in later python3's it would be nice to be able to choose to use - it's totally not urgent | 23:29 |
| mordred | just at _some_ point it's something we should do | 23:29 |
| *** Goneri has quit IRC | 23:32 | |
| *** tosky has quit IRC | 23:52 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!