| *** kmalloc has quit IRC | 00:18 | |
| *** maxamillion has quit IRC | 00:18 | |
| *** donnyd has quit IRC | 00:18 | |
| *** tdasilva has quit IRC | 00:18 | |
| *** tdasilva has joined #zuul | 00:19 | |
| *** donnyd has joined #zuul | 00:20 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add Openshift SCC and functional test https://review.opendev.org/702758 | 00:20 |
|---|---|---|
| *** maxamillion has joined #zuul | 00:20 | |
| *** kmalloc has joined #zuul | 00:20 | |
| *** wxy-xiyuan has joined #zuul | 00:21 | |
| *** zxiiro has joined #zuul | 00:31 | |
| *** tdasilva has quit IRC | 00:42 | |
| *** samccann has quit IRC | 00:42 | |
| *** clayg has quit IRC | 00:42 | |
| *** wxy-xiyuan has quit IRC | 00:42 | |
| *** maxamillion has quit IRC | 00:42 | |
| *** donnyd has quit IRC | 00:43 | |
| *** gmann has quit IRC | 00:43 | |
| *** zxiiro has quit IRC | 00:43 | |
| *** jtanner has quit IRC | 00:44 | |
| *** iamweswilson has quit IRC | 00:45 | |
| *** ofosos has quit IRC | 00:46 | |
| *** mnasiadka has quit IRC | 00:46 | |
| *** dcastellani has quit IRC | 00:46 | |
| *** gundalow has quit IRC | 00:46 | |
| *** dustinc has quit IRC | 00:46 | |
| *** portdirect has quit IRC | 00:46 | |
| *** Shrews has quit IRC | 00:47 | |
| *** guilhermesp has quit IRC | 00:47 | |
| *** mnaser has quit IRC | 00:49 | |
| *** tributarian has quit IRC | 00:51 | |
| *** ChrisShort has quit IRC | 00:52 | |
| *** johnsom has quit IRC | 00:52 | |
| tristanC | corvus: the zuul-scheduler logs running in openshift: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_ac5/702758/3/check/zuul-operator-functional-openshift/ac5c490/docker/k8s_scheduler_zuul-scheduler-0_myproject_6ac410ca-37f8-11ea-abac-fa163e5a906b_0.txt | 00:52 |
| *** webknjaz has quit IRC | 00:53 | |
| tristanC | though the operator needs to uses Route instead of Ingress (it failed with 'Failed to find exact match for networking.k8s.io/v1beta1.Ingress' in https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_ac5/702758/3/check/zuul-operator-functional-openshift/ac5c490/docker/k8s_operator_zuul-operator-58bf75bff4-lh97j_myproject_3fd5c70e-37f8-11ea-abac-fa163e5a906b_0.txt ) | 00:53 |
| *** ChrisShort has joined #zuul | 00:54 | |
| *** ChrisShort has quit IRC | 01:04 | |
| *** jamesmcarthur has joined #zuul | 01:14 | |
| *** jamesmcarthur_ has joined #zuul | 01:16 | |
| *** jamesmcarthur has quit IRC | 01:17 | |
| *** kmalloc has quit IRC | 01:17 | |
| *** mnasiadka has joined #zuul | 01:18 | |
| *** mattw4 has joined #zuul | 01:20 | |
| *** mnasiadka has quit IRC | 01:29 | |
| *** jamesmcarthur_ has quit IRC | 01:40 | |
| *** mattw4 has quit IRC | 01:41 | |
| *** igordc has quit IRC | 01:42 | |
| *** jamesmcarthur has joined #zuul | 01:42 | |
| *** jamesmcarthur has quit IRC | 01:47 | |
| *** sgw has quit IRC | 02:06 | |
| *** jamesmcarthur has joined #zuul | 02:12 | |
| *** jamesmcarthur_ has joined #zuul | 02:16 | |
| *** jamesmcarthur has quit IRC | 02:16 | |
| *** jamesmcarthur_ has quit IRC | 02:26 | |
| *** jamesmcarthur has joined #zuul | 02:34 | |
| *** bhavikdbavishi has joined #zuul | 02:54 | |
| *** bhavikdbavishi1 has joined #zuul | 03:04 | |
| *** bhavikdbavishi has quit IRC | 03:06 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 03:07 | |
| *** jamesmcarthur has quit IRC | 03:41 | |
| *** jamesmcarthur has joined #zuul | 03:42 | |
| *** jamesmcarthur has quit IRC | 03:48 | |
| *** jamesmcarthur has joined #zuul | 03:56 | |
| *** jamesmcarthur_ has joined #zuul | 03:59 | |
| *** jamesmcarthur has quit IRC | 03:59 | |
| *** rlandy has quit IRC | 04:07 | |
| *** openstackgerrit has quit IRC | 04:12 | |
| *** jtanner has joined #zuul | 04:27 | |
| *** jtanner has quit IRC | 04:33 | |
| *** jamesmcarthur_ has quit IRC | 04:55 | |
| *** jamesmcarthur has joined #zuul | 04:57 | |
| *** jamesmcarthur has quit IRC | 05:02 | |
| *** sgw has joined #zuul | 05:04 | |
| *** jtanner has joined #zuul | 05:09 | |
| *** iamweswilson has joined #zuul | 05:21 | |
| *** jtanner has quit IRC | 05:24 | |
| *** iamweswilson has quit IRC | 05:25 | |
| *** jamesmcarthur has joined #zuul | 05:26 | |
| *** jamesmcarthur has quit IRC | 05:32 | |
| *** tributarian has joined #zuul | 05:32 | |
| *** evrardjp has quit IRC | 05:34 | |
| *** evrardjp has joined #zuul | 05:34 | |
| *** tributarian has quit IRC | 05:38 | |
| *** jtanner has joined #zuul | 05:46 | |
| *** jtanner has quit IRC | 05:52 | |
| *** tributarian has joined #zuul | 05:53 | |
| *** swest has joined #zuul | 05:58 | |
| *** jamesmcarthur has joined #zuul | 06:00 | |
| *** tributarian has quit IRC | 06:00 | |
| *** jamesmcarthur has quit IRC | 06:07 | |
| *** openstackgerrit has joined #zuul | 06:11 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add --validate-tenants option to zuul scheduler https://review.opendev.org/542160 | 06:11 |
| *** saneax has joined #zuul | 06:18 | |
| *** samccann has joined #zuul | 06:30 | |
| *** samccann has quit IRC | 06:33 | |
| *** iamweswilson has joined #zuul | 06:37 | |
| *** iamweswilson has quit IRC | 06:42 | |
| *** jtanner has joined #zuul | 06:44 | |
| *** iamweswilson has joined #zuul | 06:48 | |
| *** dcastellani has joined #zuul | 06:48 | |
| *** wxy-xiyuan has joined #zuul | 06:48 | |
| *** gmann has joined #zuul | 06:48 | |
| *** samccann has joined #zuul | 06:48 | |
| *** kmalloc has joined #zuul | 06:49 | |
| *** mnaser has joined #zuul | 06:49 | |
| *** clayg has joined #zuul | 06:50 | |
| *** ChrisShort has joined #zuul | 06:50 | |
| *** tributarian has joined #zuul | 06:52 | |
| *** donnyd has joined #zuul | 06:52 | |
| *** stevthedev has joined #zuul | 06:53 | |
| *** maxamillion has joined #zuul | 06:58 | |
| *** webknjaz has joined #zuul | 06:58 | |
| *** portdirect has joined #zuul | 07:00 | |
| *** pcaruana has joined #zuul | 07:00 | |
| *** jamesmcarthur has joined #zuul | 07:02 | |
| *** johnsom has joined #zuul | 07:02 | |
| *** dustinc has joined #zuul | 07:03 | |
| *** Shrews has joined #zuul | 07:06 | |
| *** jamesmcarthur has quit IRC | 07:09 | |
| *** mnasiadka has joined #zuul | 07:14 | |
| *** gundalow has joined #zuul | 07:15 | |
| *** evgenyl has joined #zuul | 07:28 | |
| *** wxy-xiyuan has quit IRC | 07:35 | |
| *** tdasilva has joined #zuul | 07:48 | |
| *** guilhermesp has joined #zuul | 07:48 | |
| *** jpena|off is now known as jpena | 08:01 | |
| *** avass has joined #zuul | 08:25 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: OIDCAuthenticator: add capabilities, scope option https://review.opendev.org/702275 | 08:25 |
| *** reiterative has joined #zuul | 08:30 | |
| *** tosky has joined #zuul | 08:32 | |
| *** dmellado has quit IRC | 08:35 | |
| *** dmellado has joined #zuul | 08:37 | |
| *** sgw has quit IRC | 08:39 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: DNM: Try out test sharding for tox-py35 https://review.opendev.org/702473 | 08:44 |
| *** themroc has joined #zuul | 08:50 | |
| *** sgw has joined #zuul | 08:55 | |
| *** themroc has quit IRC | 09:02 | |
| *** jamesmcarthur has joined #zuul | 09:06 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Add spec for scale out scheduler https://review.opendev.org/621479 | 09:09 |
| *** themroc has joined #zuul | 09:09 | |
| *** jamesmcarthur has quit IRC | 09:10 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Add spec for scale out scheduler https://review.opendev.org/621479 | 09:10 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: OIDCAuthenticator: add capabilities, scope option https://review.opendev.org/702275 | 09:28 |
| *** bhavikdbavishi has quit IRC | 09:34 | |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: Cleanup exception logging in static provider https://review.opendev.org/702828 | 09:37 |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: Cleanup exception logging in static provider https://review.opendev.org/702828 | 09:39 |
| *** jangutter has joined #zuul | 09:48 | |
| *** zbr is now known as zbr|rover | 10:09 | |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Report buildset result in MQTT reporter https://review.opendev.org/702838 | 10:27 |
| *** jangutter has quit IRC | 11:29 | |
| *** chandankumar has joined #zuul | 11:41 | |
| *** panda is now known as panda|drappt | 11:46 | |
| openstackgerrit | Benjamin Schanzel proposed zuul/zuul master: Handle Erroneous Cron Strings in TimerDriver https://review.opendev.org/702237 | 11:46 |
| *** bhavikdbavishi has joined #zuul | 11:53 | |
| openstackgerrit | Benjamin Schanzel proposed zuul/zuul master: Handle Erroneous Cron Strings in TimerDriver https://review.opendev.org/702237 | 11:55 |
| *** bhavikdbavishi has quit IRC | 12:03 | |
| *** jpena is now known as jpena|lunch | 12:05 | |
| openstackgerrit | Benjamin Schanzel proposed zuul/zuul master: Allow Passing of Jitter Values in TimerDriver https://review.opendev.org/702854 | 12:08 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Report buildset result in MQTT reporter https://review.opendev.org/702838 | 12:08 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Report buildset result in MQTT reporter https://review.opendev.org/702838 | 12:31 |
| *** bhavikdbavishi has joined #zuul | 12:36 | |
| *** rlandy has joined #zuul | 12:59 | |
| *** jpena|lunch is now known as jpena | 13:10 | |
| *** jamesmcarthur has joined #zuul | 13:15 | |
| openstackgerrit | Benjamin Schanzel proposed zuul/zuul master: Allow Passing of Jitter Values in TimerDriver https://review.opendev.org/702854 | 13:21 |
| *** jamesmcarthur has quit IRC | 13:24 | |
| *** jamesmcarthur has joined #zuul | 13:25 | |
| *** panda|drappt is now known as panda | 13:25 | |
| *** jamesmcarthur has quit IRC | 13:33 | |
| *** electrofelix has joined #zuul | 13:35 | |
| zbr|rover | can we do something about reindexing zuul docs, breaking google search is a real PITA | 13:36 |
| zbr|rover | i cannot force google to reindex because I cannot pass the validation | 13:37 |
| *** bhavikdbavishi has quit IRC | 13:41 | |
| *** ianychoi_ has quit IRC | 13:44 | |
| *** jamesmcarthur has joined #zuul | 13:44 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Import dhall-operator https://review.opendev.org/702104 | 13:45 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Import zuul application from dhall-zuul https://review.opendev.org/702105 | 13:46 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Replace existing operator tasks with the new application https://review.opendev.org/702106 | 13:46 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add OpenShift SCC and functional test https://review.opendev.org/702758 | 13:46 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add custom image override https://review.opendev.org/702866 | 13:46 |
| *** zxiiro has joined #zuul | 14:01 | |
| *** rishabhhpe has joined #zuul | 14:02 | |
| Shrews | zbr|rover: why not just use the search tool on https://zuul-ci.org/docs/zuul/ ? | 14:02 |
| openstackgerrit | Benjamin Schanzel proposed zuul/zuul master: Allow Passing of Jitter Values in TimerDriver https://review.opendev.org/702854 | 14:03 |
| *** jamesmcarthur has quit IRC | 14:04 | |
| Shrews | zbr|rover: also, part of the reason for the reorg is to make things easier to find at a glance (still a work-in-progress, of course). I'd love to know what you're searching for that isn't easily findable from the root level doc page. | 14:08 |
| tristanC | Shrews: i had to use the search tool to find where the spec list landed | 14:15 |
| pabelanger | I'd love something, it TOC, that linked directly to each stanza for configration. eg: pipeline, job, project, project-template. I often look to reference it, but always end up using search because 'job content' doesn't have it. Where 'Project Configuration' does, and my brain never remembers | 14:16 |
| pabelanger | s/it/in | 14:16 |
| tristanC | pabelanger: +1 :) | 14:16 |
| Shrews | pabelanger: tristanC: https://review.opendev.org/701977 | 14:17 |
| Shrews | that was my biggest complaint, too :) | 14:17 |
| pabelanger | \o/ | 14:17 |
| tristanC | Shrews: something i find missing from sphinx is being able to customize the toc. I wonder if we shouldn't create it manually so that we can add subtitle and controls the depth level | 14:19 |
| Shrews | tristanC: yeah, i think mnaser had that issue too. we can debate if the Developer Guide should be it's own section or not (pulling it out of Reference). Don't know if that would make it easier to find. | 14:19 |
| Shrews | tristanC: i don't know how to do that (or if it can even be done). would be nice if we could get more control over it. i searched for options but did not find any solutions | 14:20 |
| tristanC | Shrews: for example, shouldn't we add one more level to the reference section, and group the project config YAML into it's own branch? | 14:20 |
| Shrews | tristanC: i wanted to do that with the config yaml, but you can't change the depth for a single item in the toc | 14:21 |
| Shrews | so we can only affect the entire Reference section, which would expose more levels for all of the docs there | 14:22 |
| Shrews | i'm not pleased with that sphinx shortcoming, tbh | 14:23 |
| tristanC | maybe we could use a update-index script to manually generate the toc using sphinx directive, at least for the main index | 14:23 |
| rishabhhpe | clarkb: fungi: i tried building the image with config drive: true as u suggested in your nodepool.yaml file but still the config field is not getting populated to true and even i changed the nova.conf also as suggested by fungi .. can u please let me know what can be done now | 14:24 |
| *** ianychoi has joined #zuul | 14:25 | |
| fungi | rishabhhpe: hopefully you meant BOOTING the image with config drive: true? | 14:28 |
| Shrews | pabelanger: i think "Job Content" might be poorly named because that's where I always started my search for the job yaml, too. Glad to know it wasn't just me | 14:28 |
| rishabhhpe | clarkb: also the image file shared by you also does not populating the config drive | 14:28 |
| rishabhhpe | fungi: yes | 14:28 |
| fungi | rishabhhpe: building the image has nothing to do with config drive options | 14:28 |
| *** jamesmcarthur has joined #zuul | 14:28 | |
| fungi | rishabhhpe: so this sounds like maybe nova isn't honoring the setting in nodepool. can you share a copy of the nodepool.yaml your launcher uses? if there are usernames and passwords in it, please delete those first | 14:29 |
| rishabhhpe | fungi: yes you are correct .. i am not sure where my configuration is lacking now | 14:29 |
| fungi | it could be something as simple as incorrect indentation | 14:29 |
| rishabhhpe | ok .. so after deleting the username and password do i need to build the dib image again ? | 14:29 |
| zbr|rover | Shrews: i doubt that breaking google search was part of the intended outcome. lets face it, google does much better search any anyone, yet. let's fix it please. | 14:30 |
| fungi | rishabhhpe: no, i was asking if you can share a copy of the nodepool.yaml from your executor with us, just reminding you not to share any usernames and passwords which might be present in that file | 14:31 |
| rishabhhpe | fungi: please find my nodepool.yaml file -: http://paste.openstack.org/show/788471/ .. i do not have username password setting in it | 14:31 |
| Shrews | well that's not a very helpful response | 14:32 |
| rishabhhpe | fungi: got it .. i am not giving any username and password field in it .. it is required to provide that ? | 14:32 |
| fungi | zbr|rover: the hostility is not appreciated here, please keep it civil | 14:33 |
| *** pcaruana has quit IRC | 14:33 | |
| fungi | rishabhhpe: not required, no, i just wanted to be sure you didn't accidentally leak any credentials when sharing a file with us, that's all | 14:33 |
| fungi | rishabhhpe: so, the example we shared with you yesterday was this: https://opendev.org/openstack/project-config/src/branch/master/nodepool/nl01.openstack.org.yaml#L63 | 14:34 |
| fungi | rishabhhpe: setting config-drive in the diskimages list for the provider | 14:34 |
| fungi | at first glance it seems to be done the same in your configuration | 14:35 |
| fungi | rishabhhpe: do you happen to know what version of openstack/nova you're running in that provider? | 14:36 |
| rishabhhpe | fungi: $ sudo pip3 freeze | grep nova | 14:37 |
| rishabhhpe | python-novaclient==16.0.0 | 14:37 |
| rishabhhpe | so settings in my nodepool.yaml are correct according to u ? | 14:37 |
| fungi | rishabhhpe: i mean your openstack cloud, what version of the nova api server | 14:37 |
| fungi | not on your nodepool server | 14:38 |
| *** sgw has quit IRC | 14:38 | |
| fungi | zbr|rover: if you want to help with the search issue, a .htaccess file with permanent redirects for the files which moved in https://review.opendev.org/701608 (per my review comment at the end) might be a good next step | 14:38 |
| rishabhhpe | fungi: this is from my devstack server : http://paste.openstack.org/show/788474/ | 14:39 |
| rishabhhpe | both are same on stack and ci server | 14:39 |
| fungi | rishabhhpe: python-novaclient is a different package than nova. it's possible nova is installed with python2 rather than python3 so try using pip instead of pip3 and see if it's listed | 14:40 |
| rishabhhpe | fungi : please see this http://paste.openstack.org/show/788475/ | 14:42 |
| *** jamesmcarthur has quit IRC | 14:44 | |
| rishabhhpe | i can see its installation under both :( | 14:45 |
| fungi | rishabhhpe: oh, yep, i missed it because it's not installed from a release it's installed from a random git commit | 14:46 |
| fungi | i'll see if i can figure out which nova version that roughly equates to | 14:46 |
| rishabhhpe | fungi: OK, so this all may take time from your end ?if yes can u please mail me the same if i provide my email id | 14:47 |
| fungi | rishabhhpe: that commit seems to be from december 11, after the last nova release, so it's new. that's mainly what i needed to know | 14:50 |
| fungi | just wanted to rule out the possibility that you had an ancient (like 5+ years old) openstack deployment you were trying to connect nodepool to, which might have required some additional effort to integrate with | 14:51 |
| rishabhhpe | fungi: that was a old setup and now i am bringing a complete new setup with latest tools | 14:52 |
| fungi | rishabhhpe: so we're left with trying to figure out why your nodepool launcher, when configured to set config-drive: true, isn't causing nova to attach a config drive even though you're able to manually boot an instance in nova with a config drive | 14:55 |
| fungi | which probably means digging into debug logs for nodepool-launcher and maybe also nova-api | 14:56 |
| rishabhhpe | fungi: yes u find it correct | 14:56 |
| fungi | are you able to find your nodepool-launcher debug log? ours is /var/log/nodepool/launcher-debug.log but where it's written may depend on how you've installed it | 14:57 |
| fungi | i'm looking through ours and i don't think it logs the api requests it's making | 14:58 |
| rishabhhpe | i did not enables the debug logging rather i can provide you nodepool.log | 14:58 |
| rishabhhpe | hope that also works | 14:59 |
| fungi | rishabhhpe: i'm also somewhat tight on time. today is solid with meetings for me, i'm wrapping up with one right now and about to have two at the same time starting in a minute | 14:59 |
| fungi | i'm a bit stumped with your current problem, to be honest | 14:59 |
| rishabhhpe | fungi: no problem u carry on with your meeting we can look at this later .. just let me know the time if we can catch up today . | 15:00 |
| zbr|rover | fungi: before fixing the redirects, we need to trigger reindexing, look at https://stackoverflow.com/questions/9466360/how-to-request-google-to-re-crawl-my-website | 15:01 |
| zbr|rover | usually it happens in less than hour | 15:01 |
| *** sgw has joined #zuul | 15:01 | |
| zbr|rover | i see that we do not have a sitemap.xml file | 15:02 |
| tristanC | zbr|rover: iiuc, htaccess redirects would help any external index, not just google search user | 15:02 |
| zbr|rover | tristanC: yeah,... and depends how many redirects you have to write in that file. | 15:03 |
| fungi | rishabhhpe: maybe someone else in here has ideas on how to troubleshoot why nodepool-launcher isn't successfully requesting that nova attach a config drive. you did restart the nodepool-launcher service after adding that to its configuration, right? it won't see configuration changes automatically and needs a restart to reread its configuration files | 15:04 |
| *** swest has quit IRC | 15:04 | |
| *** jamesmcarthur has joined #zuul | 15:04 | |
| rishabhhpe | yes i restarted it | 15:05 |
| rishabhhpe | fungi: well this is the error i am getting in nodepool.log | 15:06 |
| rishabhhpe | http://paste.openstack.org/show/788477/ | 15:06 |
| mnaser | fwiw i think we should look at the 404s happening right now on zuul-ci.org | 15:08 |
| fungi | rishabhhpe: yeah, that looks consistent with what we've been discussing. if there's no config drive attached to the instance, glean will be unable to determine how to configure the network interfaces and then nodepool will be unable to connect to the node over ssh | 15:08 |
| fungi | mnaser: as in adding redirects for the documents which were renamed/moved, or some other 404s? | 15:09 |
| rishabhhpe | fungi: yes apart from this not other error is coming in hte log | 15:09 |
| mnaser | fungi: yes for the doc re-org | 15:09 |
| mnaser | for example yesterday i was looking for "zuul kubernetes spec" and the #1 result on google was https://zuul-ci.org/docs/zuul/developer/specs/container-build-resources.html | 15:10 |
| mnaser | searching "zuul user guide" yields https://zuul-ci.org/docs/zuul/user/index.html | 15:10 |
| *** reiterative has quit IRC | 15:11 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-website master: Add verification file https://review.opendev.org/702888 | 15:12 |
| *** jamesmcarthur has quit IRC | 15:21 | |
| zbr|rover | fungi: mnaser ^ once me (or someone else) gets verified on google search console, it is possible to add others. The console gives very good insights on how people are accessing the site. | 15:24 |
| *** bhavikdbavishi has joined #zuul | 15:29 | |
| mnaser | ya we really should do that | 15:31 |
| zbr|rover | that search issue displeases me greatly and I give a very simple example why our search sucks big time. Today i was looking on how to debug how zuul decides which jobs to run or not, i remembered someone told me about an option that I can configure. I tried searching using "debug". | 15:33 |
| *** bhavikdbavishi1 has joined #zuul | 15:34 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Replace existing operator tasks with the new application https://review.opendev.org/702106 | 15:35 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add custom image override https://review.opendev.org/702866 | 15:35 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add OpenShift SCC and functional test https://review.opendev.org/702758 | 15:35 |
| *** bhavikdbavishi has quit IRC | 15:35 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 15:35 | |
| zbr|rover | google used to work fine, so i was forced to use our quick search, got 5-6 answers and the samples where not convincing. clicking on the did not sent me to the location of debug page, but to the top of the page. I tried to use Firefox native search in page, and guess what found nothing even if the "debug" text was on the page, probably because the theme did override the browser search functionality. | 15:36 |
| zbr|rover | mainly we how have two broken ways to find information about zuul | 15:36 |
| pabelanger | I imagine, because google isn't free tool, we may not want to depend on it | 15:37 |
| zbr|rover | pabelanger: lucky for me I was able to get the right link from you. thanks! I will pay with a beer on first occasion. | 15:38 |
| zbr|rover | pabelanger:i bet electricity is not a free either for most of us, but we still pay the bills. | 15:38 |
| corvus | pabelanger, zbr|rover: this is starting to sound like it's getting off-topic | 15:40 |
| corvus | i don't think anyone has suggested that we actively don't want to add redirects? if that's the case, maybe we should be open to adding redirects? | 15:41 |
| pabelanger | yes, I think redirects are great. My comment was related to verification file | 15:42 |
| zbr|rover | corvus: ok, how about progessive fixes based on efforth/benefits? a) reindex google b) out redirects (google tool would report which one are broken!) c) improve alabaster/sphinx search. | 15:42 |
| zbr|rover | last one would clearly take days. | 15:43 |
| *** avass has quit IRC | 15:46 | |
| *** rfolco is now known as rfolco|doctor | 15:48 | |
| corvus | i'm not particularly interested in logging into google for any of this, but if one of the other maintainers wants to, i don't object. i don't think it's necessary; i think the redirects alone will solve all of the identified problems. | 15:51 |
| corvus | i'm happy to fetch the 404 list for anyone who's working on that | 15:52 |
| openstackgerrit | Jeremy Stanley proposed zuul/zuul-website master: Add and update redirects for Zuul docs reorg https://review.opendev.org/702904 | 16:02 |
| fungi | that's a programmatic transformation of the git show for Shrews's docs reorg commit | 16:03 |
| *** michael-beaver has joined #zuul | 16:04 | |
| *** ianychoi_ has joined #zuul | 16:05 | |
| corvus | fungi: that's a cool way to do it :) | 16:05 |
| *** jpena is now known as jpena|off | 16:07 | |
| fungi | i just added a review comment with the convoluted shell one-liner i used, in case anyone wants to verify | 16:07 |
| pabelanger | +2 | 16:07 |
| *** ianychoi has quit IRC | 16:08 | |
| fungi | i figured it was faster to just do that than argue about it | 16:10 |
| mnaser | fungi: THANK YOU! that is super useful in terms of making search engines happy | 16:10 |
| mnaser | and also making our potential users less confused | 16:10 |
| tristanC | corvus: if you have some time, would you know why that change https://review.opendev.org/702758 seems to randomly fails with 'Unknown configuration error' . Commenting 'recheck' fix it. Could it be related to the long stack of change using requires container image? | 16:10 |
| fungi | mnaser: no sweat, it took ~10 minutes ;) | 16:11 |
| *** rishabhhpe has quit IRC | 16:11 | |
| fungi | i'd already wasted more time than that talking about it | 16:12 |
| fungi | and was eager to stop wasting still more | 16:12 |
| mnaser | \o/ | 16:12 |
| fungi | but for the future, if we move docs around we should remember to update the .htaccess file in zuul-website as well | 16:13 |
| *** themroc has quit IRC | 16:13 | |
| fungi | in the course of adding those i noticed that the reorg had actually broken our /start quick link | 16:13 |
| tristanC | corvus: i guess the real exception should be logged in opendev's scheduler logs | 16:14 |
| corvus | fungi: oh, just noted something we should fix in the htaccess change; should be easy for anyone to do if you're busy | 16:15 |
| corvus | tristanC: yeah, i'll look in a few mins | 16:15 |
| fungi | corvus: nah, i'll take a look | 16:16 |
| openstackgerrit | Jeremy Stanley proposed zuul/zuul-website master: Add and update redirects for Zuul docs reorg https://review.opendev.org/702904 | 16:17 |
| fungi | corvus: updated ^ | 16:18 |
| mnaser | hm | 16:22 |
| mnaser | does zuul see testing against gerrit3+ ? | 16:22 |
| mnaser | im trying to get gerrithub integrated and it seems to return a 401 on getting the version (i checked the http password twice and reset it twice) | 16:23 |
| fungi | given corvus is working on zuul for upstream gerrit and i believe they dogfood their latest code, i expect it works? | 16:24 |
| fungi | he had examples of it commenting on changes there anyway | 16:24 |
| clarkb | quickstart also tests gerrit3 | 16:25 |
| fungi | oh, right, so it does! | 16:25 |
| fungi | i'm heading out to grab lunch but am happy to check out the redirects when i get back if they merge while i'm out | 16:25 |
| *** sugaar has quit IRC | 16:28 | |
| *** electrofelix has quit IRC | 16:29 | |
| *** sugaar has joined #zuul | 16:29 | |
| corvus | mnaser: hrm, i don't know why that shouldn't work | 16:31 |
| corvus | mnaser: maybe try using curl to hit the endpoint to get zuul out of the picture first | 16:31 |
| corvus | mnaser: curl --digest -u user:pass https://review.gerrithub.io/a/accounts/self | 16:35 |
| corvus | mnaser: that's what zuul is going to do by default; it also supports basic rather than digest auth if that's necessary to make it work | 16:36 |
| corvus | i haven't logged into gerrithub, so i haven't tried that | 16:36 |
| corvus | pabelanger, Shrews: want to +3 https://review.opendev.org/702904 | 16:37 |
| clarkb | doesthe zuul vhost allow redirects from the htaccess file? iirc that has to be explicitly allowed | 16:42 |
| corvus | clarkb: we have /start | 16:43 |
| clarkb | oh right | 16:43 |
| clarkb | also I think that is what the allow override list we had to pass through for the LE change allows so should be good to go | 16:44 |
| *** mattw4 has joined #zuul | 16:45 | |
| openstackgerrit | Merged zuul/zuul-website master: Add and update redirects for Zuul docs reorg https://review.opendev.org/702904 | 16:45 |
| *** chandankumar is now known as raukadah | 16:49 | |
| *** dtroyer has quit IRC | 16:52 | |
| mnaser | corvus: ok right, i can replicate and get a 401 to gerrithub but not opendev | 17:01 |
| mnaser | corvus: i wonder if we really need to make an authenticated called to get the version though. it seems available via the unauthenticated api | 17:02 |
| clarkb | the /start redirect works for me now | 17:02 |
| clarkb | zbr|rover: did you want to test some of your urls? | 17:02 |
| corvus | mnaser: we don't, but if that doesn't work, then later things that do require auth won't work. so it's helpful to fail early | 17:02 |
| mnaser | my "zuul kubernetes spec" google search is working :) | 17:03 |
| mnaser | corvus: yep, i agree with that, i wonder if gerrithub has certain things locked down | 17:03 |
| corvus | mnaser: try some other urls, like self or project list? | 17:04 |
| mnaser | hmm | 17:05 |
| mnaser | curl to https://review.gerrithub.io/changes/?q=is:open+owner:self returns "Must be signed-in" without the /a/ in the url | 17:05 |
| zbr|rover | clarkb: https://zuul-ci.org/docs/zuul/user/client-user.html | 17:05 |
| mnaser | but adding the /a/ makes it return unauthorized | 17:05 |
| openstackgerrit | Merged zuul/zuul master: Extract project config YAML into ref docs https://review.opendev.org/701977 | 17:06 |
| mnaser | i guess i could omit the password for now | 17:06 |
| *** mattw4 has quit IRC | 17:07 | |
| mnaser | im assuming gerrithub probably has some funky auth | 17:08 |
| clarkb | zbr|rover: looks like that file was deleted and all it did was include another file. So we need to add a redirect to the location of the file it included. | 17:08 |
| corvus | mnaser: try basic instead of digest? | 17:08 |
| mnaser | corvus: curl seems to return unauthorized too doing basic auth | 17:09 |
| *** mattw4 has joined #zuul | 17:10 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Add client-user redirect https://review.opendev.org/702926 | 17:11 |
| clarkb | zbr|rover: ^ I think that will take care of it | 17:11 |
| zbr|rover | clarkb: any reasons for not enabling google site tools? that tools is not GA | 17:11 |
| clarkb | I don't know what google site tools is | 17:11 |
| clarkb | (so couldn't tell you) | 17:11 |
| zbr|rover | clarkb: a tool that enable a webmaster(owner) to see how google indexed his site. ask to url removals, reindexing, seeing pages with errors (incoming links) | 17:12 |
| zbr|rover | is mainly a query tool, but google does not allow random people to use unless they verify ownership (one of 4 methods) | 17:13 |
| zbr|rover | obviously as you do not want competition to de-list you :D | 17:13 |
| zbr|rover | i quite nice, last time I used it it was 2+ years ago, but today I seen that it improved a lot. | 17:14 |
| zbr|rover | once the first webmaster verifies he can add others by adding their google addresses, so no need to create one verification for each user. | 17:15 |
| corvus | mnaser: https://github.com/jenkinsci/gerrit-code-review-plugin/blob/master/CONTRIBUTING.md#gerrithub-configuration doesn't look like anything unusual | 17:16 |
| mnaser | corvus: i wonder if its because they use http for git and not for the actual api | 17:16 |
| clarkb | I'm not sure I want to be involved in that. google's account rules are a bit terrifying at times and I don't want them to break zuul things because they decide I've broken some fi rule or something. | 17:16 |
| clarkb | its possible that jimmy might be willing to help with that if they already have setups in place for other web properties but probably need to have maintainers decide that is what we want to do | 17:17 |
| corvus | mnaser: lol also https://opendev.org/airship/armada/commit/226175a62a0a2cf4effb366b0dc4ae27573167d7 has http instructions too :) | 17:17 |
| mnaser | corvus: ahah | 17:17 |
| corvus | or, at least, had | 17:18 |
| mnaser | i think in those cases, they're using the api for git operations, not the api | 17:18 |
| corvus | mnaser: typically in gerrit, it's the same auth | 17:19 |
| mnaser | corvus: right, but i'm figuring that because gerrithub is some "saas"-ified gerrit, it may have limitations | 17:19 |
| * mnaser drafts up an email | 17:19 | |
| clarkb | you should be able to ask luca | 17:19 |
| corvus | mnaser: you know luca? | 17:20 |
| mnaser | i do not, i was planning to send an email to info@gerritforge.com which was at the footer of the site | 17:20 |
| corvus | 1 sec | 17:20 |
| corvus | (i privately asked mnaser to email luca and cc me) | 17:27 |
| *** jpena|off is now known as jpena | 17:29 | |
| *** rishabhhpe has joined #zuul | 17:30 | |
| rishabhhpe | fungi: Hi did you get a time to check that issue ? | 17:31 |
| *** evrardjp has quit IRC | 17:34 | |
| *** evrardjp has joined #zuul | 17:34 | |
| clarkb | rishabhhpe: does your nodepool-launcher and nodepool-builder use that same config file? | 17:39 |
| clarkb | I wonder if we are only updating the builder's config | 17:39 |
| rishabhhpe | i think so both will use same config file only because if i change anything in that file say create some error then my nodepool commands will give error | 17:40 |
| *** saneax has quit IRC | 17:40 | |
| clarkb | ok just double checking | 17:41 |
| rishabhhpe | clarkb: but still is there any way with which i can be double sure same file is getting used ? | 17:41 |
| clarkb | rishabhhpe: have you tried to manually boot the image explicitly enabling config drive to see if that works? | 17:41 |
| clarkb | rishabhhpe: check the running process command something like `ps -elf | grep nodepool` the -c flag specifies the config path | 17:42 |
| rishabhhpe | yes if i am manually booting the image by providing config-drive true it is taking the parameter | 17:42 |
| *** rfolco|doctor is now known as rfolco | 17:42 | |
| openstackgerrit | Merged zuul/zuul-website master: Add client-user redirect https://review.opendev.org/702926 | 17:44 |
| clarkb | ok so the cloud side should be working then, but for some reason that isn't getting passed through from nodepool | 17:44 |
| fungi | rishabhhpe: if you `ps auxww|grep nodepool-launcher` you should see the full command-line your launcher service is running with. make sure the -c parameter listed there is referring to the same exact file location as the nodepool config you're editing | 17:49 |
| rishabhhpe | fungi: clarkb: i am not getting -c param in the command listed by you http://paste.openstack.org/show/788492/ also i tried with pstree also but still not getting the full text | 17:51 |
| fungi | interesting, maybe it's falling back to a default conffile path then | 17:51 |
| clarkb | ok it probably relies on the default path then whcih I think is /etc/nodepool/nodepool.yaml | 17:51 |
| clarkb | rishabhhpe: is ^ the file you have been editing? | 17:51 |
| fungi | ours looks like: /usr/bin/python3 /usr/local/bin/nodepool-launcher -c /etc/nodepool/nodepool.yaml -l /etc/nodepool/launcher-logging.conf -p /var/run/nodepool-launcher/nodepool-launcher.pid | 17:52 |
| clarkb | yes that is the default for launcher | 17:52 |
| fungi | `nodepool-launcher --help` says: -c CONFIG path to config file (default: /etc/nodepool/nodepool.yaml) | 17:52 |
| fungi | so yes, i guess ours is just unnecessarily explicit | 17:53 |
| rishabhhpe | fungi: clarkb: i did not changed anything in configuration just followed the zuul from scratch link and yes my nodepool.yaml stays in /etc/nodepool directory | 17:54 |
| clarkb | I'm just confirming that is the file we have edited | 17:55 |
| rishabhhpe | clarkb: no i have not editted anything in that file | 17:55 |
| fungi | isn't that the file you're editing to add the config-drive: true? | 17:56 |
| rishabhhpe | yes in that file only i had edited .. i mean to say apart from this configuration .. system level configuration i had not changed like getting debug logging and where to place the logs and all .. everything is placed in default path like my nodepool.pid is generated in default path /var/run/nodepool | 17:58 |
| clarkb | I expect the next step is going to be turning on openstacksdk debug logging and checking what it says it is requseting | 17:59 |
| clarkb | is it possible there is a clouds.yaml setting that overrides config drive stuff? | 18:00 |
| clarkb | mordred: ^ | 18:00 |
| clarkb | rishabhhpe: are you able to share your clouds.yaml file? you can remove the username and password info | 18:00 |
| rishabhhpe | OK sure i will share it | 18:00 |
| pabelanger | clarkb: corvus: anything I can help with for zuul / nodepool release? Last I see, we just need tags / release annoucements prepared? | 18:01 |
| clarkb | pabelanger: I think that may be it. Both already have release notes that capture the important updates | 18:02 |
| rishabhhpe | clarkb: here is the file http://paste.openstack.org/show/788493/ | 18:03 |
| clarkb | https://zuul-ci.org/docs/nodepool/releasenotes.html#new-features and https://zuul-ci.org/docs/zuul/overview/releasenotes.html#in-development | 18:03 |
| fungi | clarkb: according to openstacksdk docs, we could enable debug logging by altering nodepool to add openstack.enable_logging(debug=True) | 18:04 |
| clarkb | rishabhhpe: thanks, nothing there should override config drive settings | 18:04 |
| fungi | i'm not immediately finding a way to do the same from clouds.yaml though | 18:05 |
| rishabhhpe | fungi: clarkb: next what can be done now ? | 18:06 |
| clarkb | fungi: I think we can do it from the nodepool logging config file too | 18:06 |
| clarkb | or just run nodepool with -d? | 18:07 |
| pabelanger | clarkb: ack! | 18:07 |
| fungi | -d do not run as a daemon with debug logging (default: False) | 18:07 |
| fungi | yep, that's an option | 18:07 |
| corvus | clarkb, pabelanger: i'll start the releases now | 18:07 |
| pabelanger | \o/ | 18:08 |
| clarkb | fungi: rishabhhpe ya I think try running it with --debug/-d and see if we can capture the sdk logs | 18:08 |
| corvus | clarkb: ok that i'll do release? (want to avoid confusion) | 18:08 |
| clarkb | corvus: yes | 18:09 |
| corvus | clarkb: i'd like to zuul ca12e0fe66cc0829375ac4eb5b823c29a7375528 as 3.15.0 for zuul | 18:10 |
| corvus | clarkb: that's a few commits ahead of what you suggested (e6d8b210cc416ed494b0b0248404e3e6d7ce337c) | 18:10 |
| corvus | but they are all docs changes | 18:10 |
| corvus | so might as well freeze 3.15.0 with nicer docs | 18:10 |
| rishabhhpe | clarkb: let me try | 18:11 |
| corvus | clarkb: (waiting on ack from you that looks okay) | 18:11 |
| corvus | pabelanger: ^ also | 18:11 |
| pabelanger | looking | 18:12 |
| clarkb | corvus: double checking | 18:12 |
| pabelanger | ca12e0 wfm | 18:13 |
| clarkb | confirmed those are only docs updates. lgtm | 18:13 |
| corvus | both tags pushed | 18:13 |
| rishabhhpe | clarkb: fungi: i had pasted the debug log here : http://paste.openstack.org/show/788495/ | 18:21 |
| corvus | tristanC: the error from earlier: http://paste.openstack.org/show/788496/ | 18:22 |
| fungi | rishabhhpe: that's probably not the paste url you meant, but yeah what we're talking about now is that we probably need to adjust the logging config to make it include debug logging from the openstacksdk library so we get details about what api requests it's making to nova | 18:24 |
| rishabhhpe | fungi: so for that what changes needs to be done in configuration ? | 18:25 |
| fungi | rishabhhpe: sorry, i'm in another meeting right now, but do you have a /etc/nodepool/launcher-logging.conf file? or does the ps command i mentioned earlier indicate nodepool-launcher is running with the -l option (and if so, what's the filename it mentions after -l) | 18:27 |
| rishabhhpe | no i do not have launcher-logging.conf file and there is no -l option in ps -ef|grep nodepool | 18:28 |
| *** jpena is now known as jpena|off | 18:41 | |
| corvus | Shrews: i think we should move the release notes out of the "developers" guide and back to the top level | 18:54 |
| *** rlandy is now known as rlandy|afk | 18:54 | |
| clarkb | corvus: they are linked at a top level fwiw | 18:54 |
| clarkb | https://zuul-ci.org/docs/zuul/overview/releasenotes.html and live in overview | 18:54 |
| clarkb | (though I'm looking at the website not the code, maybe it isn't organized like that in rst) | 18:55 |
| corvus | clarkb: no you're right | 18:55 |
| corvus | i think "overview" may not be the right word there. | 18:56 |
| *** openstackgerrit has quit IRC | 18:58 | |
| rishabhhpe | fungi: clarkb: actually i was starting the service for nodepool as nodepool-launcher not by systemctl utility and now when i am trying to run it with systemctl it is getting killed because on every restart /var/run/nodepool directory is getting deleted | 18:58 |
| rishabhhpe | is there any way i can overcome that | 18:58 |
| *** openstackgerrit has joined #zuul | 18:59 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: JWT drivers: Deprecate RS256withJWKS, introduce OpenIDConnect https://review.opendev.org/701972 | 18:59 |
| pabelanger | feature request, it would be nice to also see nodepool version info on zuul status UI page | 19:00 |
| pabelanger | I wonder if UI could access nodepool API to figure that out | 19:00 |
| corvus | pabelanger: it would be better for zuul-web to supply it | 19:01 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: JWT drivers: Deprecate RS256withJWKS, introduce OpenIDConnect https://review.opendev.org/701972 | 19:02 |
| corvus | pabelanger: if you're interested in implementing that, i would recommend: 1) make sure nodepool writes the info to ZK; 2) have zuul-web read that info from ZK and return it via an api/info endpoint | 19:02 |
| pabelanger | ack | 19:02 |
| pabelanger | sure, can hack on it | 19:02 |
| pabelanger | mostly thought of it while upgrading nodepool | 19:02 |
| pabelanger | (also, upgraded zuul.a.c to nodepool 3.11.0) | 19:03 |
| pabelanger | working on zuul upgrade now | 19:03 |
| corvus | pabelanger: and keep in mind that every component could be a different version, so it really needs to be a list of components | 19:03 |
| pabelanger | yah, guess that is true | 19:03 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Re-order reference index https://review.opendev.org/702962 | 19:05 |
| mnaser | runing zuul with -f automatically enables debug :< | 19:10 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: JWT drivers: Deprecate RS256withJWKS, introduce OpenIDConnect https://review.opendev.org/701972 | 19:10 |
| pabelanger | mnaser: I've found running with debugs helpful by default | 19:11 |
| corvus | that may be, but that's not why -f does that | 19:12 |
| corvus | we do not intend users to run with debug logs enabled | 19:12 |
| corvus | if there's something important that only shows up in debug, it should be moved to info | 19:12 |
| pabelanger | I'm just reading up on -f, I wasn't aware of it | 19:12 |
| mnaser | so that means the k8s stuff is by default outputting everything as debug | 19:12 |
| corvus | -f runs with debug due to historical reasons | 19:12 |
| corvus | yeah, it's a long complex process to unwind it, and i'm not sure anyone is pushing on that. | 19:13 |
| corvus | the end result is we should have -f and -d do 2 separate things | 19:13 |
| pabelanger | ah, didn't know we added -f | 19:15 |
| corvus | maybe we just need to make -f not imply -d now? i'm not sure where we are in the process. tobiash might remember. | 19:15 |
| openstackgerrit | Mohammed Naser proposed zuul/zuul-helm master: Added support for configuring disk_limit_per_job https://review.opendev.org/702963 | 19:17 |
| tobiash | corvus: you mean https://review.opendev.org/#/c/635649? | 19:17 |
| tobiash | I still need to address review comments on that, but I forgot about that | 19:19 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add OpenShift SCC and functional test https://review.opendev.org/702758 | 19:20 |
| pabelanger | mnaser: for 702963, does that mean, every config setting a user wants to tweak needs to be exposed? Bringing back bad memories puppet-zuul and plumbing changes though all the layers | 19:20 |
| mnaser | pabelanger: corvus actually has a change that allows you to manage tehe secret externally | 19:21 |
| corvus | tobiash, mnaser: yeah i think 635649 is the next step, then once everyone is using images with "-f" we can change "-d" to stop implying foreground. | 19:21 |
| pabelanger | mnaser: ah, cool! | 19:22 |
| pabelanger | for ansible-role-zuul, just shipped a minimal template, and if people wanted to toggle bits, they could pass in their own. Would be aweomse to do the same with helm | 19:23 |
| mnaser | i mean you could technically do that too by using templating, thats how i templated the connections | 19:23 |
| pabelanger | yah, I'm a big fan these days of passing in the whole config object, then each settings | 19:24 |
| pabelanger | is helm still a thing that runs inside your k8s? Much like heat does? | 19:25 |
| mnaser | no with helm 3 its more of a templating engine | 19:25 |
| pabelanger | ack, thanks | 19:26 |
| mnaser | corvus: did we somehow manage to remove the ability to define extra files :( | 19:27 |
| mnaser | oh thats weird | 19:27 |
| mnaser | i never pushed that change ugh | 19:27 |
| fungi | pabelanger: are you thinking of something like a versions endpoint which enumerates the versions of zuul and nodepool running on each executor, merger, builder, launcher...? | 19:28 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: change default calling as a module https://review.opendev.org/690057 | 19:28 |
| openstackgerrit | Mohammed Naser proposed zuul/zuul-helm master: Add extra files for secret https://review.opendev.org/702965 | 19:29 |
| fungi | it would also be fairly trivial to collect info on what versions of python dependencies are present in the module search path of each of those as well. i have some sample code i can provide which i'm using to do that in one of my personal projects | 19:29 |
| pabelanger | fungi: not sure, I mostly wanted to have an 'easy' way to check the version number of nodepool and zuul. Today, for zuul I can look at UI for zuul-web, but I guess that isn't the version number for all zuul services | 19:30 |
| fungi | also reporting info on the ansible envs for each executor could be a nifty addition | 19:30 |
| fungi | basically have one api method you can call to get information on the versions of zuul/nodepool/ansible and dependencies present on all of the components, so you can easily spot when you may have missed upgrading some or have unexpected inconsistencies between them | 19:31 |
| clarkb | corvus: see note on https://review.opendev.org/#/c/702962/1 | 19:42 |
| *** rishabhhpe has quit IRC | 19:43 | |
| mordred | morning all | 19:51 |
| mordred | clarkb: re: sdk and config-drive - for which settings? it shouldn't be doing much with config-drive in general? | 19:52 |
| clarkb | mordred: for the nova boot request for a config drive | 19:52 |
| clarkb | mordred: the setting is apparently set in the nodepool config but the instances that are booted do not get config drives | 19:52 |
| clarkb | if manually booted then it all works | 19:52 |
| mordred | oh, that's weird | 19:53 |
| mordred | yes - i agree, sdk logs would be nice | 19:53 |
| tristanC | corvus: that traceback is not very helpful... could it be raised from another exception handler? | 19:55 |
| corvus | clarkb: i did that originally, but then i realized that *_def was subservient to config. so jobs comes right after config+*_def. the alternative would be for jobs to come before config, but config is more important; that seems backwards. | 19:56 |
| corvus | tristanC: i agree; i didn't see any other logs nearby. | 19:57 |
| tristanC | this time it took 8 rechecks to pass through the 'Unknown configuration error' | 19:57 |
| corvus | there's clearly a code bug | 19:58 |
| clarkb | corvus: it reads to me like jobs is an expansion of what is introduced in job_def | 19:58 |
| tristanC | iiuc, job that requires artifacts results in sql queries during freeze, and perhaps when there is too many parents the scheduler could choke and report that generic error comment? | 19:58 |
| clarkb | "here is how you make a job" and then "this is how you make jobs useful" | 19:58 |
| corvus | clarkb: oh, i see what you mean. but i see all of the config reference as a unit. i don't think we should interrupt describing how to configure the system and insert how to write jobs in the middle of that. | 19:59 |
| corvus | clarkb: perhaps some xrefs within job_def would help bridge the gap? | 19:59 |
| corvus | especially in the *run attributes? | 20:00 |
| corvus | (here's how you say what playbook to run; follow this link to see what you can do in the playbook) | 20:00 |
| clarkb | corvus: ++ was just going to suggest from the *run: directives | 20:00 |
| pabelanger | and upgraded zuul.a.c to 3.15.0 | 20:01 |
| *** tosky has quit IRC | 20:04 | |
| *** bhavikdbavishi has quit IRC | 20:06 | |
| mnaser | hmm | 20:08 |
| mnaser | does zuul include the upstream branches by any chance (or what would be a way you'd be able to compare the pristine branch vs local branch without doing a remote pull) | 20:09 |
| mnaser | i'm working on moving a piece of software to run under zuul and it does this thing where it tries to discover what are the changes it is testing | 20:10 |
| clarkb | mnaser: yes, I believe tehre are remotes for the source with appropriate pointers | 20:10 |
| clarkb | mnaser: is this a github project that is trying to check intermediate commits in a PR? | 20:10 |
| clarkb | (I really don't like that pattern, but I've seen it and developing a standard role for extracting that info might be a good idea) | 20:11 |
| mnaser | clarkb: i think this is one of those "lets reduce the amount of work we do by testing only the changes" | 20:11 |
| mnaser | "This means that e.g. a git diff origin/<branch>..<branch> will show the changes being tested." -- ok cool | 20:11 |
| fungi | i think in that case zuul will show there are no changes | 20:13 |
| clarkb | ya its not the origin remote | 20:13 |
| clarkb | I'm trying to figure this out. I had to look it up forever ago because kata does the walk through each commit in a PR and test them thing | 20:13 |
| fungi | because zuul "tests the future" by providing your test environment with the speculative future states of all repositories involved based on whatever dependency information it's aware of | 20:13 |
| *** sshnaidm is now known as sshnaidm|afk | 20:14 | |
| fungi | it presents the future as if it were the present | 20:14 |
| fungi | though in most cases (as long as you can guarantee every commit will be tested before merging) it's likely sufficient to compare HEAD against HEAD^1 | 20:15 |
| pabelanger | for https://zuul-ci.org/docs/zuul/reference/job_def.html#attr-job.match-on-config-updates does a config update, include a change to a job playbook? Or just zuul.yaml files them self? I am guessing just zuul.yaml files | 20:15 |
| clarkb | fungi: not if using github | 20:15 |
| fungi | and rely on the fact that there's another build ahead of that one where HEAD^1 is actually HEAD | 20:15 |
| fungi | clarkb: which is why i said "as long as you can guarantee every commit will be tested before merging" | 20:15 |
| fungi | if it's a github project following a rebase/squash methodology then it probably still holds | 20:16 |
| fungi | so long as the pr updates are squashed into the previous pr commit (so more like commit --amend) | 20:17 |
| clarkb | ya | 20:17 |
| clarkb | maybe it is the origin ref. | 20:18 |
| clarkb | heads/foo should be the zuul checkout for sure | 20:18 |
| fungi | i'm really wishing i'd gotten a good photo of one of the "no rebase" road signs in mexico | 20:18 |
| clarkb | yup found it link in a moment | 20:20 |
| clarkb | mnaser: fungi https://opendev.org/zuul/zuul/src/branch/master/zuul/merger/merger.py#L766-L771 | 20:21 |
| clarkb | its still a potentially speculative base, so each job would be responsible for their own piece of the delta pie | 20:21 |
| clarkb | fungi: ^ which answers your question | 20:22 |
| fungi | aha, okay so for a github pr with multiple commits, origin remote will possibly have the state of a pr it's marked as dependent on, or sequenced after, or else the actual origin branch tip | 20:24 |
| clarkb | fungi: I think in teh github case I think it means each PR has to be tested that way | 20:24 |
| clarkb | yup | 20:24 |
| fungi | right | 20:24 |
| fungi | so origin/master..master can still have multiple commits in it, but it will only be the commits present in your pr | 20:25 |
| mordred | yah | 20:25 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: ensure-tox: use portable pip call https://review.opendev.org/702978 | 20:26 |
| clarkb | zbr|rover: does ^ mean that python3 only systems don't have `pip` but do have `pip3` and the python module 'pip'? | 20:27 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: ensure-tox: use portable pip call https://review.opendev.org/702978 | 20:27 |
| clarkb | one problem with ^ is it will start running all tox targets that don't specify a python under python3 by default for many platforms | 20:27 |
| clarkb | (becuse our ansible connection is done via python3) | 20:28 |
| zbr|rover | clarkb: all systems with python3 have pip3, not pip (unless someone....) | 20:28 |
| mordred | zbr|rover: you sure? | 20:28 |
| clarkb | zbr|rover: well my pip is python3 | 20:28 |
| clarkb | and I have pip3 too | 20:28 |
| mordred | yes - same | 20:29 |
| clarkb | mostly my concern with that change is tox runs jobs under the python it is installed under by default | 20:29 |
| clarkb | which means any target that doesn't specify a python could suddenly switch python versions after that changem erges | 20:29 |
| mordred | clarkb: it was already installing tox using an arbitrary python though | 20:29 |
| clarkb | mordred: well in our case it was the "system default python" | 20:29 |
| clarkb | mordred: because that is how dib sets things up | 20:29 |
| mordred | so Im not sure this changes the fact that pythonless tox envs are indeterminate | 20:29 |
| mordred | clarkb: fair point | 20:30 |
| zbr|rover | clarkb: let me put it this way: for centos-8 there is no tox package and if you install the system without the outdated python2, the ensure-tox role will choke big time. | 20:30 |
| clarkb | zbr|rover: because there is no `pip` only `pip3`? | 20:30 |
| zbr|rover | yeah! | 20:30 |
| mordred | is that installing pip from packages? | 20:31 |
| zbr|rover | the only thing we know for sure is that we have one python installed, that is the ansible_python.executable one. | 20:31 |
| mordred | because I'll keep beating my dead horse that installing pip from distro packages is an exercise in pain ... but good to know | 20:31 |
| zbr|rover | using module calling, we reach its pip. | 20:31 |
| clarkb | I'll have to think on this over lunch. But that change will likely break openstack stable jobs runnin on not bionic | 20:32 |
| clarkb | granted those stable jobs should specify a python but we've already had to go through this with bionic defaulting to python3 | 20:32 |
| mordred | yah - I grok the issue that's trying to be solved and think the end result here will be more consistent - but I also agree with clarkb about thinking about it | 20:33 |
| zbr|rover | atm its is impossible for me to make use of ensure-tox, hopefully we can fix that. | 20:34 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: DNM: test arbritary uid support for zuul image https://review.opendev.org/702980 | 20:34 |
| zbr|rover | i hope that everyone agrees that we should aim to use platform default python, unless someone is overriding it manually. | 20:35 |
| clarkb | zbr|rover: that is not what your change does though | 20:36 |
| clarkb | ypur change uses the ansible python which us not the platform default in many cases | 20:36 |
| zbr|rover | clarkb: it is, ansible team worked a lot to assure this. (assuming we are not talking about an outdated/buggy version) | 20:37 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: DNM: test arbritary uid support for zuul image https://review.opendev.org/702980 | 20:37 |
| zbr|rover | clarkb: or you prefer me to propose something like PIP=`command -v pip3 pip | head -n1` ? | 20:38 |
| clarkb | auto is really new | 20:38 |
| clarkb | its gonna be hard set in many cases | 20:38 |
| zbr|rover | since 2.8 detection works well, we already removed harcoding in most places in tripleo. | 20:39 |
| clarkb | personally I think we should prefer 'pip' if present as that tends to be the default | 20:39 |
| zbr|rover | if one is on 2.7, he is on his own. | 20:39 |
| clarkb | zbr|rover: but zuul supports more than 2.8 and 2.9 | 20:39 |
| mordred | yah | 20:39 |
| *** armstrongs has joined #zuul | 20:40 | |
| zbr|rover | the reality is that the current hardcoded `pip` from the role may not exist, but `<> -m pip` is guaranteed to exist. | 20:40 |
| mordred | yes - but is going to be a behavior change for a set of current users | 20:41 |
| zbr|rover | in the end I do not want to waste too much energy on this, I would be glad to use even a `command -v pip pip3` like you said. | 20:41 |
| mordred | so has the potential to be a breakign change | 20:41 |
| zbr|rover | as long we fix the problem. | 20:41 |
| mordred | agreee - fixing the problem is a good thing | 20:41 |
| mordred | zbr|rover: I think command -v pip pip3 might be the safest approach for now | 20:42 |
| zbr|rover | mordred: ok, updating it. | 20:42 |
| mordred | much as I do like the simplicity of python -m pip | 20:42 |
| zbr|rover | we can work on that one later, when is less pressure. i support it too. | 20:43 |
| tristanC | corvus: a child change of the 'Unknown configuration error' seems to be in an odd state, both of its job are in 'waiting' state, currently in the status page: https://zuul.opendev.org/t/zuul/status | 20:43 |
| tristanC | it's https://review.opendev.org/#/c/702980/, i was hoping it would demonstrate why we need https://review.opendev.org/#/c/650246/ in zuul image. iiuc, if both job are 'waiting', then it's never going to start | 20:45 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: ensure-tox: improve pip detection https://review.opendev.org/702978 | 20:47 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: ensure-tox: improve pip detection https://review.opendev.org/702978 | 20:48 |
| mordred | zbr|rover: typo | 20:49 |
| *** armstrongs has quit IRC | 20:49 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: ensure-tox: improve pip detection https://review.opendev.org/702978 | 20:50 |
| tristanC | corvus: nvm, it was waiting for the build-image of the zuul change | 20:53 |
| *** sgw has quit IRC | 20:59 | |
| *** rlandy|afk is now known as rlandy | 21:03 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: ensure-tox: improve pip detection https://review.opendev.org/702978 | 21:04 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: ensure-tox: improve pip detection https://review.opendev.org/702978 | 21:06 |
| fungi | ianw: one other building block. we can indeed parse the pip log to find out what wheels we downloaded from pypi: sed -n 's,.*Downloading from URL .*/\([^/]*\.whl\)#.*,\1,p' pip.log | 21:08 |
| fungi | though might want to extend the pip wheel command to something like `pip --log pip.log wheel ...` so we can more directly control where the appending log resides | 21:08 |
| fungi | oh, oops, that was meant for #openstack-infra | 21:09 |
| fungi | sorry for the noise, folks | 21:09 |
| *** jamesmcarthur has joined #zuul | 21:11 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Replace existing operator tasks with the new application https://review.opendev.org/702106 | 21:13 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add custom image override https://review.opendev.org/702866 | 21:13 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: Add OpenShift SCC and functional test https://review.opendev.org/702758 | 21:13 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-operator master: DNM: test arbritary uid support for zuul image https://review.opendev.org/702980 | 21:13 |
| *** sgw has joined #zuul | 21:14 | |
| *** jamesmcarthur has quit IRC | 21:17 | |
| *** jamesmcarthur has joined #zuul | 21:18 | |
| zbr|rover | clarkb: mordred : i think ensure-tox now ready https://review.opendev.org/#/c/702978/ | 21:24 |
| *** jamesmcarthur has quit IRC | 21:25 | |
| clarkb | zbr|rover: thanks that version looks safe for existing users | 21:26 |
| zbr|rover | clarkb: i hope so, i wasn't able to test it on rdo due to how that repo is consumed there. | 21:27 |
| *** rfolco has quit IRC | 21:29 | |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Docs: move project config docs to user reference https://review.opendev.org/702992 | 21:48 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Move overview section to reference https://review.opendev.org/702995 | 21:55 |
| *** sgw has quit IRC | 22:01 | |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Docs: re-order reference index https://review.opendev.org/702962 | 22:13 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Docs: move project config docs to user reference https://review.opendev.org/702992 | 22:13 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Docs: move overview section to reference https://review.opendev.org/702995 | 22:13 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Docs: add admin reference section https://review.opendev.org/702997 | 22:13 |
| *** jamesmcarthur has joined #zuul | 22:13 | |
| corvus | Shrews: perhaps we should consider making the file hierarchy a little more flat so we don't have to keep updating redirects? | 22:13 |
| Shrews | corvus: I guess we could. Isn't most of the redirecting done now though? Open to whatever is easier. | 22:15 |
| corvus | Shrews: well, those 4 changes move stuff around that will need to update the redirects | 22:16 |
| clarkb | corvus: http://paste.openstack.org/show/788503/ is the reason for that change having problems | 22:16 |
| clarkb | possibly a github3 bug | 22:16 |
| clarkb | pabelanger: ^ have you seen that before? | 22:16 |
| corvus | clarkb: neat. that's not ringing a bell | 22:17 |
| * clarkb pulls up api docs for github | 22:17 | |
| clarkb | https://developer.github.com/v3/pulls/reviews/#list-reviews-on-a-pull-request is what we are retriving when we fail | 22:18 |
| clarkb | I wonder if you can leave a review that isn't associated to a commit on a PR | 22:19 |
| *** jamesmcarthur has quit IRC | 22:19 | |
| clarkb | http://paste.openstack.org/show/788504/ I think that is what is happening there | 22:20 |
| clarkb | also nice of github to make that available anonymously | 22:21 |
| corvus | clarkb: "neat" a review without a commit? | 22:24 |
| clarkb | corvus: ya whats even better is that is a "changes requested" review which is associated to code changes that are edited on top of the code | 22:26 |
| clarkb | its really weird that they wouldn't have an associated commit id as a result | 22:26 |
| clarkb | but I've found https://github.com/sigmavirus24/github3.py/issues/825 | 22:26 |
| clarkb | jlk: ^ I know you mentioned you weren' | 22:26 |
| clarkb | er | 22:26 |
| clarkb | weren't doing much with github3 because biking, but any idea if there was any further progress on that? | 22:26 |
| clarkb | I'm trying to sort out what exactly merged and why we appear to still hit this | 22:27 |
| *** jamesmcarthur has joined #zuul | 22:28 | |
| clarkb | oh the merged change is the change that added the code that is hitting the bug | 22:28 |
| clarkb | not a fix | 22:28 |
| jlk | yeah, no fix yet. Also we haven't done a release in a while, which I was going to do, but alas. | 22:32 |
| clarkb | jlk: actually looks like pabelanger got a fix merged | 22:32 |
| clarkb | (I see it in the code, now looking for it in the github side) | 22:32 |
| clarkb | but I think we need a release to consume it | 22:32 |
| jlk | ah, yeah | 22:33 |
| clarkb | https://github.com/sigmavirus24/github3.py/pull/944 there we go | 22:33 |
| clarkb | alright considering that I'm just gonna remove the depends-on on our side as the PR has already merged | 22:33 |
| clarkb | then we can approve it and land our code knowing the next github3 release will fix this | 22:33 |
| jlk | there, I closed the issue | 22:33 |
| jlk | I'll poke around at doing a release. | 22:33 |
| clarkb | thanks! | 22:34 |
| *** jamesmcarthur has quit IRC | 22:35 | |
| *** jamesmcarthur has joined #zuul | 22:47 | |
| *** jamesmcarthur has quit IRC | 22:48 | |
| *** jamesmcarthur has joined #zuul | 22:52 | |
| *** smcginnis is now known as smcginnis|PTO | 22:52 | |
| tristanC | https://review.opendev.org/702758 shows zuul-operator working on openshift and kubernetes. though i had to use sf zookeeper image because all the other i tried failed to start | 22:55 |
| *** jamesmcarthur has quit IRC | 22:55 | |
| clarkb | fungi: makes an interesting point on https://review.opendev.org/#/c/671435/ about zuul's api. Zuulians may want to read that and see if they have any good ideas | 22:58 |
| clarkb | long story short is in order to provide live zuul status in a ci system like gerrit it would be good if we could retrieve status data for changes without tenant info | 22:58 |
| clarkb | because a change may be tested in more than one tenant | 22:58 |
| fungi | now you've got me intrigued | 22:58 |
| fungi | oh, that point | 22:58 |
| fungi | yeah, maybe just return status data for all tenants visible to the caller if that change is active in pipelines for any of them? | 23:00 |
| clarkb | maybe we can aggregate all tenant jobs for a per change status? | 23:00 |
| clarkb | jinx | 23:00 |
| fungi | though that could present some minor ui challenges too | 23:00 |
| mordred | tristanC: interesting re: zk image. corvus mnaser - y'all haven't had zk image issues have you? makes me think it would be worth figuring out what's different here | 23:02 |
| mnaser | mordred, tristanC: the chart i was using ran fine, esp in the zuul-system umbrella chart in CI | 23:04 |
| clarkb | mordred: I've not run zk out of a containter yet, but they give you a tarball with everything in it but java and it just works with my openjdk java | 23:04 |
| clarkb | all that to say its weird | 23:04 |
| tristanC | It's because openshift starts the service with arbritary uid, and the popular zookeeper images failed with permission denied (either on data, or conf) | 23:04 |
| mordred | ah - gotcha. zk failed on openshift - not on plain k8s | 23:05 |
| tristanC | the one from bitnani seems to work, but it requires an environment variable to start | 23:05 |
| corvus | the quick-start job runs with zk in a container | 23:05 |
| clarkb | like you should be able to FROM openjdk wget | tar x CMD $pathto zkServer.sh start | 23:05 |
| tristanC | here is docker.io/library/zookeeper running in k8s: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_d3a/702106/13/check/zuul-operator-functional-k8s/d3a3db4/docker/k8s_zk_zuul-zk-0_default_7bcc2511-3e26-410c-be28-0135893d7e8b_0.txt | 23:06 |
| mordred | tristanC: have I mentioned I don't like that "feature" ? :) | 23:06 |
| tristanC | and here is the same image in openshift: https://4dc2ad2f50a7fc6565c9-5b268c91353402091e090d56d63539d9.ssl.cf1.rackcdn.com/702758/5/check/zuul-operator-functional-openshift/2afb6c5/docker/k8s_zk_zuul-zk-0_myproject_2f3f4c82-387f-11ea-8792-fa163e725965_5.txt | 23:06 |
| clarkb | how do you manage persistent data ni those cases? | 23:06 |
| tristanC | clarkb: when using persistent volume mount point, openshift does set the right uid | 23:07 |
| fungi | persistent data isn't cloud-native, get with the times! sheesh ;) | 23:07 |
| fungi | clearly you're not familiar with the newest trend, dataless computing | 23:07 |
| tristanC | mordred: actually, that feature is quite good because it reveal issue with your deployment, a service shouldn't be writting to the container rootfs, the data should be in a volume | 23:08 |
| clarkb | fungi: if you love NIH you'll love dataless computing. You get to reinvent everything each time you reboot :) | 23:08 |
| mordred | tristanC: "should" is often theoretical though and there's times when writing files that are meant to be ephemeral is fine | 23:09 |
| mordred | tristanC: but - it's not going away, so my complaining isn't going to accomplish anything useful | 23:09 |
| tristanC | mordred: right, but in that case, all the failure were related to data not meant to be ephemeral | 23:09 |
| mordred | nod | 23:10 |
| tristanC | mordred: ephemeral folder like /tmp or /var/tmp works as expected in openshift too | 23:10 |
| clarkb | Shrews: you may want to check my comment on https://review.opendev.org/#/c/702828/2/nodepool/driver/static/provider.py to ensure I've read the behavior there properly | 23:54 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!