Friday, 2020-04-03

*** saneax has quit IRC00:13
*** rfolco has quit IRC01:29
*** swest has quit IRC01:32
*** ysandeep|away is now known as ysandeep|rover01:44
*** swest has joined #zuul01:46
*** ysandeep|rover is now known as ysandeep|rover|b02:36
*** Goneri has quit IRC03:49
*** bhavikdbavishi has joined #zuul03:50
*** bhavikdbavishi1 has joined #zuul03:53
*** bhavikdbavishi has quit IRC03:54
*** bhavikdbavishi1 is now known as bhavikdbavishi03:54
*** ysandeep|rover|b is now known as ysandeep|rover04:19
*** evrardjp has quit IRC04:36
*** evrardjp has joined #zuul04:37
*** y2kenny has quit IRC05:43
*** dpawlik has joined #zuul06:06
*** jamesmcarthur has quit IRC06:07
*** saneax has joined #zuul06:08
*** igordc has quit IRC06:21
*** avass is now known as Guest5361906:52
*** avass has joined #zuul06:52
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Adds roles to install and run hashicorp packer  https://review.opendev.org/70929206:57
*** jcapitao has joined #zuul07:06
tobiashtristanC, corvus: how shall we proceed with https://review.opendev.org/619156 (zk auth). Can we help there?07:11
tobiashbolg: ^07:11
*** jpena|off is now known as jpena07:16
*** bhavikdbavishi has quit IRC07:23
*** gtema has joined #zuul07:23
*** jamesmcarthur has joined #zuul07:41
*** tosky has joined #zuul07:44
*** jamesmcarthur has quit IRC07:47
openstackgerritDaniel Pawlik proposed zuul/zuul-jobs master: [DNM] - testing updated software-factory images  https://review.opendev.org/71721007:48
*** rpittau|afk is now known as rpittau07:56
*** ysandeep|rover is now known as ysandeep|rover|l08:03
*** bhavikdbavishi has joined #zuul08:04
*** dpawlik has quit IRC08:11
*** dpawlik has joined #zuul08:11
openstackgerritTobias Henkel proposed zuul/zuul master: Make reporting asynchronous  https://review.opendev.org/69125308:25
*** harrymichal has joined #zuul08:27
*** ysandeep|rover|l is now known as ysandeep|lunch08:32
*** jamesmcarthur has joined #zuul08:43
*** jamesmcarthur has quit IRC08:48
openstackgerritTobias Henkel proposed zuul/zuul master: Make reporting asynchronous  https://review.opendev.org/69125308:50
*** rmart04 has joined #zuul08:50
openstackgerritAlbin Vass proposed zuul/zuul master: WIP: Add support for ansible callbacks  https://review.opendev.org/71726009:00
*** hashar has joined #zuul09:03
*** jamesmcarthur has joined #zuul09:07
*** ysandeep|lunch is now known as ysandeep|rover09:09
*** jamesmcarthur has quit IRC09:12
openstackgerritSorin Sbarnea proposed zuul/zuul-jobs master: Add support for RedHat platforms on install-podman  https://review.opendev.org/71657809:14
openstackgerritJan Kubovy proposed zuul/zuul master: WIP: Store unparsed branch config in Zookeeper  https://review.opendev.org/70571609:29
openstackgerritJan Kubovy proposed zuul/zuul master: Prepare Zookeeper for scale-out scheduler  https://review.opendev.org/71726909:29
zbrtobiash: tristanC : can you help me with https://review.opendev.org/#/c/690057 ?09:30
zbris see additional changes proposed to this role and multi-platform testing is really needed.09:31
*** jamesmcarthur has joined #zuul09:47
*** jamesmcarthur has quit IRC09:55
*** jamesmcarthur has joined #zuul10:04
*** bhavikdbavishi has quit IRC10:25
*** rpittau is now known as rpittau|bbl10:29
*** jcapitao is now known as jcapitao_lunch10:40
*** jamesmcarthur has quit IRC10:47
openstackgerritAlbin Vass proposed zuul/zuul master: WIP: Add support for ansible callbacks  https://review.opendev.org/71726011:00
*** bhavikdbavishi has joined #zuul11:03
openstackgerritAlbin Vass proposed zuul/zuul master: WIP: Add support for ansible callbacks  https://review.opendev.org/71726011:04
*** ysandeep|rover is now known as ysandeep|break11:05
*** jamesmcarthur has joined #zuul11:14
*** jamesmcarthur has quit IRC11:20
openstackgerritAlbin Vass proposed zuul/zuul master: WIP: Add support for ansible callbacks  https://review.opendev.org/71726011:31
*** jpena is now known as jpena|lunch11:38
*** ysandeep|break is now known as ysandeep|rover11:48
tristanCzbr: commented11:53
tristanCtobiash: corvus: iirc we decided to not implement zk-auth as tls is enough to prevent annonymous access (which is consistent with gearman). Thus, unless it is still benefical, I would like to abandon the zk-auth changes.11:55
tobiashtristanC: ah thanks, I lost track on this topic11:55
tristanCtobiash: i did an initial zk tls setup using the shellscript in the operator, and we decided that using the cert-manager crd would be better.12:01
tristanCtobiash: would you mind having a look at this change, it is quite useful to work with the operator integration tests locally: https://review.opendev.org/#/c/714163/612:02
*** jcapitao_lunch is now known as jcapitao12:07
*** rpittau|bbl is now known as rpitau12:08
*** rpitau is now known as rpittau12:09
openstackgerritJan Kubovy proposed zuul/zuul master: Prepare Zookeeper for scale-out scheduler  https://review.opendev.org/71726912:10
tobiashtristanC: lgtm12:10
tobiashtristanC: I've responded to your question on https://review.opendev.org/70881212:14
tristanCtobiash: according to https://opendev.org/zuul/zuul/src/branch/master/tox.ini#L59 , mypy is not running in the tests12:15
tobiashtristanC: ah you mean the test folder12:15
tobiashgood catch12:15
*** jamesmcarthur has joined #zuul12:15
tristanCtobiash: and my second issue is that mypy is not checking code inside untyped function, unless we run it in --strict. thus it seems like some annotation are not validated and they may become incorrect12:17
tristanCtobiash: however, that is great to hear they are already useful within some IDE, thus their addition lgtm12:19
tobiashwell strict is not possible unless we annotate all of zuul12:19
*** jamesmcarthur has quit IRC12:20
zbrtristanC: how about running "pip install tox" regardless if "tox" command is present or not?12:21
zbri am not aware of other way to assure it is upgraded when needed12:21
tristanCtobiash: for example, those annotations are not tested by mypy https://review.opendev.org/#/c/709735/8/zuul/scheduler.py@340 , because the constructor arguments are not typed12:22
openstackgerritJan Kubovy proposed zuul/zuul master: WIP: Store unparsed branch config in Zookeeper  https://review.opendev.org/70571612:22
tristanCzbr: perhaps if the upgrade_tox toggle is set, then the role should run the upgrade command, whenever tox is already installed or not12:23
zbris not so easy, you could also have additional constraints12:23
zbrif you do a "tox>1.2.3 tox-pyenv tox-docker" kind of stuff.12:24
tristanCzbr: then it's no longer just upgrading tox is it?12:24
zbrtristanC: tox plugins are key for running tox in certain use-cases, i do see them as part of the same tool12:26
bolgtristanC: I am very in favor of using those annotations (https://review.opendev.org/708812) it really helps navigating the code. What IDE are you using?12:27
zbrdefault is w/o plugins, just "tox", but others will need to enable extra plugins. for example core tox is unable to find pyenv pythons.12:27
tristanCzbr: understood, but this is triggered by a variable named 'upgrade_tox', which sounds missleading if it is used to install tox plugins12:27
zbralso there is a plugin named tox-venv which enable it to use venv instead of the more problematic virtualenv :D12:28
*** rlandy has joined #zuul12:28
tristanCzbr: or at least the commit message says so12:28
bolgtristanC, tobias: my approach would be to add them part by part as it is touched12:28
zbrtristanC: i did not say that your remark was not correct, but i am looking for a more reliable solution12:29
zbrin the end I could add a conditonal: if params are unchanged, use old behavior, else do the new.12:29
zbrat least, this approach would minimize risks12:29
bolgtristanC: regarding  https://review.opendev.org/#/c/709735/8/zuul/scheduler.py@340 yes you are right, buy IDE is at least not confusing all possible classes implementing those properties :)12:30
tristanCbolg: bolg: don't get me wrong, i am a big fan of type annotation. I'm using emacs with flycheck-mypy, which is using the mypy check implementation, and without the `--strict` argument, mypy simply skip function that are not typed, thus i don't benefit from https://review.opendev.org/#/c/709735/8/zuul/scheduler.py@34012:31
tristanCbolg: and regarding the addition to test file, since zuul ci doesn't check them (mypy only run on the zuul/ directory, not the tests/), then there is a risk the annotation bitrot12:32
bolgtristanC: undestood. I am trying to add them to the relevant code whenever i touch it. Regarding the test, yeah I agree there are bit redundant there12:34
openstackgerritJan Kubovy proposed zuul/zuul master: Improve typings for driver event ingestion  https://review.opendev.org/71729712:36
openstackgerritJan Kubovy proposed zuul/zuul master: WIP: Driver event ingestion  https://review.opendev.org/71729912:43
*** jpena|lunch is now known as jpena12:45
avasscorvus: I started with this: https://review.opendev.org/#/c/717260/4 but I get the feeling that the way we write ansible.cfg should be overhauled instead12:49
*** jamesmcarthur has joined #zuul12:51
avasscorvus: let's say someone wants to use the full_skip callback: https://docs.ansible.com/ansible/latest/plugins/callback/full_skip.html12:55
*** jamesmcarthur has quit IRC12:56
avasscorvus: then we'd have to make sure the config is added to the [defaults] section in ansible.cfg and not create another section called [defaults]12:56
openstackgerritJan Kubovy proposed zuul/zuul master: Scheduler test app manager  https://review.opendev.org/70881212:56
openstackgerritJan Kubovy proposed zuul/zuul master: Use scheduler manager consistently in tests  https://review.opendev.org/70954212:56
openstackgerritJan Kubovy proposed zuul/zuul master: Refactor executor_client in tests  https://review.opendev.org/70967212:56
openstackgerritJan Kubovy proposed zuul/zuul master: Refactor merge_client in tests  https://review.opendev.org/70967612:56
openstackgerritJan Kubovy proposed zuul/zuul master: Refactor nodepool in tests  https://review.opendev.org/70970312:56
openstackgerritJan Kubovy proposed zuul/zuul master: Refactor zookeeper in tests  https://review.opendev.org/70970912:56
openstackgerritJan Kubovy proposed zuul/zuul master: Consolidate scheduler pause/exit as hibernation  https://review.opendev.org/70972312:56
openstackgerritJan Kubovy proposed zuul/zuul master: Refactor `self.event_queues` in tests  https://review.opendev.org/70999012:56
openstackgerritJan Kubovy proposed zuul/zuul master: Scheduler's pause/resume functionality  https://review.opendev.org/70973512:56
openstackgerritJan Kubovy proposed zuul/zuul master: Separate connection registries in tests  https://review.opendev.org/71295812:56
openstackgerritJan Kubovy proposed zuul/zuul master: Prepare Zookeeper for scale-out scheduler  https://review.opendev.org/71726912:56
openstackgerritJan Kubovy proposed zuul/zuul master: WIP: Store unparsed branch config in Zookeeper  https://review.opendev.org/70571612:56
avassso instead of adding a [ansible_callback "<callback_name>"] to zuul.conf we could just add [ansible_config "<ansible.cfg section name>"] that adds data to that section12:57
openstackgerritJan Kubovy proposed zuul/zuul master: Improve typings for driver event ingestion  https://review.opendev.org/71729712:57
openstackgerritJan Kubovy proposed zuul/zuul master: WIP: Driver event ingestion  https://review.opendev.org/71729912:57
zbrtristanC: thanks for ensure-tox comments, going back to implement them. Maybe more important is https://review.opendev.org/#/c/716578/ re install-podman12:58
openstackgerritJan Kubovy proposed zuul/zuul master: Connect merger to Zookeeper  https://review.opendev.org/71622112:59
openstackgerritJan Kubovy proposed zuul/zuul master: Connect executor to Zookeeper  https://review.opendev.org/71626212:59
openstackgerritJan Kubovy proposed zuul/zuul master: Connect fingergw to Zookeeper  https://review.opendev.org/71687512:59
avasscorvus: that would make timeouts/retries and things like that configurable as well12:59
*** rfolco has joined #zuul13:01
bolgtristanC, tobiash, corvus: thanks for your reviews on the scale-out-scheduler topic, since most of them just prepare tests and do not introduce breaking changes, do you think we could go forward with them or something missing?13:02
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: Add phoronix-test-suite job  https://review.opendev.org/67908213:03
openstackgerritSorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded  https://review.opendev.org/69005713:11
tristanCbolg: the topic looks solid to me, at least until the `Scheduler's pause/resume functionality` change13:12
bolgtristanC: yep thats what i meant :) till there we could go forward with it13:13
bolgtristanC: thank for the comments on the pause/resume I will look into it13:19
*** bhavikdbavishi has quit IRC13:26
tobiashtristanC, corvus: I've added a question on zk tls (https://review.opendev.org/712531)13:42
openstackgerritMerged zuul/zuul-operator master: Add nodepool external config  https://review.opendev.org/71531113:52
openstackgerritMerged zuul/zuul-operator master: Adapt the integration playbook to be usable locally  https://review.opendev.org/71416313:52
*** jamesmcarthur has joined #zuul13:52
mordredtobiash: I believe it can be used directly - if you see https://review.opendev.org/#/c/712817/7/doc/source/examples/zoo.cfg13:53
mordredtobiash: (iirc, this was a thing that was discovered as the work progressed, that pem encoding as an option in addition to jks format13:53
AJaegerhere's a zuul-jobs change to add phoronix test jobs, it had previously two +2s and got a small update- anybody wants to review, please? https://review.opendev.org/#/c/679082/13:55
*** zxiiro has joined #zuul13:57
openstackgerritMerged zuul/zuul-operator master: Add nodepool kubernetes pod label to integration test  https://review.opendev.org/71531613:57
*** jamesmcarthur has quit IRC13:57
*** Goneri has joined #zuul13:58
*** ysandeep|rover is now known as ysandeep|away13:59
tobiashmordred: ah cool, thx14:10
tristanCtobiash: would you mind having a look to https://review.opendev.org/706639 it's a spec update to define job volume for the zuul CRD14:23
tobiashtristanC: do you want to change to camelcase in the spec as well?14:25
*** cdearborn has joined #zuul14:25
tristanCtobiash: yep, that's done in https://review.opendev.org/#/c/707190/114:27
tobiashoh that's the followup14:27
tobiashk14:27
openstackgerritMerged zuul/zuul master: Scheduler test app manager  https://review.opendev.org/70881214:28
openstackgerritMerged zuul/zuul master: Use scheduler manager consistently in tests  https://review.opendev.org/70954214:28
openstackgerritMerged zuul/zuul master: Refactor executor_client in tests  https://review.opendev.org/70967214:28
openstackgerritMerged zuul/zuul master: Refactor merge_client in tests  https://review.opendev.org/70967614:28
openstackgerritMerged zuul/zuul master: Refactor nodepool in tests  https://review.opendev.org/70970314:28
openstackgerritMerged zuul/zuul master: Refactor zookeeper in tests  https://review.opendev.org/70970914:29
openstackgerritTobias Henkel proposed zuul/zuul master: Add release note for zookeeper tls support  https://review.opendev.org/71731214:29
tobiashtristanC, corvus, mordred: does that release note look right for zk tls? ^14:30
*** jamesmcarthur has joined #zuul14:30
*** y2kenny has joined #zuul14:30
corvusavass: hrm, do we really want to open the possibility of setting the "defaults" section?  we've wanted very tight control over how zuul interacts with ansible.  i think maybe anything that goes in the defaults section is something we should keep hard-coded in the executor for now....14:31
openstackgerritTobias Henkel proposed zuul/zuul master: Add release note for zookeeper tls support  https://review.opendev.org/71731214:31
corvustobiash: that note looks good, but i also wanted to make the note in zuul have doc links; and then maybe even go back and revise the nodepool note to include them as well.14:33
mnaserso, an update on the zuul website gatsby-ifying14:35
openstackgerritSorin Sbarnea proposed zuul/zuul-jobs master: Add support for RedHat platforms on install-podman  https://review.opendev.org/71657814:35
corvustobiash: the other thing to know about the zk tls stack is that we hit a problem with the intermediate registry which caused https://review.opendev.org/712817 to fail; i've added debugging to the registry server to try to track down the problem if it happens again, but the problem is on upload, and that blob is now permanently corrupt.  we can probably rebase it after its parents land and it might work then14:36
corvus(however, it passed previous testing, so i'm confident in it)14:36
mnaserhttps://i.imgur.com/OMVUC28.png -- this is what it looks like, and that's gatsby-ified :)14:36
corvusavass: i sent you a private message here in irc; can you check that you got it?14:36
mnaseri will push up a patch shortly with the wip and i think i'll takea that opportunity to add some stuff to zuul/zuul-jobs to build a gatsby site14:37
corvusmnaser: that looks pretty much the same to my eyes :)14:37
mnasercorvus: that's exactly the goal ;)14:37
tobiashcorvus: ah ok14:39
corvustobiash: i think maybe just link to the howtos/zookeeper.rst file?14:40
*** EmilienM is now known as EvilienM14:40
corvusi guess we'll need an anchor at the top of that to :ref14:41
tobiashcorvus: yes, fighting with local doc build on mac...14:41
corvustobiash: also could link to :attr:`zookeeper`  maybe14:41
AJaegermnaser: want to review 679082 again, please?14:42
*** jamesmcarthur has quit IRC14:44
*** jamesmcarthur has joined #zuul14:44
AJaegercorvus, zuul-maint, do we want to merge the stack for changing the roles install-X to ensure-X by avass starting at https://review.opendev.org/#/c/716663/ - so, merge quickly before we get more merge conflicts. Or want to wait until a certain point?14:45
*** sgw has joined #zuul14:49
openstackgerritTobias Henkel proposed zuul/zuul master: Add release note for zookeeper tls support  https://review.opendev.org/71731214:50
*** avass is now known as Guest9781414:50
*** avass has joined #zuul14:50
corvusAJaeger: i was thinking maybe start merging those on monday?  give folks a chance to raise any concerns on the mailing list14:50
avasscoruvs: ah sorry, logged off my work computer14:52
AJaegercorvus: that works - let me leave a comment on the first change saying that14:52
avasscorvus: I'm not sure why we want such tight control over ansible.cfg to be honest14:53
corvusavass: most of the settings we have set there are due to lessons we learned the hard way.  things about fact caching, and retries, etc.  i'm open to allowing more customization, but maybe if there isn't a pressing need, we should keep control so we can make sure it works reliably well for all users.14:55
* tobiash remembers many lessons we learned the hard way14:57
*** jamesmcarthur has quit IRC15:00
*** jamesmcarthur has joined #zuul15:00
avasscorvus: well that seems like different things to me, it's still possible to default to values that are known to work well and recommend users to not touch those, or blacklist something15:01
openstackgerritSorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded  https://review.opendev.org/69005715:01
avasscorvus: it's not like it's needed, but adding configuration for callbacks exclusively seems like something that can be handled in a more general way15:02
corvusavass: there's a risk here -- the more users customize that, the harder it is for us to know that it's going to work well and that any future changes we make aren't going to break.  i'm not completely opposed, but i think we should have a very compelling use case.15:02
corvusavass: if you did the [ansible_config "section_name"] approach, you'd still have to merge the callback whitelist15:05
avasscorvus: no since the callback_whitelist would be in [ansible_config "defaults"] :)15:06
avassunless there's a callback hardcoded in zuul of course15:06
corvusavass: then you'd have to hope the user whitelists the zuul callbacks15:06
corvusthere is15:06
*** klindgren_ has quit IRC15:06
corvusi mean, it's not impossible, i'm just pointing out that the general approach isn't necessarily "cleaner" or "easier"15:06
*** klindgren has joined #zuul15:07
corvusavass: i like the approach in 71726015:07
avasscorvus: we'd have to make sure that callback_config_section can't be set to [defaults] or [ssh_connection] in that case15:09
avassI guess the easy approach is to only allow [callback_<name>]15:09
corvuswhat's the difference between callback_plugins and callback_whitelist?15:10
*** jcapitao has quit IRC15:11
avasscorvus: not sure, but I was wondering why ara didn't need to be whitelisted15:11
corvusoh _plugins is the path, right?15:11
avasscorvus: that's my guess15:11
corvusah, the zuul plugin is enabled by setting stdout_callback, so that doesn't collide after all15:12
corvusavass: iiuc, i think maybe builtin plugins need to be enabled by adding to whitelist, but non-built-in-plugins only need to be in the callback_plugins path?15:13
*** yoctozepto has quit IRC15:17
avasscorvus: something else I think we want but not necessary need is to add env variables to ansible or configure callbacks per tenant15:18
avassI'll check15:18
corvusavass: we might need to merge callback_plugins no matter what15:18
avasscorvus: my guess is yes: https://docs.ansible.com/ansible/latest/reference_appendices/config.html#default-callback-whitelist15:18
avasscorvus: that's if we don't want to force people to install the callbacks in the callback directory :)15:18
*** yoctozepto8 has joined #zuul15:18
corvusavass: it's probably worth looking at how the ara plugin is installed and how we find out its directory.  see getAraCallbackPlugin15:18
corvusi think we'd want folks to install plugins by adding them to the extra packages that zuul-manage-ansible installs15:18
avassI'll take a look at that15:21
openstackgerritAlbin Vass proposed zuul/zuul master: WIP: Add support for ansible callbacks  https://review.opendev.org/71726015:26
avasscorvus: ^ how about we do that instead?15:26
avasscorvus: and we only support adding [callback_$foo] to ansible.cfg for now15:27
corvusavass: sounds good to me15:27
avasscorvus: do we want to filter the name in [ansible_callback "<name>"] so it's not possible to add something to [defaults], or should we ust not support that?15:29
avass^ since I guess you can do [ansible_callback "nocows = False"] otherwise :)15:31
*** igordc has joined #zuul15:32
corvusavass: oh i thought your change would only write out "[callback_nocows = False]" in that case?15:32
avasscorvus: it still needs to add it to the whitelist so it would be something like callback_whitelist =\n nocows = False15:33
avasswith nocows = False being indented of course, but I think I checked yesterday and configloader still sees that as a separate value as long as there's an equal sign in there15:35
fungii'm glad i'm not the only one who appreciates cows15:36
* mordred feels like we're missing olaph15:36
corvusavass: ok.  well, we should probably filter it anyway just to keep anything unexpected from happening :)15:37
fungiyep, i miss him every day15:37
avasscorvus: I'm looking forward to the "ensureNoCows" testcase :)15:37
corvusmordred: have a way to send him a message?  http://paste.openstack.org/show/791596/15:39
corvusavass: heh, i'm kind of surprised we don't have one already :)15:40
corvusthough, i bet at this point, if we turned on cows, a bunch of tests would break15:40
*** rpittau is now known as rpittau|afk15:43
mordredcorvus: sent!15:44
*** gtema has quit IRC15:48
*** ysandeep|away is now known as ysandeep15:49
*** ysandeep is now known as ysandeep|away15:56
y2kennyFor the build summary page, the web interface fetch zuul-manifest.json from the log server.  Does the log server have to be the same as the web UI server?  I am getting "Network Error (Unable to fetch URL...)" even though I can copy and paste the url and the file is there.16:10
fungiy2kenny: it does not have to be the same (we certainly don't run it that way in opendev) but you may need to do some work to convince browsers it's not a cross-site/cross-domain exploit16:12
fungitypically using cors headers16:12
tobiashmordred: did you validate that the ansi color code removal worked for the line comments?16:12
tobiashjudging from my test case the regex doesn't seem to match the escape sequences16:13
mordredtobiash: yes - lemme see if I can find the patch16:13
y2kennyI was wondering about that but I thought fetch/get is not part of the cross-site stuff (but I haven't touch those in a long time.)  fungi, when you sing using cors headers, which part of the system do I need to modify?16:14
mordredtobiash: https://review.opendev.org/#/c/716604/16:14
mordredtobiash: the pylint comments there had ansi codes16:14
y2kennysaid*16:14
tobiashweird16:14
mordredy2kenny: the system hosting the log files16:14
y2kennyok16:15
fungiy2kenny: i think it'll be on the site where the logs are hosted, so say that requests from $other_site are allowed16:15
mordredy2kenny: it needs to publish cors headers allowing the domain where the dashboard is hosted16:15
clarkby2kenny: the webserver serving the content (log server in your case?) needs to set access-control-allow-origin header to include the domain that is retrieving the content (the dashboard url) or you can *16:15
tobiashmordred: oh, looks like this missed the binary char in front of the sequence https://review.opendev.org/#/c/716623/4/roles/tox/files/cinder-pylint-output.txt16:15
y2kennyI will dig into that.  Thanks fungi, mordred, clarkb16:15
fungiyou bet, glad i knew the answer for a change ;)16:16
mordredtobiash: oh - might have been copy-paste :(16:16
tobiashyeah, something seems to have filtered it16:17
corvusmordred, tristanC: in https://review.opendev.org/716305 i noticed in the preview that the refresh button has changed; do we want to try to get it to look like it currently does, or go with the new look?16:32
tristanCy2kenny: is there something setting the Content-Security-Policy header on your zuul-web ui?16:32
tristanCcorvus: oh indeed... well i don't mind the new look :)16:33
tristanCthere is proably a css expression that would make it look better, but i don't know which one16:35
mordredcorvus: yeah - I don't mind the new look, but also would be fine trying to get it to match the old look16:35
*** evrardjp has quit IRC16:36
*** evrardjp has joined #zuul16:37
corvustobiash: which patch were you running with that caused a leak?16:37
corvustobiash: memory leak16:37
tobiashcorvus: https://review.opendev.org/71485216:38
corvustobiash: ok.  i was about to look at https://review.opendev.org/712939 and wanted to make sure that wasn't it :)16:38
tobiashit hasn't merged16:38
tobiashcorvus: that one is in prod since three weeks :)16:39
*** y2kenny has quit IRC16:41
*** yoctozepto8 is now known as yoctozepto16:43
openstackgerritTobias Henkel proposed zuul/zuul-jobs master: Support multiple matchers when parsing tox output  https://review.opendev.org/71626316:46
openstackgerritTobias Henkel proposed zuul/zuul-jobs master: Don't silently ignore exceptions when parsing tox output  https://review.opendev.org/71676616:46
openstackgerritTobias Henkel proposed zuul/zuul-jobs master: Strip source dir from file comments  https://review.opendev.org/71626416:47
tobiashmnaser: ^16:47
openstackgerritTobias Henkel proposed zuul/zuul-jobs master: Ignore absolute paths after stripping work dir  https://review.opendev.org/71704216:47
openstackgerritTobias Henkel proposed zuul/zuul-jobs master: Add test cases for tox line comment parsing  https://review.opendev.org/71734116:47
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Update dhall-kubernetes version  https://review.opendev.org/71734216:47
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Update dhall-kubernetes version to v4.0.0  https://review.opendev.org/71734216:51
*** bhavikdbavishi has joined #zuul16:57
*** gtema has joined #zuul17:11
*** jpena is now known as jpena|off17:14
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Add job_volumes CR spec attribute  https://review.opendev.org/70664217:14
*** bhavikdbavishi1 has joined #zuul17:18
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Update attributes to camelCase  https://review.opendev.org/70719317:19
*** gtema has quit IRC17:19
*** gtema has joined #zuul17:20
*** bhavikdbavishi has quit IRC17:20
*** bhavikdbavishi1 is now known as bhavikdbavishi17:20
*** harrymichal has quit IRC17:21
openstackgerritTobias Henkel proposed zuul/zuul-jobs master: Add test cases for tox line comment parsing  https://review.opendev.org/71734117:22
*** gtema has quit IRC17:25
*** jamesmcarthur has quit IRC17:26
*** jamesmcarthur has joined #zuul17:29
*** dpawlik has quit IRC17:30
*** jamesmcarthur has quit IRC17:33
*** jamesmcarthur has joined #zuul17:34
openstackgerritTobias Henkel proposed zuul/zuul-jobs master: Add test cases for tox line comment parsing  https://review.opendev.org/71734117:34
openstackgerritClark Boylan proposed zuul/nodepool master: Check dib image build timeout more often  https://review.opendev.org/71735717:39
clarkbfungi: ianw ^ I think that might be at least semi related to the dib process leaks17:39
zbrclarkb: corvus : does install-podman at https://review.opendev.org/#/c/716578/ look ready?17:41
clarkbzbr: I think it should be ensure-podman to go along with the refactor that avass is doing17:42
clarkbzbr: the idea being things might be preinstalled or not and ensure comunicates that a bit better17:42
zbr:(17:42
zbrclarkb: i was not aware of ensure-podman work, but i do support it. still why not using a symlink instead for the rename?17:48
zbrand more important: that refactoring does not bring any "meat" for me at the table as it does not enable me to install podman on redhat platforms.17:49
zbrif we merge avass work soon, like today, i am happy to rebase my work on top of it.17:50
zbras long we endup with a zuul role that makes podman available on the node, i will be very happy.17:50
tobiashzbr: afair above avass work will be merged on monday17:51
clarkbI guess we can rebase the other direction17:52
zbrdid I understand correctly that the direction is to replace install-* with ensure-* in order to make it more obvious for users that roles are about getting to a state and not about an "action"17:52
clarkbzbr: that was my understanding17:53
zbrfor me ensure ~= install, at least from the configuration management point of view.17:53
zbrobiously that if something is already installed (and working), you should succeed :D17:54
clarkbzbr: part of it is we had both mixed in depending on the role. So the goal was to be consistent and ensure was chosen as it better reflects the inteint17:54
clarkb*intent17:54
zbrnothing against, it already faced at least one confused user.17:54
*** hashar is now known as hasharDinner17:55
zbrwhat is not fully clear to me is why having both, i would have used symlinks and only document that we will remove them after a while.17:57
clarkbzbr: as a transition. Symlinks may work though ime they should be avoided for artifacts like this as half they time they don't do what you expect17:58
clarkbthey break python packaging for example17:58
clarkband wouldn't work on windows machines17:59
zbrclarkb: as long you do not deploy on windows they work even with packages :D17:59
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Add job_volumes CR spec attribute  https://review.opendev.org/70664218:00
openstackgerritTristan Cacqueray proposed zuul/zuul-operator master: Update attributes to camelCase  https://review.opendev.org/70719318:00
zbrdoes zuul-jobs work on windows? last time i check it was impossible to run tox -e linters even on BSD systems due to use of incompatible find args18:02
clarkbzbr: they are ansible roles and ansible runs on windows. I don't expect they would be used in a zuul context though18:03
fungizbr: i would consider reasonable portability adaptations to roles/playbooks/jobs folks might want to run on those platforms as welcome18:04
fungibut only if folks are really trying to use them from zuul18:04
zbrfungi: zbr: the only reason why you didn't see a PR already is that I was waiting for the new ansible-lint release in order to make it.18:04
fungiand not just gratuitous portability work for the sake of it even if nobody might every use it18:04
zbrmy desktop is a fruity BSD one so I am in the first line of finding such bugs, most of them are very easy to fix.18:05
fungii do run a lot of openbsd, but i've not been bold enough to try using it for my workstation18:06
fungii mostly rely on it for headless network devices18:06
zbrmacos18:06
fungioh, "fruit"18:07
fungigot it18:07
zbrthe rotten one18:07
fungiheh18:07
zbrbtw, if you have time to read (and vote) on https://github.com/ansible/ansible-lint/issues/716 it would be great18:08
zbri started to work on adding a "molecule_whitelist" to it in order to avoid hacks like we needed for zuul_return, but progress is kinda slow, maybe I will have something ready by monday.18:10
zbrerrata: "module_whitelist"18:11
openstackgerritMerged zuul/zuul master: Protect getCachedChanges from concurrent modification  https://review.opendev.org/71527018:11
openstackgerritMerged zuul/zuul master: Install unzip on all platforms  https://review.opendev.org/71491918:15
corvuszbr: there's a thread on the zuul-discuss list: http://lists.zuul-ci.org/pipermail/zuul-discuss/2020-April/001203.html18:16
corvuszbr: i'm looking for a review from mnaser on https://review.opendev.org/716578 before we proceed since he had concerns earlier18:18
zbrhehe, yesterday mail is still backlog. i will reply to show my support for the move, is a good one.18:18
corvuszbr: cool, thx :)18:19
corvuswe're just waiting till monday since it's a big move in case anyone had any concerns, and they had a mail backlog :)18:20
corvuszbr: and probably a review from tristanC too18:21
corvus(re the podman change)18:21
openstackgerritJames E. Blair proposed zuul/nodepool master: Fix shutdown ordering  https://review.opendev.org/71713418:26
openstackgerritJames E. Blair proposed zuul/nodepool master: Add GCE driver tests  https://review.opendev.org/71713318:26
*** rmart04 has quit IRC18:38
*** jamesmcarthur has quit IRC18:39
*** hasharDinner has quit IRC18:55
*** hashar has joined #zuul18:57
openstackgerritMohammed Naser proposed zuul/zuul-website master: WIP: switch website to Gatsby  https://review.opendev.org/71737119:05
clarkbLooking into that nodepool builder behavior with dib processes leaking I think my change is not necessary (its redundant for how control flow happens there and I missed it on my first pass)19:11
clarkbwhat I have noticed is that we do p.wait() and the subprocess docs warn this could deadlock with using PIPE19:11
clarkbI wonder if we are deadlocking there which would also explain the lack of logging about timeouts19:12
clarkbtobiash: ^ hvae you ever seen that with dib builds under nodepool-builder? basically over time disk-image-create processes accumulate19:12
mnaserzbr: i hate to be annoying but given we're moving towards ensure-* -- could you revise that so we don't have to rename it after?19:12
mnasersorry, i know you've been waiting to land that for a while19:12
mnaserbtw, for those interested, you can clone that gatsby change locally and run 'yarn install' and you should be good to go with it running locally19:12
mnaseri will be working on some zuul jobs/roles19:13
mnaserand take advantage of that to add some generic jobs in zuul-jobs19:13
clarkbmnaser: ignoring the netlify tooling of the potential future, what does gatsy workflow look like for us? its edit markdown, run build, copy/publish output basically the same workflow as sphinx ya?19:14
fungimnaser: yeah, it sounds like zbr is already on board with the install->ensure rename and is rebasing onto that19:15
tobiashclarkb: I didn't notice leaked dib-image-create processes, but I can have a look on our builder tomorrow if I find any19:16
clarkbtobiash: thanks19:16
openstackgerritMerged zuul/zuul master: Consolidate scheduler pause/exit as hibernation  https://review.opendev.org/70972319:23
*** bhavikdbavishi has quit IRC19:23
openstackgerritMohammed Naser proposed zuul/zuul-website master: WIP: switch website to Gatsby  https://review.opendev.org/71737119:24
*** bhavikdbavishi has joined #zuul19:25
openstackgerritMohammed Naser proposed zuul/zuul-website master: WIP: switch website to Gatsby  https://review.opendev.org/71737119:26
*** gtema has joined #zuul19:26
openstackgerritMerged zuul/zuul master: Refactor `self.event_queues` in tests  https://review.opendev.org/70999019:27
*** jamesmcarthur has joined #zuul19:28
*** jamesmcarthur has quit IRC19:28
*** jamesmcarthur has joined #zuul19:29
openstackgerritMohammed Naser proposed zuul/zuul-website master: WIP: switch website to Gatsby  https://review.opendev.org/71737119:29
mnaserclarkb: yep19:30
*** gtema has quit IRC19:30
mnaserclarkb: now just to clarify that we will not be committing built output though, so you dont really need to run build, you can use 'yarn develop' which will start a local server too19:31
clarkbright we don't commit sphinx output either19:31
mnaseryeah19:31
mnaseroops our yarn role is broken19:32
mnaserWarning: apt-key output should not be parsed (stdout is not a terminal)\ngpg: failed to start agent '/usr/bin/gpg-agent': No such file or directory\ngpg: can't connect to the agent: No such file or directory19:32
AJaegeranybody wants to review the tests for the parser jobs https://review.opendev.org/#/c/717341/? Then we can merge the complete stack. mordred , corvus, what do you think?19:36
corvusAJaeger: i'd love an ack from mnaser on https://review.opendev.org/717341 but then i agree, merge it all19:38
openstackgerritMohammed Naser proposed zuul/zuul-jobs master: install-yarn: add coverage for all platforms  https://review.opendev.org/71737519:38
clarkbmnaser: gpg2 runs an agent daemon. I'm not sure if that is a hard dep or not, but if it is a hard dep you'll need to ensure that tool is installed I bet19:38
clarkb(its annoying bceause if you want ot load a key for a short period of time or off removable media you have to go find that process and kill it19:38
mnaserAJaeger, corvus: oh that's awesome.  i think i would like to make a small revision on top of it where we add tests/__init__.py inside 'files' for the golangci-lint jobs.  just to make sure that didn't break19:39
mnaserlet me make a patch on top of it to check that its ok19:40
AJaegertobiash: one question on 717341: I don't see where the tests run. The tests look fine but I must be blind or something is broken ;(19:41
AJaegermnaser: sure, go for it.19:41
mnaserAJaeger: it should run inside the tox-pyXX envs19:42
* AJaeger just removed his +2 after not finding the test results in the jobs that run.19:42
AJaegermnaser: those were not triggered19:42
AJaegermnaser: AH!19:42
* AJaeger is probably blind, will check19:42
tobiashAJaeger: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b36/717341/3/check/tox-py37/b36c608/testr_results.html19:42
mnaserAJaeger: wait, nevermind, that also means the golang ones are running already too19:43
mnaseri potato'd there ;)19:43
mnaser+2 from me, thanks tobiash !19:43
tobiashIt's tested in the py* tests19:43
AJaegertobiash: yeah, found them finally. Thanks19:43
AJaegermnaser: so, I'll +A, ok?19:43
corvusAJaeger: beat you to it19:43
AJaegercorvus: no extra work for me -great ;) Thanks!19:44
tobiash\o/19:45
mnaserhmm19:45
openstackgerritMerged zuul/zuul-jobs master: Add phoronix-test-suite job  https://review.opendev.org/67908219:46
mnaserdoes anyone know why our install-yarn role backs out if there is no yarn.lock file?19:46
corvusmordred: ^19:46
mordreduh19:46
mnaseri can understand bailing out in the actual 'yarn' aka run yarn role19:47
mnaserbut in install-yarn that seems much -- seee https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/install-yarn/tasks/main.yaml19:47
mordredmnaser: I think maybe because we also use it in some python jobs - and we're only wanting to opportunitistically install yarn if it's needed19:47
mordredbut - that might not be the best design choice19:47
*** dpawlik has joined #zuul19:48
mnaserseems like 'javascript' is the only playbook which references it19:48
mordredI thnk what those python jobs that want it really want is "install-yarn-if-project-needs-it"19:48
mnaserhttps://opendev.org/zuul/zuul-jobs/src/branch/master/playbooks/javascript/pre.yaml is the only reference ive found for usage19:48
mnaserlet me check hound19:48
mordredplaybooks/release/pre.yaml in zuul19:48
mnaserozj https://opendev.org/openstack/openstack-zuul-jobs/src/branch/master/playbooks/javascript/pre.yaml too19:48
corvusmnaser: (side note: be aware the error zuul is reporting on 717371 is due to the fact that the zuul-website-media repo uses the same jobs -- they work cooperatively to publish the site)19:48
mordredbut yeah - I don't see any places that use it where it being conditional is useful19:48
mnasercorvus: oh thanks, i wasn't sure right off the bat so figured "leave them in there for now" was my workaround :p19:49
corvusmnaser: yeah, i think that's good, then we can update -media to work again19:50
mnaserok id be in favour of dropping yarn.lock referencing in install-yarn, because it doesn't seem like i can find a case where it maeks sense19:50
corvuswe're not going to upload any giant binary blobs in the next little bit19:50
mnaserthanks for the context mordred :>19:50
corvusmnaser: we run playbooks/javascript/pre.yaml in python jobs19:51
corvusmnaser: well, looks like only tox-nodejs-npm19:51
corvusthat sounds like it's the sort of job we could assume yarn is wanted?19:51
mnasercorvus: i would agree with that19:52
corvusmnaser: yeah, i think i'm on board19:52
mnaserit also follows our consistency with other roles19:52
mnaseri.e. we do install tox regardless of you having a tox.ini file or not19:52
clarkb++ the role including side can make it conditional if it needs that19:54
clarkbbut the role itself should maybe be less conditional unless the necessary bits are already there (in this case yarn is installed already so noop)19:54
openstackgerritMohammed Naser proposed zuul/zuul-jobs master: install-yarn: always install  https://review.opendev.org/71737519:56
mnasercool, so that should also properly test it too19:57
openstackgerritMerged zuul/zuul-jobs master: Support multiple matchers when parsing tox output  https://review.opendev.org/71626320:00
openstackgerritMerged zuul/zuul-jobs master: Don't silently ignore exceptions when parsing tox output  https://review.opendev.org/71676620:00
openstackgerritMerged zuul/zuul-jobs master: Strip source dir from file comments  https://review.opendev.org/71626420:00
openstackgerritMerged zuul/zuul-jobs master: Ignore absolute paths after stripping work dir  https://review.opendev.org/71704220:00
corvusmnaser, clarkb, fungi: i think the gpg thing is more widespread20:01
corvushttps://zuul.opendev.org/t/zuul/build/5d5fbbc8e3894e1e821bba396c991e2220:01
corvusthe last ps of that change did not fail that job20:01
mnaserhmm20:01
mnaserhave we changed opendev's image recently?20:01
clarkbmnaser: should be every day :)20:01
corvushttps://review.opendev.org/716785 was recent20:02
mnaserhmm20:02
openstackgerritMerged zuul/zuul-jobs master: Add test cases for tox line comment parsing  https://review.opendev.org/71734120:02
mnaserwe have a -plain image now?20:02
clarkbgnupg hard deps on gpg-agent on bionic20:02
clarkb(according to ubuntu package search)20:03
clarkbwe shouldn't be able to install gnupg without gpg-agent20:03
*** saneax has quit IRC20:03
corvusdidn't we used to output the image build info in the logs?20:03
clarkbmnaser: thats related to python things (basically no more pip and virtualenv in the root install)20:03
corvushere we go: https://zuul.opendev.org/t/zuul/build/5d5fbbc8e3894e1e821bba396c991e22/log/zuul-info/zuul-info.ubuntu-bionic.txt#620:04
corvus2020-04-03 17:5320:04
corvusso indeed, that is a newly built image, after https://review.opendev.org/716785 merged20:04
mnaserit seems that gnupg is a recommended package only?20:04
mnaserhttps://nb01.openstack.org/ubuntu-bionic-0000104264.log ctrl+f gnupg20:05
corvus(i'm not 100% sure 716785 was actually deployed, that's not clear since that's currently work in progress in #opendev)20:05
mnaserand gnupg2 is also just suggested20:05
clarkbmnaser: but `gpg` is tere according to the warning message20:06
clarkber the line after the warning20:06
clarkbgpg is saying "I can't start my agent"20:06
* clarkb checks gnupg2 too deps20:07
clarkbgnupg2 deps gnupg which deps gpg-agent20:07
clarkb(those are all requires too not suggests or recommends)20:07
mnaserso20:07
clarkbfungi: ^ seems like you keep up with debian packaging, any idea how we can have a gpg intsalled without a gpg-agent binary?20:07
mnaserhttps://nb01.openstack.org/ubuntu-bionic-plain-0000000031.log installed gnupg20:08
mnaserhttps://nb01.openstack.org/ubuntu-bionic-plain-0000000043.log did not20:08
corvusshould we switch to #opendev?20:08
mnasergood point20:08
corvus(this is starting to look like an opendev problem)20:08
openstackgerritMerged zuul/zuul-jobs master: helm: collect kubernetes logs in post  https://review.opendev.org/71570920:14
*** y2kenny has joined #zuul20:26
mnaserhmm20:26
mnaseri guess we'll have to add install-nodejs for install-yarn20:27
mnaseralmost every single platform is broken because it either doesnt ship node or ships one thats too old20:27
y2kennytristanC: hey sorry, I had a 3 hrs meeting.  How do I check the Content-Security-Header for the zuul webui?20:27
fungiy2kenny: you could use a tool like wget or curl to inspect the http headers, or your browser may have a debugging console (either built in or as a popula "web developer" extension)20:31
y2kennyok... so the content security policy is on the zuul web UI side and not the log serving side?20:31
tristanCy2kenny: yes, that's something that can be set by a static file provider to prevent further html/javascript action from happening20:36
fungiright, it may or may not be present, if it is present you may have to adjust it20:36
fungiif it's not present then it's not the cause of the problem20:36
y2kennytristanC: I don't see that header from either side.  But I do have a 404 on job-output.json.gz20:37
y2kennyeven though I have job-output.json in the directory20:37
y2kennyand the network tab shows the get for zuul-manifest.json is also successful20:37
tristanCy2kenny: that's normal, iirc the webui try the .gz version first, then foldback to the uncompressed version20:37
tristanCy2kenny: what's the issue again?20:37
y2kennywhen I load the build summary, I get an error stating Network error20:38
y2kenny(hold on... the pop up disappeared)20:38
y2kennyNetwork Error (Unable to fetch URL, check your network connectivity, browser plugins, ad-blockers, or try to refresh this page) ...1d5ecd8/zuul-manifest.json20:39
y2kennyI think adding the CORS header on the log serving side gave me more stuff now20:39
y2kennybut the error still pops up20:39
y2kenny(I get the pretty UI :D)20:39
fungifor opendev's deployment, our logs are being served from a different site (actually a bunch of different sites in different domains) so we configure those sites to include "Access-Control-Allow-Origin: *" in the headers20:40
y2kenny(the console tab)20:40
fungiwhich is the most common cause of the problem20:40
y2kennyI added the header20:40
fungiokay, so if you fetch a zuul-manifest.json file you see that included in the headers of the response20:41
y2kennyand now I see the console tab but for some reason I still get pop up error... even though I think the webui actually picked up all the necessary info20:41
fungimaybe a cached response?20:42
y2kennyI thought something may be cached but shift-reload didn't remove it.... trying on Chrome (which never loaded the page before) it's fine... yea... it's probably some cache thing20:44
tristanCcorvus: btw, i added a job-volume test in https://review.opendev.org/706642 . that's something we'd like to start using and it would be convenient if the feature get merged soon.20:44
y2kennyum.... let me see what happens if I try to load the page in firefox private mode20:44
y2kennyprivate mode also no error... so it's probably some lingering thing.  Anyway, thanks guys.20:45
fungithese days the only way to be sure you've cleared a browser cache is to nuke it from orbit20:50
fungitake an angle grinder to your ssd and polish off the bits20:50
y2kennyI remember the days when a super strong magnet would've done it...  ;)20:51
fungiyup, i used to have to use a degaussing table to clear our sensitive media at a job many beers ago20:53
*** EvilienM is now known as EmilienM21:07
*** jamesmcarthur has quit IRC21:23
*** jamesmcarthur has joined #zuul21:24
*** jamesmcarthur has quit IRC21:29
*** bhavikdbavishi has quit IRC21:30
*** jamesmcarthur has joined #zuul21:36
*** hashar has quit IRC22:05
*** armstrongs has joined #zuul22:08
*** rlandy has quit IRC22:12
*** armstrongs has quit IRC22:18
*** dpawlik has quit IRC22:32
*** igordc has quit IRC22:36
*** igordc has joined #zuul22:36
*** tosky has quit IRC23:16
*** jamesmcarthur has quit IRC23:19
*** jamesmcarthur has joined #zuul23:20
*** jamesmcarthur has quit IRC23:25
*** jamesmcarthur has joined #zuul23:29

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!