Thursday, 2020-05-28

*** jamesmcarthur has quit IRC00:00
*** jamesmcarthur has joined #zuul00:04
*** threestrands has joined #zuul00:09
*** jamesmcarthur has quit IRC00:13
*** rlandy has quit IRC00:14
*** jamesmcarthur has joined #zuul00:14
*** jamesmcarthur has quit IRC00:22
*** jamesmcarthur has joined #zuul00:54
*** jamesmcarthur has quit IRC01:05
*** ysandeep|away is now known as ysandeep01:16
*** jamesmcarthur has joined #zuul01:30
*** swest has quit IRC01:31
*** sshnaidm|afk is now known as sshnaidm|off01:34
*** jamesmcarthur has quit IRC01:36
*** cloudnull has quit IRC01:43
*** swest has joined #zuul01:46
*** jamesmcarthur has joined #zuul01:46
*** cloudnull has joined #zuul01:51
*** jamesmcarthur has quit IRC01:54
*** jamesmcarthur has joined #zuul01:55
*** jamesmcarthur has quit IRC02:02
*** jamesmcarthur has joined #zuul02:29
*** jamesmcarthur has quit IRC02:43
*** threestrands has quit IRC03:49
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: WIP: Sync with configure-mirrors  https://review.opendev.org/73129304:17
*** jamesmcarthur has joined #zuul04:25
*** evrardjp has quit IRC04:33
*** evrardjp has joined #zuul04:33
*** jamesmcarthur has quit IRC04:56
*** jamesmcarthur_ has joined #zuul04:56
*** sgw has quit IRC05:06
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Add configure-os-mirrors role  https://review.opendev.org/67757805:18
AJaegeravass: I squashed our changes together ^05:26
AJaegerzuul-jobs-main, https://review.opendev.org/677578 is the new configure-os-mirrors role and ready for review, please have a look - and let's discuss how to migrate to it.05:30
*** reiterative has quit IRC05:40
*** reiterative has joined #zuul05:40
*** dpawlik has joined #zuul06:07
openstackgerritAndreas Jaeger proposed zuul/zuul-jobs master: Fix typo: Deprecrated  https://review.opendev.org/73143906:10
openstackgerritMerged zuul/zuul master: last3x release note: fix a typo  https://review.opendev.org/73137206:12
openstackgerritLida Liu proposed zuul/zuul master: WIP: Add commit id to Change for mqtt reporter  https://review.opendev.org/72247806:36
*** yolanda has joined #zuul06:43
*** hashar has joined #zuul06:51
*** saneax has joined #zuul06:59
avassAJaeger: I don't think we can filter by checking the distribution in the suite07:05
avassAJaeger: So that would have to be solved. Maybe by adding a 'release' field for each item or what fungi suggested: grouping them by 'ubuntu-bionic', 'ubuntu-xenial' etc instead of just 'ubuntu'07:07
*** asaleh_ has joined #zuul07:07
AJaegeravass: I didn't really follow that discussion, do you have a good idea on how to update?07:09
openstackgerritMerged zuul/zuul-jobs master: Fix typo: Deprecrated  https://review.opendev.org/73143907:11
avassAJaeger: I added a comment, currently we're filtering by checking if the suite includes the release of the distribution but that doesn't have to be the case07:11
avassAJaeger: and if it does we add the configuration to sources.list07:11
*** jamesmcarthur_ has quit IRC07:12
* AJaeger hopes somebody has a good idea07:13
*** jamesmcarthur has joined #zuul07:13
avassAJaeger: it's convention to add the release to the suites name so it will probably work until someone wants to add a repository that doesn't follow the convention :)07:14
*** jcapitao has joined #zuul07:14
AJaegerbad luck ;)07:17
*** jamesmcarthur has quit IRC07:19
*** jpena|off is now known as jpena07:35
*** tosky has joined #zuul07:44
openstackgerritLida Liu proposed zuul/zuul master: WIP: Add commit id to Change for mqtt reporter  https://review.opendev.org/72247807:48
zbravass: AJaeger: can you help with https://review.opendev.org/#/c/702679/ ?07:52
AJaegerzbr: such linter changes should get some more review, so let's ask later today again, please07:53
AJaegermordred, corvus, clarkb ^07:53
zbrmaybe i should split changes out of it to merge them before reconfiguring the linter?07:54
*** jamesmcarthur has joined #zuul07:54
AJaegerzbr: going to 4.3 without any noqas etc., then fixing the bugs, then adding noqa would be perfect07:59
zbrAJaeger: makes no sense what you say, cannot bump a linter w/o skipping its new rules.08:00
zbryou either fix them first, or ignore them.08:00
AJaegerzbr: ignore them in the first change08:01
*** jamesmcarthur has quit IRC08:05
*** nils has joined #zuul08:06
openstackgerritSorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Bump ansible-lint to 4.3.0  https://review.opendev.org/70267908:25
openstackgerritJan Kubovy proposed zuul/zuul master: Connect merger to Zookeeper  https://review.opendev.org/71622108:39
openstackgerritJan Kubovy proposed zuul/zuul master: Connect executor to Zookeeper  https://review.opendev.org/71626208:39
openstackgerritJan Kubovy proposed zuul/zuul master: Connect fingergw to Zookeeper  https://review.opendev.org/71687508:39
openstackgerritSorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Enable linting of test-playbooks  https://review.opendev.org/73147108:45
openstackgerritSorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Bump ansible-lint to 4.3.0  https://review.opendev.org/70267908:52
*** rpittau|afk is now known as rpittau08:53
openstackgerritJan Kubovy proposed zuul/zuul master: Enhance ZK CA script and doku  https://review.opendev.org/73147509:10
openstackgerritJan Kubovy proposed zuul/zuul master: Enhance ZK CA script and doku  https://review.opendev.org/73147509:11
openstackgerritJan Kubovy proposed zuul/zuul master: Enhance ZK CA script and doku  https://review.opendev.org/73147509:18
*** sugaar has quit IRC09:22
*** sugaar has joined #zuul09:30
openstackgerritLida Liu proposed zuul/zuul master: WIP: Add commit id to Change for mqtt reporter  https://review.opendev.org/72247809:34
*** ysandeep is now known as ysandeep|lunch09:35
openstackgerritSorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Bump ansible-lint to 4.3.0  https://review.opendev.org/70267909:45
openstackgerritAlbin Vass proposed zuul/zuul master: Use absolute path to toolsdir for zk-ca.sh config  https://review.opendev.org/73148409:49
openstackgerritAlbin Vass proposed zuul/zuul master: Use Absolute path in zk-ca.sh for openssl config  https://review.opendev.org/73148409:51
openstackgerritAlbin Vass proposed zuul/zuul master: Use Absolute path in zk-ca.sh for openssl config  https://review.opendev.org/73148409:52
*** tosky_ has joined #zuul09:57
*** tosky is now known as Guest4863309:57
*** tosky_ is now known as tosky09:57
openstackgerritSorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Bump ansible-lint to 4.3.0  https://review.opendev.org/70267910:09
*** rpittau is now known as rpittau|bbl10:18
*** ysandeep|lunch is now known as ysandeep10:18
*** jamesmcarthur has joined #zuul10:39
openstackgerritLida Liu proposed zuul/zuul master: WIP: Add commit id to Change for mqtt reporter  https://review.opendev.org/72247810:46
*** jamesmcarthur has quit IRC10:48
*** sshnaidm|off has quit IRC10:55
*** jcapitao is now known as jcapitao_lunch10:57
*** fbo|off is now known as fbo11:05
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: shake-build: add shake build system job  https://review.opendev.org/73136511:10
zbravass: AJaeger: ansible-lint bumping ^ is ready for rreview, w/o any new skips.11:15
*** jamesmcarthur has joined #zuul11:18
*** jamesmcarthur_ has joined #zuul11:19
*** jamesmcarthur has quit IRC11:23
*** jamesmcarthur_ has quit IRC11:28
tobiashcorvus, clarkb: does opendev still offload the javascript to apache/nginx? I just noticed that the opendev-promote-javascript-deployment job fails in the promote pipeline. Maybe that's the cause of not having the timezone selector in opendev11:35
*** jpena is now known as jpena|lunch11:39
*** rpittau|bbl is now known as rpittau12:05
*** jcapitao_lunch is now known as jcapitao12:13
*** rlandy has joined #zuul12:16
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add the namespace in wait-for-pods role  https://review.opendev.org/73129212:44
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add namespace in the collect-k8s-logs role  https://review.opendev.org/73131912:44
*** jpena|lunch is now known as jpena12:44
*** Goneri has joined #zuul12:46
openstackgerritTristan Cacqueray proposed zuul/zuul-jobs master: shake-build: add shake build system job  https://review.opendev.org/73136512:48
openstackgerritLida Liu proposed zuul/zuul master: WIP: Add commit id to Change for mqtt reporter  https://review.opendev.org/72247812:49
*** harrymichal has joined #zuul12:49
*** rlandy is now known as rlandy|mtg13:03
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add addons args in ensure-kubernetes role  https://review.opendev.org/73152313:06
*** sanjayu_ has joined #zuul13:08
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add the namespace in wait-for-pods role  https://review.opendev.org/73129213:09
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add namespace in the collect-k8s-logs role  https://review.opendev.org/73131913:09
*** saneax has quit IRC13:10
openstackgerritAlbin Vass proposed zuul/zuul master: zookeeper-howto: update to match zk-ca.sh  https://review.opendev.org/73152613:25
openstackgerritMerged zuul/zuul-jobs master: shake-build: add shake build system job  https://review.opendev.org/73136513:28
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: upload-artifactory: fix variable misspelling  https://review.opendev.org/73153513:37
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: upload-artifactory: fix variable misspelling  https://review.opendev.org/73153513:38
avassAJaeger: that ^ should be fine right? The documentation says 'instances' while the code was using 'instance' and I hardly believe someone other than us uses that yet13:39
avassAJaeger: also, I noticed I probably changed that role enough so I don't think that actually needs a trusted context13:39
avassAJaeger: it was using lookups earlier so I think I just got it stuck in my head that it needed a trusted context :)13:42
Open10K8SHi team.13:47
Open10K8SI made 3 patchsets on zuul-jobs repo.13:47
Open10K8SI used them on https://review.opendev.org/#/c/731262/13:47
*** sgw has joined #zuul13:47
Open10K8Splease check them13:47
Open10K8SAJaeger: updated the readme13:47
AJaegerOpen10K8S: thanks, will review later...13:48
Open10K8SAJaeger: ok, thank you.13:48
Open10K8Swill wait.13:48
Open10K8SAlso made another one related to the ensure-kubernetes role13:48
Open10K8SOne PS is waiting them to merge13:48
mordredOpen10K8S: re: adding the namespace argument to the kubectl command - does that work with the nodepool k8s-namespace driver? (I don't lknow if these roles work with that or not)13:54
clarkbtobiash: I dont think so. We deploy from the containers now but it is possible that our apache is still serving old files?13:54
clarkbmordred: ^ see tobiash's not about opemdev's dashboard missing the new timezone selector13:55
mordredclarkb: maybe we haven't restarted our zuul-web container?13:55
mordred(since we don't do static offload anymore, we don't get insta-deploys of javascript)13:55
clarkbmordred: corvus did that yesterday aiui13:57
mordredah - hrm13:57
mordredclarkb: 9e200492a0f1        bae28a4c5273        "/usr/bin/dumb-init …"   12 days ago         Up 15 hours                             zuul-web_web_113:58
mordredclarkb: the container was definitely restarted 15 hours ago - but it says created 12 days ago13:59
clarkbhrm maybe we havent built and published a new web image for some reason13:59
mordrednope - we have new images13:59
mordredon the host are:13:59
mordredzuul/zuul-web         latest              2b24f510e073        8 hours ago         455MB13:59
mordredzuul/zuul-web         <none>              bae28a4c5273        13 days ago         471MB14:00
mordredwe're running bae28a4c527314:00
fungiso haven't retrieved a new image?14:00
fungioh, on the host14:00
clarkbfungi: looks like we have14:00
fungiso we retrieved it but restarted on the old one14:00
mordredyeah - but something about the restart did not cause a new image to be used14:00
mordredyah14:00
clarkbwekeep the latest and the one we are running which matches mordreds list14:00
mordredwant me to try restarting zuul-web again?14:01
clarkbmordred: ya maybe be sure to use docker compose stop then docker compose up -d to dpible chwck the process ansible uses is working?14:02
mordredclarkb: k. looks yesterday pull and restart was run14:02
mordredso maybe restart doesn't use a new image14:03
mordredactually - why don't I try restart again real quick - just to verify the restart behavior14:03
clarkbor will only if the previous pull got an update14:03
mordredyeah. restart does not seem to pick a new image14:03
clarkbwe are always pulling the latest image in the background14:03
clarkbcould be that then14:04
mordred(which is good to know - I guess maybe that's useful behavior sometimes " I want to restart this container but I don't want to use a new image"14:04
mordreddoing down and up now14:04
mordredwe are now running the new image14:04
corvusmordred, clarkb: aha, thanks for the education :)14:07
mordredcorvus: we're learning things! :)14:07
corvuswe should also look into tobiash's observation about the js promote job being borked (even though we're not using it)14:08
mordredI guess now that I think about it it makes sense. restart is "restart this existing container" and "down / up" is "delete this container and make a new one from the same spec"14:08
mordredcorvus: ++14:08
openstackgerritLida Liu proposed zuul/zuul master: WIP: Add commit id to Change for mqtt reporter  https://review.opendev.org/72247814:10
avassmordred: no restart doesn't pick up any new config14:16
avassmordred: actually, it doesn't recreate the container14:16
avassmordred: and 'up' doesn't do anything if nothing needs doing :)14:17
*** rpittau is now known as rpittau|brb14:18
avassmordred: I believe 'up' only recreates the container if there's a new image available or the docker-compose file has a relevant update14:20
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: upload-artifactory: fix variable misspelling  https://review.opendev.org/73153514:26
*** rpittau|brb is now known as rpittau14:29
avass(just to confirm that's how it works)14:31
mordredavass: yah. I agree with you. oh - so we really couyld have just done docker-compose up -d in this case without the explicit down14:34
avassmordred: yeah14:34
avassmordred: and you can bring up specific services while you can't bring down specific services14:35
avassmordred: with 'down' that is, of course you can still kill them14:35
avassmordred: running 'up' twice shouldn't normally do anything the second time14:36
SpamapSHey everyone, congrats on turning the corner toward Zuul 4!14:53
fungithanks SpamapS!14:53
corvusZuul 4 all14:54
avasscorvus: can you take a quick look at this by the way: https://review.opendev.org/#/c/731526/114:56
avasscorvus: I found that the documentation wasn't matching what zk-ca.sh was doing when getting ready for zuul 4 :)14:56
corvusavass: oh good catch, thanks!14:56
corvustristanC: ^14:57
avasscorvus: the other part was this: https://review.opendev.org/731484 but I'm not sure if it was a specific version of openssl or why it didn't like the relative path14:59
*** jcapitao is now known as jcapitao_afk15:14
Open10K8Smordred: Regarding to nodepool k8s-namespace,15:23
Open10K8Smordred:15:24
Open10K8Sgood question15:24
Open10K8Smordred: how about this?15:24
Open10K8Swhen the namespace variable is not specified, then eliminate the -n clause15:24
Open10K8Sso we can allow the k8s-namespace also15:24
Open10K8Sin the kubeconfig context15:25
*** jamesmcarthur has joined #zuul15:25
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: upload-artifactory: documentation fix  https://review.opendev.org/73117015:28
avassAJaeger: ^ :)15:29
corvusOpen10K8S, mordred: i don't think the collect-kubernetes-logs role was originally intended for use with nodepool namespaces (instead, i think it was designed for a job-deployed k8s), but i think perhaps it could be (if used in a trusted playbook), so i think that approach may be worth taking15:30
*** jamesmcarthur has quit IRC15:30
clarkbcorvus: ya it was originally written to help debug job deployed k8s services15:31
zbrclarkb: corvus: avass guidance on https://review.opendev.org/#/c/702679/13 ?15:33
avasszbr: ++, but depending on how verbose that is we might want to lower that later15:34
zbravass: yep, i love that approach. verbose for a while until we get used to it.15:35
zbronce we gain more trust, we can lower it.15:35
corvusi don't understand why that should be in the job15:36
corvusif you want to debug it, just run it locally15:36
corvusavass: i -2d it to stop the merge (it seemed like tristanC and zbr wanted more feedback)15:38
corvusmy vote is really a -115:38
zbrno worry, i can remove the -v*15:38
corvuszbr: you could consider an env variable to make it easier to add the -vvv locally15:39
corvusi changed my -2 to a C-1,W-1 to make it easier to push through a new patchset without the -vvv15:40
zbrokey15:40
*** harrymichal has quit IRC15:40
*** harrymichal has joined #zuul15:40
openstackgerritLida Liu proposed zuul/zuul master: WIP: Add commit id to Change for mqtt reporter  https://review.opendev.org/72247815:44
*** harrymichal has quit IRC15:47
openstackgerritSorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Bump ansible-lint to 4.3.0  https://review.opendev.org/70267915:50
openstackgerritMerged zuul/zuul-jobs master: upload-artifactory: documentation fix  https://review.opendev.org/73117015:50
*** jcapitao_afk is now known as jcapitao15:53
openstackgerritMerged zuul/zuul master: zookeeper-howto: update to match zk-ca.sh  https://review.opendev.org/73152615:57
fungicorvus: not urgent, but when you get a moment i've suggested another alternative on 73124615:57
*** jamesmcarthur has joined #zuul16:01
AJaegercorvus: when you have some quiet time, could you review the configure-os-mirror change with fresh eyes, please? https://review.opendev.org/#/c/677578/ avass and myself got it passing all tests - and now it needs a good review and answering the comment that avass left based on the discussion from last night.16:02
*** ysandeep is now known as ysandeep|afk16:02
corvusfungi: i think you nailed it16:03
corvusAJaeger: will do16:04
fungicool, will push that up then. thanks!16:04
*** bhagyashris|ruck is now known as bhagyashris16:05
*** hashar has quit IRC16:05
openstackgerritJeremy Stanley proposed zuul/zuul master: Docs: CPD between non-shared-queue changes  https://review.opendev.org/73124616:06
*** rpittau is now known as rpittau|afk16:08
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add the namespace in wait-for-pods role  https://review.opendev.org/73129216:13
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add namespace in the collect-k8s-logs role  https://review.opendev.org/73131916:13
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add namespace in the collect-k8s-logs role  https://review.opendev.org/73131916:14
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add addons args in ensure-kubernetes role  https://review.opendev.org/73152316:16
*** dpawlik has quit IRC16:19
*** nils has quit IRC16:25
*** harrymichal has joined #zuul16:40
AJaegercorvus: is 702679 now good?16:40
corvuslgtm16:41
*** jcapitao has quit IRC16:43
*** asaleh_ has quit IRC16:44
*** guillaumec has joined #zuul16:47
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add the namespace in wait-for-pods role  https://review.opendev.org/73129216:52
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add namespace in the collect-k8s-logs role  https://review.opendev.org/73131916:52
*** jpena is now known as jpena|off16:53
*** harrymichal has quit IRC16:53
*** harrymichal has joined #zuul16:54
openstackgerritMerged zuul/zuul-jobs master: Bump ansible-lint to 4.3.0  https://review.opendev.org/70267916:55
*** ysandeep|afk is now known as ysandeep16:57
*** rlandy|mtg is now known as rlandy17:01
*** jamesmcarthur has quit IRC17:03
openstackgerritMerged zuul/zuul master: Use Absolute path in zk-ca.sh for openssl config  https://review.opendev.org/73148417:06
openstackgerritMerged zuul/zuul master: requirements: add explicit reference to dateutil  https://review.opendev.org/73065417:08
*** ysandeep is now known as ysandeep|away17:13
AJaegerzuul-jobs-maint, please review these small changes https://review.opendev.org/730720 https://review.opendev.org/731535 https://review.opendev.org/73073317:15
avasscorvus: sorry, no problem17:15
*** olaph has quit IRC17:20
openstackgerritSorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Enable linting of test-playbooks  https://review.opendev.org/73147117:27
corvusAJaeger, zbr: can we be more conservative with ansible-lint?17:41
corvusi feel like we're spending too much time on linting rules -- especially on ansible-lint which we very nearly stopped using completely because of how many useless rules it has17:42
corvusfor example, the "zj_" loop var is a great idea and will help us avoid errors.  the whole "don't use the git command" thing is just a red herring for us because it's so often wrong.17:43
zbrcorvus: i hope not, most of them are useful, and having decent coding standards would keep code maintainable.17:43
zbrit is very easy to add stuff to whitelist17:44
zbrsame as with flake817:44
avasszbr, corvus: I think the problem is that in the context in zuul a lot of the normal ansible rules don't really apply17:44
corvusyes, so if we're going to use linters at all (i think we'd be better off without them, but i'm compromising here :)  let's use them where they provide actual value: catching bugs.17:46
zbri am not so sure about the different context, i do not see a reason to have two quality standards17:46
fungimy understanding was that ansible-lint was intended to be the rule enforcer for ansible roles published to galaxy, while it seems unlikely most of our zuul-jobs roles would ever benefit from that17:46
corvusfungi: indeed, and the last time we had this conversation, that's why we decided its use in zuul-jobs should be minimized17:46
zbras I said, skiplist works fine for stuff we do not agree woth.17:46
corvusi have to admit, i'm frustrated that we're having it again without the previous context :(17:46
corvusit only reinforces my belief that linters are 10% useful and 90% a waste of valuable developer time arguing about whitespace17:47
openstackgerritMerged zuul/zuul-jobs master: Rename test install role to ensure-  https://review.opendev.org/73072017:47
zbrit would be more useful to see your feedback on ansible-lint issue tracker17:48
corvuszbr: i think adding most of those 3xx rules in 731471 to the whitelist would probably be a good start17:48
*** cloudnull has quit IRC17:48
zbrone of the benefit is that linter avoids wasting time debating whitespace during code reviews, it does save time, not take time.17:49
corvuszbr: on the contrary, that does not seem to be the result17:49
corvuszbr: we have a different approach in the zuul project: we just don't nitpick whitespace17:49
corvuszbr: we don't let it bother us17:49
zbrso we do have low quality standards?17:49
*** cloudnull has joined #zuul17:49
corvuszbr: whitespace is not quality17:49
corvuszbr: we promote a culture of not nitpicking17:50
avassAJaeger: ^ ;)17:50
corvusit results in people being happier17:50
zbri bet to defer that the whitespace on jinja is important, the same kind of importance as whitespace on python code17:50
corvusi think you know what i mean, let's not argue about that17:51
corvusthe point i'm trying to convey is that linting rules that catch bugs are welcome, and linting rules that are frequently erroneous or are mere stylistic preferences should be disabled17:52
corvusand that, when it does come to style, we should avoid nitpicking (yes -- sometimes style is important; but we should be careful about knowing when it is and isn't)17:53
zbrcorvus: huh, glad to hear that. Yes, a buggy one should clearly be whitelisted.17:53
zbrone of the reason some of you hated the linter is that had few buggy rules, most of them where fixed in newer versions.17:54
zbrso less false-positives17:54
corvuszbr: so for a start, must of those 3xx rules we hit frequently enough they should probably be disabled17:54
zbrcorvus: here is the fun bit: we did lint only rules/ which was bad, as test-playbooks/ is ansible too17:55
zbrthe rules/ is ok and does not need changes.17:55
zbrroles/17:55
corvuswell, we deemed it more important17:55
openstackgerritMerged zuul/zuul master: Docs: CPD between non-shared-queue changes  https://review.opendev.org/73124617:55
zbrcorvus: true, but longer term we should normalize, which is what i did.17:56
*** jamesmcarthur has joined #zuul17:56
zbrroles/ is quite in good shape from this point of view17:56
zbrcorvus: if you want I am willing to consider whitelisting E602 which is probably one of the most controversial one17:57
corvuslooking at that patch, it is approximately 50% #noqa lines -- that's a red flag17:57
avassI think we should keep the pipefail rule though, that seems like it could catch errors17:57
zbri will split it by nature of fix, it would be much easier to review.17:57
zbravass++ true17:58
corvusevery instance of "#noqa 306" in that patch is correct as written17:59
zbrfor example 303/305: whitelisting them would be a regression in quality because at this moment roles/ does respect them, is only test-playbooks/ that does not.17:59
corvusso 306 caught zero bugs there18:00
zbrmaybe in our case, but what about newly proposed code? code where OP would have missed to use a module that is idempotent and used shell instead?18:01
zbreven ansible itself does raise warnings on stuff like this18:01
zbrthat is why we have: args: warn: false18:01
zbrwhich is supposed to be respected by linter too (i think)18:02
tristanCzbr: why not using args: warn: false   then?18:02
avasshmm, I guess we only pipes with grep and anything else is just done in ansible18:02
zbrtristanC: that is a very good point.18:02
zbri will look at  warn: false bit, if it does not work I will propose a patch for the linter.18:03
corvusfrankly, i'm not worried about having two quality standards for "real" and "test" code18:05
corvusall the ansible in test-playbooks is, by definition, tested.  so the linter is unlikely to catch additional bugs.18:05
tristanCcorvus: agreed18:06
fungiis ansible-lint catching actual bugs yamllint wouldn't?18:07
*** jamesmcarthur_ has joined #zuul18:07
corvusanyway, i've spent too long on linters for today; i'm supposed to be reviewing circular dependencies; i'm going to need to ignore irc a bit to get that done18:07
fungii guess it checks that stuff you reference actually exists18:07
*** fbo is now known as fbo|off18:07
openstackgerritLida Liu proposed zuul/zuul master: WIP: Add commit id to Change for mqtt reporter  https://review.opendev.org/72247818:10
*** jamesmcarthur has quit IRC18:10
openstackgerritSorin Sbarnea (zbr) proposed zuul/zuul-jobs master: test-playbooks: improved syntax  https://review.opendev.org/73159118:12
zbrfor example is able to detect a missing module (a typo in module name or argument may even make the reviewer miss it)18:13
avassfungi: https://zuul-ci.org/docs/zuul-jobs/policy.html#ansible-linting-rules :)18:13
avassfungi: I still have to finish the no-same-owner rule though18:13
*** yolanda has quit IRC18:22
*** aspiers has quit IRC18:31
*** kmalloc has joined #zuul18:41
*** aspiers has joined #zuul18:43
Open10K8Smordred: corvus: please check the PSs18:51
Open10K8SI tried to follow your advices :)18:51
Open10K8Shttps://review.opendev.org/#/c/731292/18:51
Open10K8Shttps://review.opendev.org/#/c/731319/618:51
Open10K8Shttps://review.opendev.org/#/c/731523/18:51
*** aspiers has quit IRC18:57
avassOpen10K8S: commented on 73129219:01
Open10K8Savass: just received19:01
avassOpen10K8S: but I'd wait to see what the others think19:03
*** jamesmcarthur_ has quit IRC19:04
openstackgerritMatthieu Huin proposed zuul/zuul master: web UI: add Autoholds Requests page  https://review.opendev.org/72930719:09
avassOpen10K8S: same with 731523, it's not a -1 since it will probably not cause a problem, but I'd rather avoid setting a fact if it's not too much work :)19:11
mhuHey there, I think the REST API How-to page is good to go: https://review.opendev.org/#/c/727785/19:12
Open10K8Savass: ok, thank you avas19:12
Open10K8SI agree your idea, but we can use facts in case which we need to use several times the same fact19:12
Open10K8Sso for the addons case, that would be good19:13
Open10K8Swait-for-pod case, i think it is convenient using the fact19:13
Open10K8Show about your idea?19:13
mhuIf anybody has also time for reviews on https://review.opendev.org/#/q/topic:fix_rest_client I'd be grateful19:15
*** harrymichal has quit IRC19:22
avassOpen10K8S: I do see the convenience of it and I like tristanC's way of using defaults for that: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/haskell-stack-test/defaults/main.yaml#L519:22
avassOpen10K8S: that way the variable isn't set outside the role19:22
Open10K8Savass: looks good19:23
*** jamesmcarthur has joined #zuul19:23
Open10K8Sok then lets do like that19:23
clarkbinstalling zuul's deps in tox for unittesting isn't very quick. I think the reason is the node installation. Mostly an observation I had that we might be able to speed those jobs up a bit if we can make that faster19:23
*** harrymichal has joined #zuul19:24
openstackgerritSorin Sbarnea (zbr) proposed zuul/zuul-jobs master: Enable linting of test-playbooks  https://review.opendev.org/73147119:26
avassOpen10K8S: have a look at the guidelines as well if you haven't already. We also want to namespace the variables (where it makes sense) to avoid something else accidentally overriding it: https://zuul-ci.org/docs/zuul-jobs/policy.html#role-variable-naming-policy19:27
avassbut I'm not sure how hard we're enforcing that :)19:28
openstackgerritSorin Sbarnea (zbr) proposed zuul/zuul-jobs master: test-playbooks: avoid warnings with shell/command  https://review.opendev.org/73160519:47
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Update docs for os specific task guidelines  https://review.opendev.org/73160619:49
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Update guideline docs for os specific tasks  https://review.opendev.org/73160619:50
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Update guideline docs for os specific tasks  https://review.opendev.org/73160619:52
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Update guideline docs for os specific tasks  https://review.opendev.org/73160619:55
avasszuul-jobs-maint: I noticed the guidelines are different from what we're actually doing with multi os support. Can we update that? ^20:04
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Update guideline docs for os specific tasks  https://review.opendev.org/73160620:06
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Update guideline docs for os specific tasks  https://review.opendev.org/73160620:08
avasssorry I missed removing some words. should be ready now :)20:08
*** jamesmcarthur has quit IRC21:06
*** jamesmcarthur has joined #zuul21:07
*** jamesmcarthur has quit IRC21:12
*** jamesmcarthur has joined #zuul21:15
*** sanjayu_ has quit IRC21:31
openstackgerritGuillaume Chauvel proposed zuul/zuul master: Import user tutorials from Software Factory project blog  https://review.opendev.org/72819321:36
openstackgerritGuillaume Chauvel proposed zuul/zuul master: Add tutorial tests  https://review.opendev.org/72819421:36
*** threestrands has joined #zuul21:45
corvustobiash: all right, i left some questions on https://review.opendev.org/68535421:56
tobiashcorvus: thanks! I'll check tomorrow22:00
corvusAJaeger, avass: let's ask fungi to do the next review pass on https://review.opendev.org/677578, then tristanC.22:00
*** jamesmcarthur has quit IRC22:04
*** jamesmcarthur has joined #zuul22:05
fungii bet i even know which change that is ;)22:05
fungiyep, exactly the one i was expecting22:05
*** jamesmcarthur has quit IRC22:11
openstackgerritGuillaume Chauvel proposed zuul/zuul master: Import user tutorials from Software Factory project blog  https://review.opendev.org/72819322:21
openstackgerritGuillaume Chauvel proposed zuul/zuul master: Add tutorial tests  https://review.opendev.org/72819422:21
*** jamesmcarthur has joined #zuul22:21
*** rlandy has quit IRC22:38
*** armstrongs has joined #zuul22:42
armstrongsHey what happened in the end with the bitbucket driver did it ever get merged to master like the gitlab driver?22:43
*** armstrongs has quit IRC22:55
*** sanjayu_ has joined #zuul22:59
mnaserso i just ran into an issue with pip running in a constrained resource environment (in my case, an instance with 1gb of memory only).  in fedora-32, /tmp is tmpfs so you only end up with 487M available in /tmp23:02
mnaseryou end up running into: "Could not install packages due to an EnvironmentError: [Errno 28] No space left on device" during tox initial run with my assumption being that /tmp is running out of disk space23:02
mnaserwould it make sense for us to define TMPDIR in our jobs to be {{ ansible_user }}/tmp or something23:02
*** jamesmcarthur has quit IRC23:03
mnaserbecause / has 50G and /tmp has only 500M23:03
clarkbmnaser: might be better to make /tmp not a tmpfs23:03
mnaserclarkb: right, is there a way to do that in dib then? because dib is building an image with /tmp as tmpfs23:03
*** jamesmcarthur has joined #zuul23:03
clarkbmnaser: yes, you can either edit fstab (and just remove the /tmp entry and it will end up in the same fs as /) or use the partitioning support to put it in its own fs23:04
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add the namespace in wait-for-pods role  https://review.opendev.org/73129223:04
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add namespace in the collect-k8s-logs role  https://review.opendev.org/73131923:04
clarkbianw: ^ you may have thoughts on mnaser's fedora /tmp problem23:04
mnaserclarkb: cool well unforunately it seems like tmpfs is created by not fstab cause fstab only contains the root :(23:04
mnaser`LABEL=cloudimg-rootfs / ext4 defaults 0 1` is all i have in fstab23:05
mnaserso i think maybe something systemd-ish23:05
mnaser`  tmp.mount                                                                           loaded active mounted   Temporary Directory (/tmp)`23:05
clarkbah disable that unit then I guess23:05
clarkbthats really weird to put in a unit23:05
mnaserthis is fedora-32 fwiw23:05
clarkbbut I probably don't understand why systemd needs to be more awesome and manage things like that23:06
mnaserso i dont think infra has ran into it yet, but also infra uses larger vms23:06
clarkbya fedora tends to be a huge pain to deal with updating23:06
mnaseri guess the tricky / annoying thing is the fact that its nice to have if you have a lot of memory23:06
clarkbianw somehow managed to get through it though23:06
mnaserbut ugly if you dont23:06
mnaser(tmpfs that is)23:06
clarkbmnaser: ya an improvement to fedora might be to make that switch based on available memory23:07
guilhermespi was wondering if –no-tmpfs would do the trick23:07
guilhermespbut that's a parameter we use with disk-image-create23:07
*** jamesmcarthur has quit IRC23:07
clarkbguilhermesp: ya dib itself will build in a tmpfs by default if you have enough memory23:07
clarkbthis is to reduce disk writes23:07
mnaserguilhermesp: i think that --no-tmpfs is just for dib not to use tmpfs when it builds the image (cause i think that's a speed up)23:07
*** jamesmcarthur has joined #zuul23:07
clarkbyup23:07
mnaseri dont think this is something i can help bake into zuul/zuul-jobs eh? :\23:08
guilhermespah yeah, similar to DIB_NO_TEMPFS, nvm23:08
clarkbmnaser: you probably can as long as the remounting works fine23:08
mnaserclarkb: i think maybe the only best thing to do is create a disable-tmpfs role that i can include in my base jobs which disables tmpfs if tmpfs size <= X23:08
clarkbmnaser: oh except things might need the contents of /tmp after boot23:08
mnaserah crap, true23:08
mnaseryes, indeed23:09
mnaseri see a bunch of systemd stuff on this cleanly booted instance23:09
mnaserfor systemd-logind and dbus-broker so they're certainly _not_ thing id go about forgetting23:09
ianwdib has a ram probe and i think basically doesn't use tmpfs23:10
mnaserianw: right but we're talking about the final-generated image23:11
ianwas for fedora-32 mounting it tmp ... haven't got to 32 yet sorry :/23:11
mnaserit sems like its been mounting as tmpfs since fedora 18 lol23:12
mnaseri guess its only an issue because its a 1g memory instance and infra's instances are big enough so that tmpfs is big enough to not cause issues23:12
ianwhrrmm, ok, yeah this is the first i've heard of it23:12
guilhermespyeah that's basically the thing i guess.... like, fedora-minimal requires >1Gb >.<23:13
clarkbseems like if disabling the unit in the image build fixes it that is an easy enough change23:13
mnaserclarkb: i guess the thing is i was hoping for this to be a runtime option so a 16gb memory instance doesnt suffer because the 1g instance couldnt use tmpfs23:14
mnaserbut i cant imagine a clean solution23:14
clarkbI mean how much are they going to suffer though23:14
clarkbits better if performance sensitive things opt into tmpfs instead I think23:14
clarkbwe do that for zookeeper's data writes in zuul testing for example23:14
ianwhttps://www.freedesktop.org/wiki/Software/systemd/APIFileSystems/ does talk about it at the bottom23:15
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add addons args in ensure-kubernetes role  https://review.opendev.org/73152323:16
*** armstrongs has joined #zuul23:19
*** tosky has quit IRC23:20
mnaserhmm23:21
mnaseri dont think you can set environment variables for zuul jobs, or can you23:21
clarkbmnaser: you can in the playbooks but not top level I odn't think23:22
openstackgerritOleksandr Kozachenko proposed zuul/zuul-jobs master: Add addons args in ensure-kubernetes role  https://review.opendev.org/73152323:24
mnaserhrm, fair enough23:24
*** armstrongs has quit IRC23:28
mnaserclarkb, ianw, guilhermesp: thanks for bouncing ideas around23:29
mnaseri think we'll suggest either using a bigger instance in terms of memory or overriing TMPDIR in the plays23:29
*** rfolco|rover has quit IRC23:31
fungimnaser: clarkb: late to the party, but you could also create a swapfile, and then tempfs should page out to swap when needed23:35
fungii merely skimmed, so someone might have already said that23:35
fungihowever, if you're likely to be writing to disk anyway, then it's silly to use a tempfs for that at all23:36
fungii agree23:36
mnaserfungi: yeah, i guess that's merely a workaround.  i didn't know tmpfs will write to swapfile if it is full23:36
fungiand super-memory-constrained systems shouldn't be using one for anything more than small shared memory spaces23:37
mnaserfungi: which makes the idea of moving configure-swap role to zuul-jobs and adding an option of running it if system_memory < X interesting23:37
mnaserfungi: in this case its pip using TMPDIR and filling it up23:37
clarkbmnaser: probably doing compiles in there23:38
mnaseralso that too23:38
clarkbmnaser: another workaround may be to ensure you have wheels available23:38
fungiright, i meant low-memory systems shouldn't be using tmpfs for things like /tmp23:38
fungibut that they're likely to still want them for stuff like pre-boot /run space and /dev/shm23:39
mnaserfungi: right but i dont think i have an easy way of identifying that in run-time, as i dont want to end up with lowmem and normalmem images23:39
clarkbmnaser: I think you force everything to use disk for temp23:39
clarkbthen if something knows it is advantageous to operate on tmpfs it can opt into that23:39
clarkbthat was my zookeeper example earlier23:39
mnaserright23:39
mnaserif opendev does that, i'll do it =P23:39
fungiyeah, honestly if you try to do a kernel build in /tmp, even 8gb memory isn't going to be enough23:40
mnaseryeah building wheels is hard because in this case it's hard for me to know all the possible different wheels used by my consumers23:40
*** rfolco|rover has joined #zuul23:41
mnaserwould configuring swap increase tmpfs size23:41
mnaserconditionally creating swap seems like it might end up doing the same result, but yeah, it's annoying beacuse i'm trying to find the best possible outcome for all different scenarios, but there's none23:41
clarkbI wouldn't object to a change in our fedora builds to disable that unit23:43
*** guillaumec has quit IRC23:44
*** rfolco|rover has quit IRC23:45
fungiadding swap doesn't dynamically increase tmpfs size, that's usually something you set explicitly at mount time, but i believe you can increase it trivially with a mount -oremount --size=xxxx or whatever and not have to unmount it first23:46
*** rfolco|rover has joined #zuul23:46
*** rfolco|rover has quit IRC23:50
*** rfolco|rover has joined #zuul23:50
mnasersounds too messy23:51
*** rfolco|rover has quit IRC23:55

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!