Friday, 2021-04-23

*** hamalq has quit IRC00:13
corvusclarkb, ianw: left some comments on 787475; sorry i have to run to dinner now.01:17
*** ajitha has joined #zuul02:09
*** evrardjp has quit IRC02:33
*** evrardjp has joined #zuul02:33
*** rlandy|rover|bbl has quit IRC03:42
*** ikhan has joined #zuul03:44
*** ikhan has quit IRC03:49
*** bhavikdbavishi has joined #zuul04:08
*** bhavikdbavishi has quit IRC04:12
*** ykarel|away has joined #zuul04:18
*** ajitha has quit IRC04:18
*** ikhan has joined #zuul04:31
*** vishalmanchanda has joined #zuul04:36
*** jfoufas1 has joined #zuul04:55
*** ykarel_ has joined #zuul04:55
*** ykarel|away has quit IRC04:58
ianwcorvus: thanks, i'm going to have to look at that with fresher eyes :)  but will next week05:13
*** ykarel__ has joined #zuul05:23
*** ykarel_ has quit IRC05:23
*** ykarel_ has joined #zuul05:41
*** ykarel_ is now known as ykarel05:43
*** ykarel__ has quit IRC05:43
*** jcapitao has joined #zuul06:22
*** LiangLu has joined #zuul06:48
*** saneax has joined #zuul07:03
*** ykarel has quit IRC07:07
*** ykarel has joined #zuul07:10
*** rpittau|afk is now known as rpittau07:12
*** ykarel_ has joined #zuul07:12
*** nils has joined #zuul07:14
*** ykarel has quit IRC07:15
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Roles to create, cleanup and promote snapshots in ec2  https://review.opendev.org/c/zuul/zuul-jobs/+/78767707:41
*** ykarel_ has quit IRC07:45
*** shanemcd has quit IRC07:50
*** shanemcd has joined #zuul07:50
*** tosky has joined #zuul07:53
*** jpena|off is now known as jpena07:54
*** ykarel has joined #zuul07:55
*** harrymichal has joined #zuul08:04
*** ykarel has quit IRC08:04
*** harrymichal has quit IRC08:05
*** harrymichal has joined #zuul08:05
*** ykarel has joined #zuul08:06
openstackgerritFelix Edel proposed zuul/zuul master: Execute builds via ZooKeeper  https://review.opendev.org/c/zuul/zuul/+/77090208:22
openstackgerritFelix Edel proposed zuul/zuul master: Fix test_gerrit.TestPolling.test_config_update  https://review.opendev.org/c/zuul/zuul/+/77302308:22
openstackgerritFelix Edel proposed zuul/zuul master: Switch to ZooKeeper backed merge result events  https://review.opendev.org/c/zuul/zuul/+/78419508:22
openstackgerritFelix Edel proposed zuul/zuul master: Improve component registry  https://review.opendev.org/c/zuul/zuul/+/78768408:22
openstackgerritFelix Edel proposed zuul/zuul master: Calculate statsd metrics via ComponentRegistry  https://review.opendev.org/c/zuul/zuul/+/78768508:22
openstackgerritFelix Edel proposed zuul/zuul master: Execute merge jobs via ZooKeeper  https://review.opendev.org/c/zuul/zuul/+/78768608:22
LiangLuHi there, this is Liang from tacker team, we need to create 2-NIC nodes during Zuul FT test job, we tried to set label of node, but seems only rax provider will create 2-NIC controller node, do you have any experience of that?08:31
*** saneax has quit IRC08:33
*** harrymichal has quit IRC08:34
lyrHey. I've this error when trying to add an autohold through zuul-client https://paste.garrigue.re/?99009e4216471254#DujC4FvXJisGZRAkndsFkvVb6wdXBdDA5dN22koxyixh08:59
lyrAn ugly jwt "not enough segment"09:00
lyrautohold-list is running fine09:05
lyrrunning the equivalent command with zuul cli directly on the server works09:06
mhulyr: how did you get your token?09:06
lyrthrough the UI09:07
lyrurl https://softwarefactory/sf/user_settings.html09:07
mhulyr: that's not the right token, sorry. I guess we should clarify this in SF's documentation09:08
mhuat the moment you need to generate the token with the zuul admin CLI on the server09:08
lyrI guess "zuul create-auth-token" is the way to go ?09:08
mhucheck "zuul create-auth-token --help"09:08
mhuyes exactly09:08
lyris there a way to get a token form the UI ?09:09
lyr(for my developers' internal doc)09:09
mhunot at the moment, but there's a big patch chain to add support for the admin commands in the GUI09:09
mhuthis also adds a user page where you can get your token when authenticated09:09
mhu this way you can also use the CLI09:10
lyrok it works now, thanks mhu. Do you have an idea which SF release this user page feature will be released in ? 3.7 ?09:20
mhuit needs to land upstream first09:20
mhuI'm back from paternity leave so I can drive this effort again09:20
mhuonce we get the feature in zuul, SF will also need to switch to a more modern SSO for jwt support - we plan to replace the current SSO with keycloak09:21
mhuso there's a lot to do ... but it's on the roadmap09:22
*** ikhan has quit IRC09:37
*** ikhan has joined #zuul09:37
*** ykarel_ has joined #zuul09:39
*** ykarel has quit IRC09:41
*** ykarel_ is now known as ykarel09:42
lyrI saw keycloack on the SF 4 roadmap09:57
lyrThanks for the intel09:57
mhuyw09:57
*** ikhan has quit IRC10:12
*** ikhan has joined #zuul10:12
*** ykarel_ has joined #zuul10:15
*** ikhan has quit IRC10:17
*** ykarel has quit IRC10:18
lyrmhu the create-auth-token is temporary ?10:19
lyrGot a "Unauthorized - your token might be invalid or expired."10:19
mhulyr: yes, check the --help, there's an option to set the expiry10:19
*** jcapitao has quit IRC10:24
*** ikhan has joined #zuul10:27
*** ykarel__ has joined #zuul10:31
*** ikhan has quit IRC10:31
*** ykarel_ has quit IRC10:33
*** ykarel_ has joined #zuul10:36
*** ykarel_ is now known as ykarel10:37
*** ykarel__ has quit IRC10:39
*** ikhan has joined #zuul10:43
*** ikhan has quit IRC11:27
*** jpena is now known as jpena|lunch11:30
*** rlandy has joined #zuul11:42
*** rlandy is now known as rlandy|rover11:43
*** jpena|lunch is now known as jpena12:33
*** mgoddard has joined #zuul12:44
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: Roles to create, cleanup and promote snapshots in ec2  https://review.opendev.org/c/zuul/zuul-jobs/+/78767712:48
*** vishalmanchanda has quit IRC13:26
*** ykarel_ has joined #zuul13:36
*** ykarel has quit IRC13:39
*** ssmashnuk has joined #zuul13:39
*** y2kenny has joined #zuul13:40
ssmashnukGood morning everyone.  Zuul newby here looking to use some static WIN2019 nodes, but I'm getting the following error:  "msg": "winrm or requests is not installed: No module named 'winrm'".  Can someone point me in the right direction?13:44
avassssmashnuk: you probably need to install pywinrm on the executors :)13:48
corvusor i guess specifically that needs to be in the virtualenv the executors use to run ansible?13:49
avassssmashnuk: https://docs.ansible.com/ansible/latest/user_guide/windows_winrm.html#what-is-winrm13:49
avasscorvus: yeah13:49
avassI think there's a environment variable to handle that13:49
corvuswe should probably install that by default and include it in our images?13:50
avassssmashnuk: ANSIBLE_EXTRA_PACKAGES https://zuul-ci.org/docs/zuul/howtos/installation.html?highlight=extra_packages#ansible13:50
avasscorvus: probably a good idea, that ^ variables wasn't well documented what I could see either13:51
ssmashnukGreat, thanks corvus, I'll give it a try.13:52
ssmashnukAs a note, I do have winrm installed on my executors and using the ansible_python I can import winrm and connect.13:53
avassIs pywinrm installed on the system or in the ansible virtualenvs?13:54
corvus(zuul executors use zuul-manage-ansible to make a virtualenv for each of the [currently 2] versions of ansible they support)13:55
ssmashnukSystem, which is probably the issue.13:55
corvuscool, so the link avass pointed you to should take care of it13:55
avassssmashnuk: also you could try using openssh as well, winrm can't really handle copying files very well so the preferred way to synchronize repositories from the executor is to use ssh in some way anyway13:55
avassbut you'd need to install ssh on the windows nodes in that case13:56
openstackgerritJames E. Blair proposed zuul/zuul master: Add winrm to default executor venv packages  https://review.opendev.org/c/zuul/zuul/+/78774313:57
avasscorvus: I think it's pywinrm13:58
avassoh looks like it's in pypi with both names13:59
corvusdifferent maintainers though13:59
avassyeah13:59
corvusso it's either a helpful attempt to alias the name, or a trojan14:00
corvushard to say which14:00
openstackgerritJames E. Blair proposed zuul/zuul master: Add pywinrm to default executor venv packages  https://review.opendev.org/c/zuul/zuul/+/78774314:00
openstackgerritAlbin Vass proposed zuul/zuul master: Add pywinrm to default executor venv packages  https://review.opendev.org/c/zuul/zuul/+/78775114:13
avassoops bad copy/paste :)14:13
openstackgerritAlbin Vass proposed zuul/zuul master: Add boto3 to default executor venv packages  https://review.opendev.org/c/zuul/zuul/+/78775114:13
avass(to make it easier to use upload-logs-s3 and snapshot instances)14:14
corvus++14:14
*** rpittau is now known as rpittau|afk14:15
*** ykarel_ is now known as ykarel14:26
lyrI hitted a lot of NODE_FAILURE on a batch of change across a lot of repoisotires, since my openstack was full (not enough quota). I'm supprised the job didn't waited. Is there something like a waiting timeout I can set somewhere ?14:35
clarkblyr: you can set max-servers on the pool, but also nodepool will respect instance, cpu, and memory quotas if you set them in the cloud14:36
lyrI did set max-servers. That's why I got a "not enough quota", it's fine. But I expected the jobs to wait for a node14:37
clarkblyr: if you set max-servers then nodepool will hit that limit and wait before trying to boot more instances14:38
clarkbhaving NODE_FAILURES implies that you were not yet to max-servers and started to fail14:38
clarkband then those failures got reported back as NODE_FAILURE14:38
lyrJust to be clear, by NODE_FAILURE, I mean https://i.imgur.com/OKTdUH0.png14:39
corvuslyr: do you have logs from the launcher indicating the cause?14:40
clarkblyr: yes, roughly the way nodepool works is: check if I have room according to max-servers or quota, if there is room then boot, if boot fails three times in a row report NODE_FAILURE, if there is no room, stop processing node requests and wait until there is room14:40
clarkbmy read on that is that the boots failed while you were below max-servers and quota limits (which can happen)14:41
lyrsmell like our old openstack being crancky https://paste.garrigue.re/?bb1a96be7d200c6d#FZQFg4u3me1dauiQZf6N22WQnmYxWowD7ZSdbDWZTPDg14:41
lyrHmm sorry for bothering, it seems there have been a glitch, now it seems to be working14:44
*** ykarel is now known as ykarel|away14:50
*** ykarel|away has quit IRC14:59
*** hashar has joined #zuul15:03
openstackgerritMatthieu Huin proposed zuul/zuul-client master: [WIP] add tenant as a config option  https://review.opendev.org/c/zuul/zuul-client/+/78777015:03
*** hashar has quit IRC15:10
*** hashar has joined #zuul15:10
*** hashar has quit IRC15:10
*** hashar_ has joined #zuul15:10
openstackgerritAlbin Vass proposed zuul/zuul-jobs master: intercept-job -- self-service SSH access  https://review.opendev.org/c/zuul/zuul-jobs/+/67930615:14
*** Shrews has joined #zuul15:37
openstackgerritMerged zuul/zuul-jobs master: intercept-job -- self-service SSH access  https://review.opendev.org/c/zuul/zuul-jobs/+/67930615:42
*** hashar_ has quit IRC15:53
*** nils has quit IRC16:13
*** hamalq has joined #zuul16:27
*** hamalq_ has joined #zuul16:33
*** hamalq has quit IRC16:36
*** jpena is now known as jpena|off16:53
openstackgerritJames E. Blair proposed zuul/zuul master: Switch to ZooKeeper backed build result events  https://review.opendev.org/c/zuul/zuul/+/78293916:54
avasscorvus: speaking of NODE_FAILURES, I sometimes get boot timeouts for my digital ocean drivers that is using the statemachine driver. it looks like the provider doesn't retry if the boot times out and most of the boots (like at least 9/10) works so wouldn't it be better to retry if the timeout is exceeded?17:10
avassI mean retry up to 3 tries or something17:10
fungiin the openstack provider driver i see launch attempts retried after boot timeouts (up to 3 times before the launcher declines the request)17:11
avassso maybe that's something that should be done in the digital ocean driver or in the statemachine driver I suppose17:12
avassstatemachine framework*17:12
corvusavass: yeah, due to the way the nodepool launcher is written, it's not really feasible for the state machine driver to handle that, so for azure, i put it inside the state machine itself: https://review.opendev.org/78068217:12
corvusavass: i would like to correct that, but it's a large change to nodepool17:12
avasscorvus: ok then I know to implement that in the driver at least17:13
corvusyeah, that should work; i don't like it because it makes for a complicated state machine, and it's really just handling one thing that could go wrong, whereas if we generalize it, it could transparently handle more kinds of failures.17:14
corvusavass: but i think that's what we have to do for now until we can revisit it.  should be easy to pull that out of the statemachine drivers once we can.17:15
avassyup17:15
avassI think the ec2 driver could be suffering from that as well, but we've put a long boot timeout for now to get around it17:16
y2kennyIn Zuul web UI for build result, each build is listed to show "change" which is <change id>,<patch id> for Gerrit and the change can be use for filtering.  Is there a possibility to configure zuul to display the change's git short hash as well?  (for both display and filtering?)17:25
corvusy2kenny: that is not currently supported (and i don't think that info is currently stored in the db)17:29
y2kennycorvus: has there been similar request in the past?  is the feature worth exploring?17:31
corvusy2kenny: to my knowledge, you're the first; and i don't see why not :)17:34
y2kennycorvus: ok.  I just figure the git hash is potentially more universal in terms of finding things.  I am guessing for GitHub based job the pull request number shows up there instead?17:35
*** jfoufas1 has quit IRC17:36
corvusy2kenny: pr,sha for github17:36
corvusy2kenny: zuul doesn't really have any use for it; it's not a unique identifier for a change, and zuul deals with changes17:37
corvusbut i could see a user wanting to see what builds have run for the change(s) associated with a commit17:38
y2kennycorvus: um... right.  Ok, I will think about it some more.17:38
corvusso carrying it around as duplicate data for humans is fine; i just mention that to indicate why it's not a central piece of info for zuul17:38
y2kennyunderstood.17:38
fungii suppose each source connection driver would need to decide how commits are associated with a change17:39
fungifor a github pull request, would it be any commits included in that pull request?17:40
fungior only the "tip" commit?17:40
corvusfungi: typically the PR tip; that's what's used as the patchset identifier for the pr17:40
avassfungi: I think that's currently part of mqtt message and reports the tip17:41
fungifor a gerrit change, would it be only the patchset(s) which had that commit, or any patchset for the same change which had that commit as a patchset?17:41
corvusyeah, all the drivers have this info; it's just not used for changes17:41
fungitrue, i suppose github pr revisions and gerrit change revisions both get new commit ids... though at least gerrit is able to have the same commit id on (non-sequential) patchsets for a change17:42
corvusin all cases there is a 1->N mapping of change+patchset -> commit.  so it's possible for us to carry around the commit for a change, store it in the db, and do a reverse lookup on that commit to tell you all the jobs that ran for all the changes that pointed to that commit.17:43
corvusa change in zuul is a kind of ref, and a ref always points to a git sha17:44
corvusreversing that gets you multiple changes17:44
corvus(potentially, but rarely in practice)17:44
corvuson top of that, it's worth keeping in mind that the sha of the change's commit is rarely the repo HEAD that zuul tests (because it's usually testing synthetic merge commits)17:45
corvusbut we can still say "this is the commit that we merged with a bunch of other commits on top of the branch which moved ahead since the commit was written" when we ran this job :)17:46
corvuswe can guarantee that commit was somewhere in the dag17:46
fungiright, it's not so much "the commit that zuul tested" but rather "a convenient handle for what triggered some buildset"17:48
corvusexactly17:49
fungi(possible multiple buildsets)17:49
*** LiangLu has quit IRC17:50
fungimight represent buildsets from the same event in multiple pipelines, or rechecks of the same change, or less common collisions of that commit handle to different change revisions17:50
fungithat last is where the change identifier is more specific17:51
*** cloudnull has quit IRC18:11
openstackgerritJames E. Blair proposed zuul/zuul master: Switch to ZooKeeper backed build result events  https://review.opendev.org/c/zuul/zuul/+/78293918:33
*** nils has joined #zuul18:42
*** nils has quit IRC18:52
-openstackstatus- NOTICE: The Gerrit service on review.openstack.org is being restarted to pick up some updates, and should be available again momentarily19:03
*** nils has joined #zuul19:04
*** Eighth_Doctor is now known as Conan_Kudo19:27
*** y2kenny has quit IRC19:28
*** Conan_Kudo is now known as Eighth_Doctor19:29
*** cloudnull has joined #zuul19:47
*** nils has quit IRC20:22
openstackgerritMatthieu Huin proposed zuul/zuul-client master: Add tenant as a config option  https://review.opendev.org/c/zuul/zuul-client/+/78777020:25
mhulyr: ^20:26
clarkbmhu: can I suggest that you reformat with black in a separate change?20:26
clarkb(also if the plan is to use black I would have it enforce that in the linter job)20:27
mhuclarkb, fair enough - I used black because all my cool friends do, I caved in to peer pressure20:29
clarkbI personally wouldn't mind using it either, but not sure about others. THe important thing though is to make that its own change imo20:29
clarkband then be consistent about it if that is the format we want to use20:29
clarkbmixing the changes and then not enforcing it will just cause things to regress over time20:29
pabelangerI like black for enforcement, we are using it for ansible collections20:30
pabelangerbut agree, adding it into linters jobs to gate should also be done20:30
pabelanger(if adopted)20:30
mhupabelanger, do you have a job definition example using black I could look at?20:31
clarkbmhu: you would use the existing linters job definition but update tox.ini to use black instead of pep8/pycodestyle20:32
fungito be honest i'm not a huge fan of adding another style checker which will cause a complete update of the entire codebase just to shuffle whitespace around. seems like there's more useful things to work on20:32
clarkbyou can't use both as they do not agree on style20:32
pabelangerhttps://github.com/ansible-collections/ansible.utils/blob/main/tox.ini20:32
pabelangerwe have tox -elinters / tox -eblack20:32
clarkbfungi: this is a new codebase at least20:32
pabelanger1 check, 1 reformat20:32
mhuthanks pabelanger20:32
* pabelanger goes back to the shadows20:33
fungiclarkb: yep, but then you get the "this one repo uses a different coding style than the rest of the project"20:33
clarkbfungi: fair20:33
fungibecomes fun to explain to contributors why they need to use different coding style in different repos for the same project20:34
mhuok I'll split this in two, and we can have a further discussion on adopting black on one of the reviews20:34
corvuswe've already had that discussion20:34
corvuscan we not have it again?20:35
mhuok so I'll drop the changes pertaining to using black20:36
mhuI don't remember that discussion, but then again, I've been suffering a severe case of dad brains for the last month20:37
clarkbI think the previous discussion happened when black was very new and zuul's max python version was < black's min python version. That said at the time no one was super keen on using it20:38
corvusthe short of it is that we try to avoid spending our time on whitespace and just follow the pep8 guide loosely, especially the first admonition.20:42
* corvus goes back to not-whitespace things20:42
fungimy take on coding style is "de gustibus non est disputandum"20:44
clarkbfor me the reason I'm willing to contemplate black is it is intentionally designed to make diffs smaller and more readable for human reviewers. But I'm happy to let it be and not worry about it20:47
avassI'm still in favour of the not caring too much about formatting as long as the code is readable and sane. which would would be caught during review anyway20:49
fungiyep, i think a majority of programmers are capable of judging when they should break up array elements into additional lines for cleaner updates without a style checker preventing them from not doing it when it makes sense not to do that20:51
avassfungi: had to look that and yes :)20:51
fungiand if a diff is hard to review, please ask the author to do whatever is necessary to make it understandable20:53
openstackgerritMatthieu Huin proposed zuul/zuul-client master: Add tenant as a config option  https://review.opendev.org/c/zuul/zuul-client/+/78777021:12
*** Shrews has quit IRC21:13
corvusmhu: quick suggestion on that21:16
openstackgerritMatthieu Huin proposed zuul/zuul-client master: Add tenant as a config option  https://review.opendev.org/c/zuul/zuul-client/+/78777021:20
mhucorvus, thanks, fixed ^21:20
corvus+221:23
corvusmhu: i ran into this issue the other day (i've been making sure to use zuul-client as much as possible as a user recently) -- i'd like your thoughts on it when you have a sec: https://review.opendev.org/78764221:23
corvusi'm thinking the principle of least surprise is: strip on interactive use (because it's probably a password) and don't strip on files, because you just want the file to show up exactly as-is21:25
mhucorvus, that makes sense, if you're encrypting a file chances are you want it encrypted (and decrypted later) as is21:25
mhuwas that something that didn't make the jump from the encrypting script in the zuul repo?21:27
clarkbcorvus: ++ that behavior makes sense to me21:27
corvusmhu: i don't think so -- i thought the same thing and i went and looked, and i think the current behaviors match21:27
fungihuh, i thought we flipped the default on --strip21:32
fungihttps://review.opendev.org/714508 last year21:33
corvusfungi: we did; this flips it back halfway :)21:34
fungiahh, okay21:35
corvusthere are a lot of files that will work fine without a trailing newline; i just happened to run into one that didn't, so i noticed this.21:36
avassI think vim stripping newlines for me saves me here. I usually dump a secret string to a file and encrypt that :)21:45
fungioh, huh. for some reason i thought we'd left it off for files, on for stdin/pipe21:45
avassI'm fine with either but making sure that eventually reaches a stable state without changing the behaviour too often is probably good.21:46
*** nils has joined #zuul21:48
corvusfungi: that's what i thought; which is why i wrote the change to make it do what i thought it did :)21:48
corvusavass: indeed21:49
corvusapparently at least 3 of us thought that was the behavior21:49
corvusi also considered stripping if there is only one line of text; that would be almost foolproof except for a binary data file that happened to end with a \n.21:52
corvusbut that seems like a harder behavior to explain21:52
avassyeah I just rely on fzf to have the command in history so I usually just create a file called something like: 'infile', Ctrl+R, enter.21:54
*** rlandy|rover has quit IRC22:15
*** nils has quit IRC22:48
*** ssmashnuk has quit IRC23:23
*** tosky has quit IRC23:58

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!