Monday, 2021-05-24

*** gmann_afk is now known as gmann01:30
openstackgerritJames E. Blair proposed zuul/zuul master: Inherited dependencies are overwritten  https://review.opendev.org/c/zuul/zuul/+/79199402:03
*** jamesmcarthur has quit IRC03:52
*** jamesmcarthur has joined #zuul03:53
*** jamesmcarthur has quit IRC03:58
*** bhavikdbavishi has joined #zuul04:09
*** jfoufas1 has joined #zuul04:21
*** jamesmcarthur has joined #zuul04:23
*** jamesmcarthur has quit IRC04:40
*** jamesmcarthur has joined #zuul04:53
*** ykarel|away has joined #zuul04:54
*** freefood_ has quit IRC04:56
*** jamesmcarthur has quit IRC04:57
*** freefood_ has joined #zuul04:57
*** ykarel|away is now known as ykarel05:00
*** jamesmcarthur has joined #zuul05:10
*** jamesmcarthur has quit IRC05:15
*** jamesmcarthur has joined #zuul05:27
*** jamesmcarthur has quit IRC05:33
*** jamesmcarthur has joined #zuul05:47
*** jamesmcarthur has quit IRC05:59
*** jamesmcarthur has joined #zuul06:13
*** jamesmcarthur has quit IRC06:19
*** bhavikdbavishi has quit IRC06:37
*** lyr has left #zuul06:43
*** sshnaidm has joined #zuul06:58
*** tosky has joined #zuul07:20
*** jamesmcarthur has joined #zuul07:32
*** jamesmcarthur has quit IRC07:36
*** vishalmanchanda has joined #zuul07:42
*** bhavikdbavishi has joined #zuul07:53
*** ykarel is now known as ykarel|lunch08:27
openstackgerritGuillaume Chauvel proposed zuul/zuul-jobs master: Fix buildset-registry test on focal  https://review.opendev.org/c/zuul/zuul-jobs/+/79234908:35
*** nils has joined #zuul08:36
openstackgerritGuillaume Chauvel proposed zuul/zuul-jobs master: Fix buildset-registry test on focal  https://review.opendev.org/c/zuul/zuul-jobs/+/79234909:01
*** ykarel|lunch is now known as ykarel09:13
openstackgerritGuillaume Chauvel proposed zuul/zuul-jobs master: Fix buildset-registry test on focal  https://review.opendev.org/c/zuul/zuul-jobs/+/79234909:21
*** jpenag is now known as jpena09:30
*** jamesmcarthur has joined #zuul09:34
*** jamesmcarthur has quit IRC09:45
*** jangutter_ has joined #zuul09:50
*** jangutter has quit IRC09:53
openstackgerritGuillaume Chauvel proposed zuul/zuul-jobs master: Fix buildset-registry test on focal  https://review.opendev.org/c/zuul/zuul-jobs/+/79234909:54
openstackgerritGuillaume Chauvel proposed zuul/zuul-jobs master: Fix buildset-registry test on focal  https://review.opendev.org/c/zuul/zuul-jobs/+/79234910:08
openstackgerritGuillaume Chauvel proposed zuul/zuul-jobs master: Fix buildset-registry test on focal  https://review.opendev.org/c/zuul/zuul-jobs/+/79234910:57
*** bhavikdbavishi has quit IRC11:09
*** saneax has joined #zuul11:14
*** saneax has quit IRC11:22
*** jpena is now known as jpena|lunch11:34
*** jamesmcarthur has joined #zuul11:41
*** rlandy has joined #zuul11:42
*** jamesmcarthur has quit IRC11:46
*** bhavikdbavishi has joined #zuul11:46
*** bhavikdbavishi1 has joined #zuul11:50
*** bhavikdbavishi has quit IRC11:51
*** bhavikdbavishi1 is now known as bhavikdbavishi11:51
*** jamesmcarthur has joined #zuul12:11
*** josefwells has joined #zuul12:14
*** jamesmcarthur has quit IRC12:15
*** jamesmcarthur has joined #zuul12:15
*** jpena|lunch is now known as jpena12:24
*** saneax has joined #zuul12:36
*** bhavikdbavishi has quit IRC12:50
*** arxcruz is now known as arxcruz|rover13:00
openstackgerritMatthieu Huin proposed zuul/zuul master: zuul-client: Test buildsets, buildset-info subcommands  https://review.opendev.org/c/zuul/zuul/+/75898513:07
sean-k-mooneyo/13:43
sean-k-mooneywe have am "interesting workflow downstream" that invovled direct force pushs to the repos contolled by gerrit...13:44
sean-k-mooneyi was wondering has the problem statemnt of triggering a ci run on a force push ever come up13:45
sean-k-mooneya little googleing and i found https://github.com/kyanny/git-hooks-detect-force-update/blob/master/pre-receive13:46
sean-k-mooneywhich provide a high level way to tell using the git rev-list13:46
sean-k-mooneyi was talking to one of oure ci maintainer about this downstream than they were wondering if the git driver could be used in this case13:47
corvussean-k-mooney: from zuul's perspective, a force-push generates a ref-update event from gerrit, so can trigger a 'post' pipeline13:47
sean-k-mooneycorvus: well that was one of the questions we had13:48
sean-k-mooneywoudl it trigger a ref-update13:48
corvusyep it does13:48
sean-k-mooneycan we tell form the ref-update that its a force push or do we have to compute that13:48
corvusversus a change merge?  i don't know off hand13:49
sean-k-mooneyyes i expect ref-updated to happen anytime we submit something13:49
sean-k-mooneyit does contain the old and new rev shas13:49
sean-k-mooneyso we coudl comare the trees and determin if one is a subset of the other13:50
sean-k-mooneyif not it was a force push13:50
sean-k-mooneyi wonder if we could use a require clause on ref udpate to trigger that comparison13:51
sean-k-mooneye.g. add ref-updated as an event trigger for the pipleine but also add a require clase for force push13:51
sean-k-mooneycorvus: long term i would prefer if we could change the workflow to not allow force pushs for rebasing downstream agaisnt upstream13:52
*** migi has joined #zuul13:52
sean-k-mooneysort term we could add a nightly perodic job to make sure thing are not broken13:53
sean-k-mooneymedium term it woudl be nice if the gerrit driver could trigger on force pushes13:53
sean-k-mooneythe simple check by had would be "git checkout -f <old ref>; git merge --ff-only <new ref>" if it can fast forward then a rebase happened that was force pushed. we should be able to compute the same for the git trees in principal but im not sure we would want to do that always or how expensive it would be13:57
fungisean-k-mooney: i'm starting to wonder if when you say "force push" you don't actually mean `git push --force`13:58
fungiare you trying to say they're just bypassing code review by directly pushing to the repositories?13:59
fungibecause `git push --force` is generally used to make non-fast-forwardable changes to a branch13:59
sean-k-mooneyfungi: i do14:00
sean-k-mooneyand yes14:00
sean-k-mooneythey are fully bypassign all our ci and pushing directly to the repos owned by gerrit14:00
sean-k-mooneywhich means if there is an issue we find it the first time we propse a new patch as a backport14:01
fungiokay, but not (necessarily) with git push --force14:01
sean-k-mooneythere workflows is to do a git rebase agsint stable/* and then git push --force14:01
sean-k-mooneyas a way to import any upstream changes into the downstream branch14:02
sean-k-mooneywithout runing ci on those change or more imporantly on any local merge conflict that were resolved automatically or manually14:02
sean-k-mooneyfungi: there is a release pipleine that happens indepently of devleopes backporting change that involves testing the code after its merged into the repo14:04
sean-k-mooneybut form a deveploper point of view the current workflow can resultin breakign the branch we backport code into then we need to go fix it first14:05
sean-k-mooneyso we are tryign to figure out if we can triger a run wehn theis force push happend and at leat know about it14:06
* sean-k-mooney wishes we woudl use zuul to gate our downstream repos exclivly but cant14:08
*** jangutter has joined #zuul14:16
*** jangutter has quit IRC14:16
*** jangutter_ has quit IRC14:16
*** jangutter has joined #zuul14:16
fungioh, i see what you're suggesting, the commit will look fast-forwardable to zuul after it's updated the branch state based on the push --force. however just checking for a ff isn't necessarily going to help because a change merged normally can also be a fast forward if the branch tip is also the commit's parent14:17
sean-k-mooneyya it could. but if a rebase happened then we could not fast forward to the new tip14:18
sean-k-mooneyall our downstream change would be moved to the end i think14:19
sean-k-mooneywell in either cae i would expect it to fail which is why they force push in the first place14:19
sean-k-mooneyi was just looking at https://opendev.org/zuul/zuul/src/commit/6479892b9c41fb818b2a6ead14c0239bf33d6481/zuul/driver/gerrit/gerritconnection.py#L96-L10114:19
sean-k-mooneyand wonderign if there was a way we could add a force-updated flag to the event that was not horbile expnsive to compute14:20
sean-k-mooneythe best i can come up with curently is if new ref can be created with a fast forward update form old ref then this was a force push14:21
sean-k-mooneythis likely is not the correct place to determin that however14:22
sean-k-mooneywe do not have a git client we can use to inspect the git reflogs or similar here14:23
corvussean-k-mooney: you might be able to detect the situation if ref-updated has a user field; but i don't know that it does.14:23
sean-k-mooneyam it has a submitter14:24
sean-k-mooneyhttps://gerrit-review.googlesource.com/Documentation/cmd-stream-events.html#_ref_updated14:24
corvusthen zuul=change-merged; not-zuul=ref-updated ?14:24
sean-k-mooneythe refUpdate attribute does not https://gerrit-review.googlesource.com/Documentation/json.html#refUpdate14:24
corvuser i mean force-push14:24
corvuszuul=change-merged; not-zuul=force-push14:25
sean-k-mooneythat might work migi ^14:25
sean-k-mooneymigi: they do not use the zuul user for this right14:25
sean-k-mooneyor at least we could ask them not to it should be there own credentials14:26
sean-k-mooneycorvus: we might still want to use ref-uddated rather then change-merged14:27
corvussorry i wasn't clear14:27
sean-k-mooneyoh you mean when we get a ref-updated check if the change was merged by zuul14:27
corvusa ref-updated event whose submitter is zuul is the result of merging a change in gerrit; a ref-updated event whose submitter is not zuul is a 'git push' (possibly with --force)14:28
corvusunless you also have the zuul user doing that14:28
sean-k-mooneyright14:28
sean-k-mooneyin that case do you think we could extend the require syntax on a pipelien to allow use to express that14:29
sean-k-mooneythe same way we can use a regex for comments?14:29
corvusyou should already be able to add the user to the trigger at least14:29
sean-k-mooneyim not sure it will match on the submitter of the refupdate event but maybe it will14:30
sean-k-mooneywe dont currently seam to include that in the internal event that is used14:30
corvusif that scheme doesn't work, you can probably do some heuristics to find out of the head commit is a change in gerrit, and if not, assume it's a force-push.  we looked into that a while ago and determined it's complicated, but possible.  you could do that in a job at the head of the job graph, so you would always run it, but let it decide whether to run anything else.14:31
sean-k-mooneyjust looking at this https://opendev.org/zuul/zuul/src/commit/6479892b9c41fb818b2a6ead14c0239bf33d6481/zuul/driver/gerrit/gerritconnection.py#L60-L15114:31
sean-k-mooneyoh there is  event.account14:31
sean-k-mooneyah right that how its populated14:32
corvusyeah we try to include account for all the gerrit events14:33
sean-k-mooneyso we would use https://sf.hosted.upshift.rdu2.redhat.com/docs/zuul/reference/drivers/gerrit.html#attr-pipeline.trigger.%3Cgerrit%20source%3E.username14:33
sean-k-mooneypipeline.trigger.<gerrit source>.username14:33
sean-k-mooneycorvus: im not sure if we can do a negitive regex can we14:35
sean-k-mooneythe example is username: ^zuul$.14:35
sean-k-mooneybut i assume username: !^zuul$. is not a thing14:35
corvussean-k-mooney: you should be able to do a negative lookahead match14:37
sean-k-mooneyi hate those but sure14:37
sean-k-mooneyi mean its not terrible but they can get slow14:38
corvussean-k-mooney: note that there's some stalled work in progress to remove that from some places in zuul, but i suspect if we do that we will add a "not-username" field to compensate.  long story short: should be good for now, but there may be a deprecation/migration in the future.14:38
corvussean-k-mooney: yeah, that's why we'd like to remove them.14:38
guillaumeccorvus, sean-k-mooney : in order to be able to setup new tutorials in any order (to init, or re-init repos: https://review.opendev.org/c/zuul/zuul/+/732067/41..48/playbooks/tutorial/roles/init-repository/tasks/main.yaml)   I made this series of patch https://review.opendev.org/c/zuul/zuul/+/739078  (on hold until all ZK patches are merged)14:38
sean-k-mooneyits just (?!^zuul$) then14:38
guillaumecie: reconfigure zuul on ref-upadted event14:38
corvusguillaumec: nice14:40
*** ykarel is now known as ykarel|away14:40
*** lyarwood has joined #zuul14:40
sean-k-mooneycorvus: actully i realised im bing naive14:42
sean-k-mooneycorvus: we dont actully have zuul merge our stuff yet. we shoudl be currently we wait for zuul and jenkins to report back and then submit via the gerrit ui14:43
sean-k-mooneycorvus: but we  could fix that with a ok_to_merge lable that we set to +1 and have zuul actully submit going forward14:43
sean-k-mooneyor somethign like that.14:44
*** jangutter_ has joined #zuul14:47
*** jangutter has quit IRC14:51
*** jangutter has joined #zuul15:03
*** jangutter_ has quit IRC15:07
openstackgerritMatthieu Huin proposed zuul/zuul master: zuul-client: Test buildsets, buildset-info subcommands  https://review.opendev.org/c/zuul/zuul/+/75898515:16
*** jangutter_ has joined #zuul15:33
*** jangutter has quit IRC15:37
*** jangutter has joined #zuul16:09
*** jangutter_ has quit IRC16:13
*** jamesmcarthur has quit IRC16:29
openstackgerritJames E. Blair proposed zuul/zuul master: Fix race with watches in ExecutorAPI  https://review.opendev.org/c/zuul/zuul/+/79230016:29
openstackgerritJames E. Blair proposed zuul/zuul master: Execute builds via ZooKeeper  https://review.opendev.org/c/zuul/zuul/+/78898816:29
openstackgerritMatthieu Huin proposed zuul/zuul-client master: Add buildsets, buildset-info to subcommands  https://review.opendev.org/c/zuul/zuul-client/+/75290916:30
openstackgerritMatthieu Huin proposed zuul/zuul master: zuul-client: Test buildsets, buildset-info subcommands  https://review.opendev.org/c/zuul/zuul/+/75898516:31
openstackgerritJames E. Blair proposed zuul/zuul master: Cache unparsed config files in Zookeeper  https://review.opendev.org/c/zuul/zuul/+/77145516:31
*** ykarel|away has quit IRC16:32
*** jamesmcarthur has joined #zuul16:42
openstackgerritJames E. Blair proposed zuul/zuul master: Add ExecutorApi  https://review.opendev.org/c/zuul/zuul/+/77090216:44
openstackgerritJames E. Blair proposed zuul/zuul master: Change zone handling in ExecutorApi  https://review.opendev.org/c/zuul/zuul/+/78783316:44
openstackgerritJames E. Blair proposed zuul/zuul master: Switch to string constants in BuildRequest  https://review.opendev.org/c/zuul/zuul/+/79184916:44
openstackgerritJames E. Blair proposed zuul/zuul master: Clean up Executor API build request locking and add tests  https://review.opendev.org/c/zuul/zuul/+/78862416:44
openstackgerritJames E. Blair proposed zuul/zuul master: Fix race with watches in ExecutorAPI  https://review.opendev.org/c/zuul/zuul/+/79230016:44
openstackgerritJames E. Blair proposed zuul/zuul master: Execute builds via ZooKeeper  https://review.opendev.org/c/zuul/zuul/+/78898816:44
*** jpena is now known as jpena|off17:23
*** bhavikdbavishi has joined #zuul17:53
*** sshnaidm is now known as sshnaidm|afk18:02
*** jfoufas1 has quit IRC18:06
*** Shrews has joined #zuul18:22
openstackgerritJames E. Blair proposed zuul/zuul master: Add ExecutorApi  https://review.opendev.org/c/zuul/zuul/+/77090218:23
openstackgerritJames E. Blair proposed zuul/zuul master: Change zone handling in ExecutorApi  https://review.opendev.org/c/zuul/zuul/+/78783318:23
openstackgerritJames E. Blair proposed zuul/zuul master: Switch to string constants in BuildRequest  https://review.opendev.org/c/zuul/zuul/+/79184918:23
openstackgerritJames E. Blair proposed zuul/zuul master: Clean up Executor API build request locking and add tests  https://review.opendev.org/c/zuul/zuul/+/78862418:23
openstackgerritJames E. Blair proposed zuul/zuul master: Fix race with watches in ExecutorAPI  https://review.opendev.org/c/zuul/zuul/+/79230018:23
openstackgerritJames E. Blair proposed zuul/zuul master: Execute builds via ZooKeeper  https://review.opendev.org/c/zuul/zuul/+/78898818:23
msuszkoI've got nodepool with azul driver to upload images to Azure (using HTTP proxy). Azul client is just pretty compared to azure sdk.18:47
msuszkoImage build using dib is 2.4GiB, and VMs /dev/sda is only 2GiB. It could by resized while VM is stopped. How do you grow an image? Uploading 28GiB of zeros seems impractical...18:53
clarkbmsuszko: we use the growroot dib element which installs a unit that on bootup expands the size of the disk and fs to fill the space set by the cloud19:03
msuszkoI have growroot element. But azure /dev/sda is 2GiB, there is no space to grow.19:04
clarkboh I see, does azure not provide larger flavors?19:04
clarkbmsuszko: the clouds we use provider instance flavors with larger disk (or we boot from volume and provide larger volume sizes) and then growroot fills that19:05
clarkbI'm not familiar enough with how azure lays out its VMs to know if there is something special you need to do19:06
msuszkoThis is "VHD" disk image made with dib. Actually format flag for qemu-img is -O vpc. This image has to be fixed, and aligned to 1MiB.19:09
msuszkoI have no clue if image can be grown after upload.19:09
clarkbour vhd images built with dib are booted in rackspace on xen and they are resizeable there. I do not know if azure can do that or not19:10
msuszkoI could grow it to 30GB before, but it means uploading extra 28GB of zeros19:10
clarkbthe constraint for us on xen is the size of the backing disk which growroot expands to fill19:11
msuszkovhd for azure is different, and dib had to be patched19:11
clarkbhttps://docs.microsoft.com/en-us/azure/virtual-machines/linux/resize-os-disk-gpt-partition that implies it is possible but all of the steps there involve stopping the VM, resizing the root disk, then starting the vm and running growroot19:13
clarkbis it possible to specify the root disk size on boot maybe?19:13
clarkbrather than after the fact?19:13
clarkb https://docs.microsoft.com/en-us/rest/api/compute/virtualmachines/createorupdate#storageprofile points to https://docs.microsoft.com/en-us/rest/api/compute/virtualmachines/createorupdate#osdisk which says diskSizeGB can set it19:15
clarkbI suspect that if you boot with ^ setting a value larger than the size of the vhd then growroot will do what you want on boot19:15
msuszkoi was looking for way to resize image, but specifying disk size while invoking vm is worth checking, thanks19:15
fungiyeah, to be clear, for openstack server instances we specify the size on boot, we don't resize the block device just the filesystem written to it19:26
*** jamesmcarthur_ has joined #zuul19:40
*** jamesmcarthur has quit IRC19:43
msuszkosolved :-) Added `"diskSizeGB": 30,` in https://review.opendev.org/c/zuul/nodepool/+/781187/6/nodepool/driver/azurestate/adapter.py#40219:49
*** josefwells_ has joined #zuul19:51
clarkbadding the disk size as a parameter that can be supplied is probably a good improvement19:51
*** josefwells has quit IRC19:54
msuszkoDefinitely19:54
*** vishalmanchanda has quit IRC19:59
*** rlandy is now known as rlandy|biab20:11
openstackgerritMatthieu Huin proposed zuul/zuul master: zuul-client: test change-status command  https://review.opendev.org/c/zuul/zuul/+/75984020:22
*** jamesmcarthur_ has quit IRC20:28
*** bhavikdbavishi has quit IRC20:38
*** jamesmcarthur has joined #zuul20:48
*** josefwells__ has joined #zuul20:50
openstackgerritMatthieu Huin proposed zuul/zuul-client master: Add console-stream subcommand  https://review.opendev.org/c/zuul/zuul-client/+/75123820:51
openstackgerritMatthieu Huin proposed zuul/zuul-client master: Add change-status subcommand  https://review.opendev.org/c/zuul/zuul-client/+/75983820:52
*** josefwells_ has quit IRC20:54
*** rlandy|biab is now known as rlandy21:00
openstackgerritJames E. Blair proposed zuul/zuul master: WIP: Lock/unlock nodes on executor server  https://review.opendev.org/c/zuul/zuul/+/77461021:07
*** saneax has quit IRC22:05
*** saneax has joined #zuul22:05
*** openstackgerrit has quit IRC22:05
*** Shrews has quit IRC22:20
*** jamesmcarthur has quit IRC22:48
*** snapiri has quit IRC23:14
*** snapiri has joined #zuul23:14
*** kgz has quit IRC23:17
*** tosky has quit IRC23:18
*** kgz has joined #zuul23:33
*** systemd has joined #zuul23:36
*** openstack has joined #zuul23:51
*** ChanServ sets mode: +o openstack23:51
*** jamesmcarthur has joined #zuul23:57

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!