Thursday, 2019-07-18

« Wednesday, 2019-07-17 Index Friday, 2019-07-19 »
*** jamesmcarthur has quit IRC00:06
*** jamesmcarthur has joined #zuul00:08
*** jamesmcarthur has quit IRC00:09
*** jamesmcarthur has joined #zuul00:09
*** jamesmcarthur has quit IRC00:33
*** jamesmcarthur has joined #zuul00:33
*** jamesmcarthur has quit IRC00:39
*** jamesmcarthur has joined #zuul01:09
*** jamesmcarthur has quit IRC01:15
*** jamesmcarthur has joined #zuul01:40
*** mattw4 has quit IRC02:09
*** threestrands has joined #zuul02:43
*** bhavikdbavishi has joined #zuul02:44
*** bhavikdbavishi has quit IRC02:48
*** jamesmcarthur has quit IRC02:54
*** jamesmcarthur has joined #zuul02:55
*** jamesmcarthur has quit IRC02:59
*** jamesmcarthur has joined #zuul03:35
*** bhavikdbavishi has joined #zuul03:44
*** bjackman has joined #zuul04:14
*** sgw has joined #zuul04:26
*** swest has joined #zuul04:48
openstackgerritSimon Westphahl proposed zuul/zuul master: Record and report time for trigger events  https://review.opendev.org/67120404:56
*** jamesmcarthur has quit IRC05:10
*** bolg has joined #zuul05:30
*** pcaruana has joined #zuul05:44
*** logan- has quit IRC06:00
*** logan_ has joined #zuul06:01
*** logan_ is now known as logan-06:01
*** igordc has quit IRC06:14
*** saneax has joined #zuul06:31
*** raukadah is now known as chandankumar06:35
*** hwangbo has quit IRC07:04
*** threestrands has quit IRC07:24
*** hashar has joined #zuul07:48
*** hashar has quit IRC08:00
*** yolanda has joined #zuul08:09
*** yolanda has quit IRC08:10
*** sshnaidm|afk is now known as sshnaidm08:27
*** tosky has joined #zuul08:31
*** hashar has joined #zuul08:40
*** arxcruz|ruck is now known as arxcruz09:01
*** hashar_ has joined #zuul09:06
*** hashar has quit IRC09:06
*** electrofelix has joined #zuul09:28
*** armstrongs has joined #zuul09:52
openstackgerritMatthieu Huin proposed zuul/zuul master: Zuul CLI: allow access via REST  https://review.opendev.org/63631509:55
armstrongshey guys quick question is there any plans to make the zuul-scheduler highly available. Also what approaches do other users take in production to mitigate it being a single service? I was thinking of having my monitoring trigger an action that enabled a passive server with has a disabled service, which can be enabled and started.09:55
openstackgerritMatthieu Huin proposed zuul/zuul master: Use a requests session to simplify auth'd calls  https://review.opendev.org/67051109:56
*** hashar_ is now known as hashar09:58
*** jamesmcarthur has joined #zuul10:15
*** jamesmcarthur has quit IRC10:19
*** hashar has quit IRC10:29
*** hashar has joined #zuul10:39
*** hashar has quit IRC10:45
*** bjackman has quit IRC10:59
*** bjackman has joined #zuul11:01
*** bjackman has quit IRC11:27
*** bjackman has joined #zuul11:59
Shrewsarmstrongs: as far as plans, yes. see https://review.opendev.org/62147912:12
*** rfolco is now known as rfolco|rover12:12
*** saneax has quit IRC12:32
*** armstrongs has quit IRC12:33
*** armstrongs has joined #zuul12:33
armstrongsthanks12:35
openstackgerritMerged zuul/zuul-jobs master: Add add-launchpad-credentaials test job  https://review.opendev.org/67020712:42
mordredarmstrongs: as far as mitigation, we've been running the opendev zuul with the scheduler as a SPOF for many years now and haven't needed anything complex to deal with it. it's a really stable service. that said - we do have a cron job that saves the queue state every few minutes so that if something happens we can restore from a previously saved state ... lemme see if I can find an example of that for you12:44
mordredarmstrongs: https://opendev.org/opendev/puppet-zuul/src/branch/master/manifests/status_backups.pp#L3112:44
*** hashar has joined #zuul12:44
openstackgerritMerged zuul/zuul-jobs master: Add add-sshkey test job  https://review.opendev.org/67020812:44
mordredarmstrongs: so we save that off every minute - and keep 2 hours worth of saved queue states12:45
mordredif you decide to do a passive server, you could do a similar status queue save - but save it onto the passive server so that if you needed to failover to it, you could have your failover start the service and then restore the queue state12:46
openstackgerritTobias Henkel proposed zuul/zuul master: Evaluate CODEOWNERS settings during canMerge check  https://review.opendev.org/64455712:54
openstackgerritTobias Henkel proposed zuul/zuul master: Evaluate CODEOWNERS settings during canMerge check  https://review.opendev.org/64455712:56
*** bjackman has quit IRC12:57
armstrongsmordred: Thanks very much looks good13:00
*** bjackman has joined #zuul13:01
openstackgerritMerged zuul/zuul master: Record and report time for trigger events  https://review.opendev.org/67120413:05
*** hashar has quit IRC13:17
*** hashar has joined #zuul13:20
mhufbo, tristanC nhicher dmsimard13:21
mhuhttps://firepoker.io/#/games/6201471a-f2a1-246d-0a04-6ae1a127c86113:21
mhuoops13:21
tristanCmordred: we'll propose openapi documentation for the other endpoint this sprint. Is it normal zuul.opendev.org doesn't serve the latest web content?13:22
tobiashtristanC: it should, unless there are zuul-web changes required13:25
tobiashmaybe the automated rollout is broken after the switch to the new tenant13:25
openstackgerritMerged zuul/zuul master: Parse nodeset.group.nodes as a list  https://review.opendev.org/67137213:28
*** bjackman has quit IRC13:29
mordredtobiash, tristanC: yeah - I think we need to hup the zuul-web13:31
mordredtristanC: I pushed up a patch adding docs for /info and /tenant/{tenant}/info so I could play with doing that - it was actually really pleasing - fantastic job on all of that!13:31
Shrewstobiash: when you get time, i'd like to get your input on https://review.opendev.org/66737113:32
fboShrews: fyi I was able to reproduce the issue and then tested 667371. With the change the pool is not pause anymore and jobs attached to others label in the same pool are started.13:44
fboI haven't had time to comment on the change yet, i'll do soon.13:45
*** sgw has quit IRC13:47
*** bhavikdbavishi has quit IRC13:50
openstackgerritMerged zuul/zuul master: Fix GitEventFilter.__init__  https://review.opendev.org/61198913:54
*** jeliu_ has joined #zuul13:55
*** jeliu_ has quit IRC14:00
*** bhavikdbavishi has joined #zuul14:03
*** sgw has joined #zuul14:05
*** yolanda has joined #zuul14:09
Shrewsfbo: thx. that is helpful feedback to have14:14
*** jamesmcarthur has joined #zuul14:19
*** jamesmcarthur has quit IRC14:23
*** jamesmcarthur has joined #zuul14:24
*** jamesmcarthur has quit IRC14:43
tobiashShrews: thanks for the reminder, I'll look at it14:44
tobiashShrews: done, this looks like a clever way to solve that14:59
corvuswhat will that look like to a user running 'nodepool list'?15:03
tobiashit will look like several nodes are in state building for the same static ndoe15:04
corvushow do they get deleted?15:06
*** bolg has quit IRC15:07
fboit seems the static node ids change each time a building node is created (that what I observed)15:09
corvusoh i see15:14
corvusso this might be a little confusing for a user, but probably something they can figure out15:15
corvuswe made a lot of changes to the static driver so that "nodepool list" made sense15:16
corvus(ie, it always lists all the nodes)15:16
corvusi still think ideally we can make an algorithm change so we don't need the placeholder nodes, but maybe this is a good compromise for the moment -- this functionality is important.15:18
fboHow the ids evolved https://softwarefactory-project.io/paste/show/1546/15:21
fbobut I don't see that as a significant issue.15:22
corvusfbo: wow, that's kinda neat actually.  i can live with that for a while.  :)15:23
Shrewstobiash: fbo: corvus: im much more comfortable with this solution now that it doesn't touch the common driver code (which should be reserved for fixing a deficiency in the driver api only)15:28
Shrewsfbo: thx for that paste too15:28
*** bhavikdbavishi has quit IRC15:37
*** mattw4 has joined #zuul15:46
*** bhavikdbavishi has joined #zuul15:49
*** bolg has joined #zuul15:54
*** bolg has quit IRC15:57
*** mattw4 has quit IRC15:58
*** hashar has quit IRC16:09
*** hwangbo has joined #zuul16:11
openstackgerritMerged zuul/nodepool master: Don't pause static pool on single label quota  https://review.opendev.org/66737116:16
*** sshnaidm is now known as sshnaidm|off16:42
*** panda is now known as panda|off16:51
*** igordc has joined #zuul16:54
*** chandankumar is now known as raukadah16:56
*** AshBullock has joined #zuul17:06
*** mattw4 has joined #zuul17:08
AshBullockHey, I would like to setup an event listener on github when a tag is pushed to master, I am thinking this would be the push event listener, does anyone have any example configuration I could look at to set this up?17:09
clarkbpabelanger: ^ you may have something like that set up17:09
pabelangerhttps://github.com/ansible/project-config/blob/master/zuul.d/pipelines.yaml#L12417:10
pabelangerthat is our tag pipeline, via SpamapS17:10
AshBullockthanks alot, will try this out17:10
pabelangerthere is also release / pre-release based on tag format17:11
*** mattw4 has quit IRC17:14
*** mattw4 has joined #zuul17:14
*** igordc has quit IRC17:15
*** igordc has joined #zuul17:16
*** tosky has quit IRC17:17
*** panda|off has quit IRC17:19
*** panda has joined #zuul17:21
*** armstrongs has quit IRC17:38
*** AshBullock has quit IRC17:39
*** aprice has quit IRC17:45
*** hogepodge has quit IRC17:45
*** clayg has quit IRC17:45
*** guilhermesp has quit IRC17:46
*** hogepodge has joined #zuul17:47
*** aprice has joined #zuul17:47
*** clayg has joined #zuul17:48
*** guilhermesp has joined #zuul17:48
*** bhavikdbavishi has quit IRC17:56
*** hwangbo has quit IRC18:05
*** irclogbot_1 has quit IRC18:07
*** altlogbot_3 has quit IRC18:07
*** irclogbot_3 has joined #zuul18:09
*** altlogbot_2 has joined #zuul18:10
*** electrofelix has quit IRC18:10
*** jeliu_ has joined #zuul18:38
*** hwangbo has joined #zuul18:59
mnaserso nodepool related thing19:14
mnaserdoes nodepool wait that no vms use that specific image before deleting it19:14
mnaseror does it just have a task that runs occasionally and delete those images19:15
fungithe latter. which is normally fine except in bfv environments19:15
fungiwhere the deletes will fail until those backing volumes are cleaned up19:15
fungibut assuming that eventually happens, nodepool will eventually remove the associated images19:16
fungii think19:16
fungiShrews: ^ it does retry image deletes on failure, right?19:16
ShrewsYeah19:17
fungiwe've got some old images hanging around in vexxhost-sjc1 at the moment (4 it looks like) which may be blocked on root volume removal for nodes. i'm regenerating that report now to check19:18
Shrewsgenerally, every operation nodepool does is repeated until success, unless there is a limit (e.g., launches)19:19
fungiyeah, i count four images over a week old there. will loop over the root volumes and see which ones reference those19:19
Shrewsor it finds a reason to not retry19:20
fungiahh, the aforementioned images were uploads nodepool had lost track of for whatever reason19:50
fungithey weren't blocked on bfv19:50
fungiglance let me delete them with no trouble19:51
*** pcaruana has quit IRC20:01
corvuswe don't need to rush this, but, if someone is interested in approving the supercedes pipeline option before tomorrow, i can restart opendev zuul with it and we can try it out -- https://review.opendev.org/67067020:05
corvus(i'm planning a restart regardless to pick up the fixes to the job definition change check)20:06
tobiashcorvus: unfortunately it's too late for me to do a serious review on this20:43
clarkbI can probably take a look in a bit once I'm satisfied I've done what I can to stabilize gitea short of rebuilding 7 servers20:44
corvustobiash: we can wait for you to do a serious review if you want, or you could agree in principle and defer to clarkb.... or... whatever :)  it's most helpful to know what people want and we can adjust schedules accordingly20:46
tobiashcorvus: ++ for the principle, I like that :)20:47
tobiashcorvus: actually the change was simpler than I initially thought20:51
tobiash+320:51
*** jeliu_ has quit IRC21:03
corvusyeah, that's what i said after writing it too :)21:11
fungi25% code change, 75% documentation and tests... as it should be!21:18
*** igordc has quit IRC21:27
*** igordc has joined #zuul21:28
openstackgerritMerged zuul/zuul master: Add "supercedes" pipeline option  https://review.opendev.org/67067021:33
*** mattw4 has quit IRC23:07
*** bolg has joined #zuul23:29
*** bolg has quit IRC23:33
openstackgerritJames E. Blair proposed zuul/zuul master: Allow projects to borrow ssh keys  https://review.opendev.org/67163723:34
« Wednesday, 2019-07-17 Index Friday, 2019-07-19 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!