Thursday, 2021-03-25

*** dwilde has quit IRC00:04
*** tosky has quit IRC00:12
*** dosaboy has quit IRC01:00
*** xek_ has joined #openstack-infra01:07
*** xek has quit IRC01:09
*** jamesdenton has quit IRC01:16
*** jamesden_ has joined #openstack-infra01:17
*** dchen has quit IRC01:29
*** dchen has joined #openstack-infra02:13
*** hamalq has quit IRC02:24
*** dchen has quit IRC03:24
*** psachin has joined #openstack-infra03:25
*** bdodd has quit IRC03:35
*** ykarel|away has joined #openstack-infra03:42
*** rcernin has quit IRC03:51
*** rcernin has joined #openstack-infra03:57
*** abhishekk|away is now known as abhishekk04:21
*** vishalmanchanda has joined #openstack-infra04:23
*** psachin has quit IRC04:29
*** dchen has joined #openstack-infra04:31
*** psachin has joined #openstack-infra04:33
*** ajitha has joined #openstack-infra04:43
*** jamesden_ has quit IRC04:54
*** jamesdenton has joined #openstack-infra04:54
*** ykarel|away is now known as ykarel05:04
*** ralonsoh has joined #openstack-infra06:49
*** jaicaa has quit IRC07:04
*** jaicaa has joined #openstack-infra07:08
*** dklyle has quit IRC07:11
*** lpetrut has joined #openstack-infra07:12
*** jcapitao has joined #openstack-infra07:15
*** rcernin has quit IRC07:16
*** slaweq has joined #openstack-infra07:18
*** tkajinam has quit IRC07:33
*** eolivare has joined #openstack-infra07:33
*** tkajinam has joined #openstack-infra07:33
*** hemna has quit IRC07:49
*** hemna has joined #openstack-infra07:50
*** jamesdenton has quit IRC07:56
*** jamesdenton has joined #openstack-infra07:57
*** jpena|off is now known as jpena07:59
*** andrewbonney has joined #openstack-infra08:10
*** amoralej|off is now known as amoralej08:11
*** rpittau|afk is now known as rpittau08:13
*** sboyron has joined #openstack-infra08:19
*** hashar has joined #openstack-infra08:25
*** ykarel_ has joined #openstack-infra08:36
*** ykarel has quit IRC08:38
*** ykarel_ is now known as ykarel08:43
*** tosky has joined #openstack-infra08:43
*** gfidente|afk is now known as gfidente08:44
*** ociuhandu has joined #openstack-infra08:48
*** ykarel_ has joined #openstack-infra08:53
*** ykarel has quit IRC08:55
*** lucasagomes has joined #openstack-infra09:00
*** ykarel__ has joined #openstack-infra09:02
*** ykarel_ has quit IRC09:05
*** rcernin has joined #openstack-infra09:06
*** rcernin has quit IRC09:06
*** rcernin has joined #openstack-infra09:07
*** dciabrin_ is now known as dciabrin09:07
*** sshnaidm|afk is now known as sshnaidm09:24
*** ociuhandu has quit IRC09:28
cgoncalveshey infra team, could someone please delete stable/{queens,rocky,stein} Octavia branches please? All three Octavia release series have been EOL'd long ago.09:30
*** ykarel__ has quit IRC09:30
cgoncalvesrelated patch failing at Zuul: https://review.opendev.org/c/openstack/octavia-tempest-plugin/+/75872509:30
*** rcernin has quit IRC09:34
toskycgoncalves: not an infra people, but just in case: did you close (abandon) all open patches for those branches? I know you can't remove branches if there are open patches09:34
*** ykarel__ has joined #openstack-infra09:34
*** ykarel__ has quit IRC09:34
cgoncalvestosky, no open patches09:35
cgoncalveshttps://review.opendev.org/q/project:openstack/octavia+(branch:stable/queens+OR+branch:stable/rocky+OR+branch:stable/stein)+status:open09:35
cgoncalvestosky, this request has been an open item for the stable/infra teams for a while now. I'm just pinging folks again ;)09:36
toskyoh, oki, sorry :)09:37
cgoncalvestosky, no worries! it was a good point of yours though, made me double-check :)09:38
*** ociuhandu has joined #openstack-infra09:39
*** rcernin has joined #openstack-infra09:42
*** rcernin has quit IRC09:57
*** hashar has quit IRC10:20
*** hashar has joined #openstack-infra10:39
*** ykarel has joined #openstack-infra10:39
*** ralonsoh_ has joined #openstack-infra10:48
*** ralonsoh has quit IRC10:48
*** rcernin has joined #openstack-infra10:51
*** rcernin has quit IRC10:52
*** rcernin has joined #openstack-infra10:52
openstackgerritDaniel Bengtsson proposed openstack/pbr master: Move flake8 as a pre-commit local target.  https://review.opendev.org/c/openstack/pbr/+/78213810:54
*** lpetrut has quit IRC10:56
*** ociuhandu has quit IRC11:00
*** ociuhandu has joined #openstack-infra11:00
fungicgoncalves: tosky: manually deleting branches is a very time-consuming process and we've got hundreds requested recently by different openstack teams. i'm trying to work on a solution to provide permissions for the release team to automate deletion of them the same way they handle branch creation11:34
fungiin the past we usually just temporarily delegated access to someone in charge of stable branch management for openstack and let them work through the list, but none of them really have time to do it either11:35
*** rlandy has joined #openstack-infra11:35
*** dtantsur|afk is now known as dtantsur11:35
*** rcernin has quit IRC11:45
*** rcernin has joined #openstack-infra11:46
*** jpena is now known as jpena|lunch11:48
*** lpetrut has joined #openstack-infra11:52
*** ociuhandu has quit IRC11:52
*** stephenfin has quit IRC12:00
*** jcapitao is now known as jcapitao_lunch12:01
*** mciecierski has joined #openstack-infra12:02
*** ralonsoh_ is now known as ralonsoh12:02
*** psachin has quit IRC12:04
*** hashar is now known as hasharLunch12:10
*** ociuhandu has joined #openstack-infra12:18
*** ociuhandu has quit IRC12:27
*** jpena|lunch is now known as jpena12:29
*** rcernin has quit IRC12:29
*** ociuhandu has joined #openstack-infra12:30
*** ociuhandu_ has joined #openstack-infra12:35
*** nweinber has joined #openstack-infra12:36
*** ociuhandu has quit IRC12:39
*** ociuhandu_ has quit IRC12:40
*** ociuhandu has joined #openstack-infra12:50
*** rcernin has joined #openstack-infra12:55
*** ociuhandu has quit IRC12:56
*** lbragstad has quit IRC13:01
*** rcernin has quit IRC13:01
*** amoralej is now known as amoralej|lunch13:04
*** lbragstad has joined #openstack-infra13:05
*** ociuhandu has joined #openstack-infra13:09
*** jcapitao_lunch is now known as jcapitao13:09
*** ociuhandu has quit IRC13:14
*** ociuhandu has joined #openstack-infra13:15
*** dwilde has joined #openstack-infra13:19
*** dansmith has quit IRC13:22
*** dansmith has joined #openstack-infra13:24
*** jamesdenton has quit IRC13:26
*** jamesdenton has joined #openstack-infra13:26
*** rcernin has joined #openstack-infra13:26
*** rcernin has quit IRC13:31
*** bdodd has joined #openstack-infra13:58
*** ociuhandu has quit IRC14:03
*** amoralej|lunch is now known as amoralej14:07
*** hasharLunch is now known as hashar14:09
openstackgerritSlawek Kaplonski proposed openstack/project-config master: Add neutron stable cores +2 powers in stable neutron-lib branches  https://review.opendev.org/c/openstack/project-config/+/78301114:14
*** ociuhandu has joined #openstack-infra14:24
*** ysirndjuro has joined #openstack-infra14:44
openstackgerritMerged openstack/project-config master: Add neutron stable cores +2 powers in stable neutron-lib branches  https://review.opendev.org/c/openstack/project-config/+/78301114:49
*** __ministry1 has joined #openstack-infra14:54
*** __ministry1 has quit IRC14:55
slaweqthx fungi for quick merge of ^^14:55
funginp14:55
fungiit should take effect once the deploy job reports back on the change14:56
*** abhishekk is now known as abhishekk|afk15:00
*** dwilde has quit IRC15:04
*** dwilde has joined #openstack-infra15:06
*** jamesdenton has quit IRC15:08
*** dklyle has joined #openstack-infra15:09
*** jamesdenton has joined #openstack-infra15:10
*** gfidente has quit IRC15:12
*** jamesdenton has quit IRC15:17
*** jamesdenton has joined #openstack-infra15:17
*** gfidente has joined #openstack-infra15:21
*** rcernin has joined #openstack-infra15:27
*** rcernin has quit IRC15:32
weshay|ruck0/ hey infra.. just wondering if this patch is stuck in zuul? 774500,115:43
*** abhishekk|afk is now known as abhishekk15:45
fungiit looks like it has 3 builds which are queued, and don't have node assignments yet, i'll check the scheduler log15:49
*** lpetrut has quit IRC15:57
funginothing is jumping out for me yet in the logs and the executor and merger queues aren't off the charts, executors aren't all blocking or anything16:00
fungicorvus: ^ just a heads up, in case this is zk-related16:01
fungilooks like it started working on the pep8 job for that at 15:42:16 (so 25 minutes ago)16:07
fungibut the py36 job for it started at 03:10 and completed in under 5 minutes16:08
fungilooks like it keeps reconsidering the pep8 job every 20-30 minutes16:10
fungi2021-03-25 03:07:24,131 DEBUG zuul.Pipeline.openstack.gate: [e: 8e3ff2fe01ef43018cfb19dc508206c3] Adding node request <NodeRequest 200-0013422532 <NodeSet ubuntu-focal [<Node None ('ubuntu-focal',):ubuntu-focal>]>> for job openstack-tox-pep8 to item <QueueItem 0x7f17fe5b1130 for <Change 0x7f17fe5b1f70 openstack/tripleo-heat-templates 774500,1> in gate>16:19
fungiso that's the node request we're looking for16:19
fungii think16:19
fungidoesn't look like it ever returned as far as the scheduler is concerned16:20
fungino mention of it in nl01's debug log, though nl02 considered it for airship-kna1-airship and declined: node type(s) [ubuntu-focal] not available16:24
*** ykarel has quit IRC16:24
fungino mention on nl0316:25
funginl04 booted a node for it16:25
fungi2021-03-25 03:28:36,795 INFO nodepool.NodeLauncher: [e: 8e3ff2fe01ef43018cfb19dc508206c3] [node_request: 200-0013422532] [node: 0023674868] Node is ready16:26
*** ykarel has joined #openstack-infra16:26
fungithat node is still ready and locked according to `nodepool list`16:27
fungi| 0023674868 | ovh-bhs1            | ubuntu-focal                 | e47ff8f2-8e17-4e7b-89d9-0192dfc26def | 158.69.64.240   |                                         | ready    | 00:12:58:21  | locked   |16:27
fungiokay, so the scheduler is spamming this in its log:16:29
fungi2021-03-25 16:29:06,867 DEBUG zuul.nodepool: [e: 8e3ff2fe01ef43018cfb19dc508206c3] Unable to revise locked node request <NodeRequest 200-0013422532 <NodeSet ubuntu-focal [<Node None ('ubuntu-focal',):ubuntu-focal>]>>16:29
fungicorvus: ^ looks like i may need some help on the zk side16:29
fungii can also bring it up in #zuul16:29
corvusthe 'unable' msg is fine16:30
corvusthat's normal and expected16:30
fungiahh, okay. and yes i just checked and the current debug log says that for >1k node requests16:32
* weshay|ruck reads16:32
fungi>1k unique node request ids i mean16:32
fungiso clearly we haven't leaked >1k nodes16:33
weshay|ruckthanks for looking folks!16:34
fungiweshay|ruck: thanks for bringing it to our attention, there are big changes underway for zuul and we're trying to make sure we haven't introduced regressions16:34
weshay|rucksounds good16:35
corvusfungi: nodepool request-list says that request is pending16:37
corvusso for whatever reason, nodepool hasn't marked the request as completed16:38
fungiweird16:38
fungiso probably nothing to do with the recent scheduler changes16:38
fungicorvus: it's also not the only outstanding node request for that change, it's just the only one i ran down the request number for16:39
corvusunlikely, unless somehow zuul decided to overwrite the node request16:39
*** stephenfin has joined #openstack-infra16:40
*** ykarel has quit IRC16:40
*** hamalq has joined #openstack-infra16:47
*** ociuhandu has quit IRC16:48
*** jpena is now known as jpena|off16:49
*** hamalq_ has joined #openstack-infra16:49
*** hamalq has quit IRC16:52
fungiokay, so at 03:07:24 there were four node requests issued for that change16:53
fungi200-0013422533 (the one for openstack-tox-py36) was satisfied16:54
fungi200-0013422531 (tripleo-ci-centos-8-content-provider), 200-0013422532 (openstack-tox-pep8), and 200-0013422534 (openstack-tox-py38) were not16:54
fungi200-0013422531 was picked up by nl04 but failed 3/3 launch attempts16:58
fungidoesn't seem it went back and got retried by anyone else though16:59
fungi200-0013422534 was also picked up by nl04 and booted 0023674870 to satisfy it17:00
fungiso we have four node requests, all (including the good one which behaved normally) were all handled by nl0417:01
fungione worked fine, one failed multiple attempts but was never declined/returned, and two were satisfied but then never used (possibly not marked completed)17:02
*** lucasagomes has quit IRC17:02
*** derekh has quit IRC17:03
*** rpittau is now known as rpittau|afk17:06
*** derekh has joined #openstack-infra17:06
*** sshnaidm is now known as sshnaidm|afk17:11
fungithe scheduler debug logs confirm of the three waiting builds, no completion of the node request was seen17:12
*** mciecierski has quit IRC17:12
*** derekh has quit IRC17:17
*** derekh has joined #openstack-infra17:18
fungilooking at the node list, we have 5 in total which are in a ready+locked state, all are in ovh-bhs1, four of those were all booted in the same 15 minute span roughly 14 hours ago, while te fifth is only 2 hours old17:18
fungicorvus: what are the options here? restart nl04 (will that release the locks?), dequeue the change (that will restart builds for everything behind it which has completed though), promote something in that pipeline (restarts builds similarly), mumble mumble znode surgery to either release/delete/update the affected node requests, something else?17:18
*** tobberydberg has quit IRC17:18
*** otherwiseguy has quit IRC17:18
*** ddurst has quit IRC17:18
*** jpena|off has quit IRC17:19
*** dhill has quit IRC17:19
*** janders has quit IRC17:19
*** bradm has quit IRC17:20
*** dmellado has quit IRC17:20
*** mugsie has quit IRC17:20
*** jhesketh has quit IRC17:20
*** ramishra has quit IRC17:20
*** soniya29 has quit IRC17:20
*** arxcruz has quit IRC17:20
*** amorin has quit IRC17:20
*** dulek has quit IRC17:20
*** osmanlicilegi has quit IRC17:20
*** valleedelisle has quit IRC17:20
*** brtknr has quit IRC17:20
*** smcginnis has quit IRC17:20
*** DinaBelova has quit IRC17:20
*** lastmikoi has quit IRC17:20
*** kota_ has quit IRC17:20
*** systemd has quit IRC17:20
*** ttx has quit IRC17:20
*** logan- has quit IRC17:20
*** abhishekk has quit IRC17:20
*** calbers has quit IRC17:20
*** sorrison has quit IRC17:20
*** jmccrory has quit IRC17:20
*** mugsie has joined #openstack-infra17:20
*** tobberydberg has joined #openstack-infra17:20
*** ttx has joined #openstack-infra17:20
*** amorin has joined #openstack-infra17:20
*** calbers has joined #openstack-infra17:20
*** soniya29 has joined #openstack-infra17:20
*** systemd has joined #openstack-infra17:20
fungiunfortunately it's still not clear to me why nodepool never marked those requests completed17:20
*** janders has joined #openstack-infra17:20
*** smcginnis has joined #openstack-infra17:20
fungioh, i didn't look for tracebacks. will do that next17:20
*** jmccrory has joined #openstack-infra17:20
*** DinaBelova has joined #openstack-infra17:20
*** derekh has quit IRC17:20
*** ddurst has joined #openstack-infra17:20
*** jpena|off has joined #openstack-infra17:20
*** dhill has joined #openstack-infra17:20
*** kota_ has joined #openstack-infra17:20
*** brtknr has joined #openstack-infra17:20
*** sorrison has joined #openstack-infra17:20
*** ramishra has joined #openstack-infra17:20
*** arxcruz has joined #openstack-infra17:20
*** valleedelisle has joined #openstack-infra17:20
*** osmanlicilegi has joined #openstack-infra17:20
*** otherwiseguy has joined #openstack-infra17:20
*** jhesketh has joined #openstack-infra17:20
*** ttx has quit IRC17:20
*** stevebaker has quit IRC17:20
*** tomaw has quit IRC17:20
*** HenryG has quit IRC17:20
*** tinwood has quit IRC17:20
*** JayF has quit IRC17:20
*** eandersson has quit IRC17:20
*** akahat has quit IRC17:20
*** dpawlik has quit IRC17:20
*** dciabrin has quit IRC17:20
*** larsks has quit IRC17:20
*** another_larsks has joined #openstack-infra17:20
*** tinwood has joined #openstack-infra17:20
*** dciabrin has joined #openstack-infra17:20
*** eandersson has joined #openstack-infra17:21
*** dmellado has joined #openstack-infra17:21
*** tomaw has joined #openstack-infra17:21
*** dpawlik has joined #openstack-infra17:21
*** JayF has joined #openstack-infra17:21
*** mordred has quit IRC17:21
*** wolsen has quit IRC17:21
*** logan- has joined #openstack-infra17:21
*** ttx has joined #openstack-infra17:22
fungino smoking gun. timeouts waiting for server to come up on the ones which had retries17:23
*** abhishekk has joined #openstack-infra17:23
*** lastmikoi has joined #openstack-infra17:26
*** abhishekk has quit IRC17:27
*** abhishekk has joined #openstack-infra17:27
*** rcernin has joined #openstack-infra17:28
*** akahat has joined #openstack-infra17:30
*** rcernin has quit IRC17:35
*** JayF has quit IRC17:42
*** tinwood has quit IRC17:42
*** hamalq_ has quit IRC17:42
*** gfidente has quit IRC17:42
*** tkajinam has quit IRC17:42
*** eolivare has quit IRC17:42
*** dchen has quit IRC17:42
*** vishalmanchanda has quit IRC17:42
*** ganso has quit IRC17:42
*** cloudnull has quit IRC17:42
*** dmsimard has quit IRC17:42
*** paladox has quit IRC17:42
*** jmorgan has quit IRC17:42
*** arne_wiebalck has quit IRC17:42
*** zaro has quit IRC17:42
*** chaconpiza has quit IRC17:42
*** dmellado has quit IRC17:42
*** dpawlik has quit IRC17:42
*** another_larsks has quit IRC17:42
*** ramishra has quit IRC17:42
*** brtknr has quit IRC17:42
*** bdodd has quit IRC17:42
*** sboyron has quit IRC17:42
*** armax has quit IRC17:42
*** sshnaidm|afk has quit IRC17:42
*** EmilienM has quit IRC17:42
*** icey has quit IRC17:42
*** redrobot has quit IRC17:42
*** geguileo has quit IRC17:42
*** Anticimex has quit IRC17:42
*** spy has quit IRC17:42
*** dviroel has quit IRC17:42
*** markmcclain has quit IRC17:42
*** jroll has quit IRC17:42
*** ildikov has quit IRC17:42
*** lamt has quit IRC17:42
*** bhagyashris has quit IRC17:42
*** eharney has quit IRC17:42
*** stephenfin has quit IRC17:42
*** lbragstad has quit IRC17:42
*** nweinber has quit IRC17:42
*** ricolin has quit IRC17:42
*** adriant has quit IRC17:42
*** frickler has quit IRC17:42
*** yoctozepto has quit IRC17:42
*** openstackgerrit has quit IRC17:42
*** gary_perkins has quit IRC17:42
*** jonher has quit IRC17:42
*** prometheanfire has quit IRC17:42
*** noonedeadpunk has quit IRC17:42
*** pots has quit IRC17:42
*** weshay|ruck has quit IRC17:42
*** spotz has quit IRC17:42
*** akahat has quit IRC17:42
*** abhishekk has quit IRC17:42
*** lastmikoi has quit IRC17:42
*** DinaBelova has quit IRC17:42
*** dansmith has quit IRC17:42
*** xek_ has quit IRC17:42
*** owalsh has quit IRC17:42
*** zbr|rover has quit IRC17:42
*** trident has quit IRC17:42
*** ianw has quit IRC17:42
*** bnemec has quit IRC17:42
*** jistr has quit IRC17:42
*** corvus has quit IRC17:42
*** kevinz has quit IRC17:42
*** gouthamr has quit IRC17:42
*** gmann has quit IRC17:42
*** diablo_rojo_phon has quit IRC17:42
*** portdirect has quit IRC17:42
*** rm_work has quit IRC17:42
*** zbr|rover has joined #openstack-infra17:42
*** armax has joined #openstack-infra17:42
*** Anticimex has joined #openstack-infra17:42
*** corvus has joined #openstack-infra17:42
*** paladox has joined #openstack-infra17:42
*** owalsh has joined #openstack-infra17:42
*** stephenfin has joined #openstack-infra17:42
*** adriant has joined #openstack-infra17:43
*** eolivare has joined #openstack-infra17:43
*** sshnaidm|afk has joined #openstack-infra17:43
*** xek_ has joined #openstack-infra17:43
*** sboyron has joined #openstack-infra17:43
*** hamalq has joined #openstack-infra17:43
*** spy has joined #openstack-infra17:43
*** noonedeadpunk_ has joined #openstack-infra17:43
*** lbragstad has joined #openstack-infra17:43
*** jonher has joined #openstack-infra17:43
*** sboyron has quit IRC17:43
*** gmann has joined #openstack-infra17:43
*** larsks has joined #openstack-infra17:43
*** prometheanfire has joined #openstack-infra17:43
*** gouthamr has joined #openstack-infra17:43
*** lbragstad has quit IRC17:43
*** bnemec has joined #openstack-infra17:44
*** pots has joined #openstack-infra17:44
*** DinaBelova has joined #openstack-infra17:44
*** sboyron has joined #openstack-infra17:44
*** dchen has joined #openstack-infra17:44
*** rm_work has joined #openstack-infra17:44
*** markmcclain has joined #openstack-infra17:44
*** jistr has joined #openstack-infra17:44
*** tkajinam has joined #openstack-infra17:44
*** nweinber has joined #openstack-infra17:44
*** ianw has joined #openstack-infra17:44
*** tinwood has joined #openstack-infra17:44
*** lbragstad has joined #openstack-infra17:44
*** amoralej is now known as amoralej|off17:44
*** dmellado has joined #openstack-infra17:44
*** lastmikoi has joined #openstack-infra17:44
*** HenryG has joined #openstack-infra17:44
*** dansmith has joined #openstack-infra17:44
*** icey has joined #openstack-infra17:44
*** trident has joined #openstack-infra17:45
*** abhishekk has joined #openstack-infra17:45
*** parallax has quit IRC17:45
*** ganso has joined #openstack-infra17:45
*** bhagyashris has joined #openstack-infra17:45
*** cyberpear has quit IRC17:46
*** frickler has joined #openstack-infra17:47
*** bdodd has joined #openstack-infra17:47
*** ramishra has joined #openstack-infra17:47
*** frickler_ has joined #openstack-infra17:47
*** parallax has joined #openstack-infra17:48
*** cyberpear has joined #openstack-infra17:49
*** 7YUAAAAAD has joined #openstack-infra17:49
*** frickler_ has quit IRC17:49
*** eolivare has quit IRC17:49
*** jcapitao has quit IRC17:50
*** akahat has joined #openstack-infra17:50
*** cloudnull has joined #openstack-infra17:51
*** jhesketh has quit IRC17:52
*** jhesketh has joined #openstack-infra17:52
*** dwilde has quit IRC17:53
*** mordred has joined #openstack-infra17:54
*** jmorgan has joined #openstack-infra17:54
*** ysirndjuro has quit IRC17:57
*** rlandy has quit IRC17:57
*** hashar has quit IRC17:57
*** fresta_ has quit IRC17:57
*** auristor has quit IRC17:57
*** StevenK has quit IRC17:57
*** SotK has quit IRC17:57
*** freerunner has quit IRC17:57
*** mwhahaha has quit IRC17:57
*** hogepodge has quit IRC17:57
*** masayukig has quit IRC17:57
*** johnsom has quit IRC17:57
*** thedac has quit IRC17:57
*** melwitt has quit IRC17:57
*** nicolasbock has quit IRC17:57
*** mnaser has quit IRC17:57
*** freerunner has joined #openstack-infra17:57
*** mwhahaha has joined #openstack-infra17:57
*** StevenK has joined #openstack-infra17:57
*** hogepodge has joined #openstack-infra17:57
*** fresta has joined #openstack-infra17:57
*** rlandy has joined #openstack-infra17:57
*** nicolasbock has joined #openstack-infra17:57
*** ysirndjuro has joined #openstack-infra17:57
*** hashar has joined #openstack-infra17:57
*** thedac has joined #openstack-infra17:57
*** auristor has joined #openstack-infra17:57
*** masayukig has joined #openstack-infra17:58
*** melwitt has joined #openstack-infra17:58
*** jistr has quit IRC17:58
*** jpena|off has quit IRC17:58
*** jmccrory has quit IRC17:58
*** dtantsur has quit IRC17:58
*** iurygregory has quit IRC17:58
*** donnyd has quit IRC17:58
*** verdurin has quit IRC17:58
*** rpittau|afk has quit IRC17:58
*** jamespage has quit IRC17:58
*** vdrok has quit IRC17:58
*** mattmceuen has quit IRC17:58
*** jrosser has quit IRC17:58
*** erbarr has quit IRC17:58
*** rajinir has quit IRC17:58
*** seongsoocho has quit IRC17:58
*** mrmartin has quit IRC17:58
*** coreycb has quit IRC17:58
*** knikolla has quit IRC17:58
*** PrinzElvis has quit IRC17:58
*** Nebby has quit IRC17:58
*** rpioso has quit IRC17:58
*** srwilkers has quit IRC17:58
*** yonglihe has quit IRC17:58
*** clarkb has quit IRC17:58
*** philroche has quit IRC17:58
*** walshh_ has quit IRC17:58
*** fungi has quit IRC17:58
*** mnaser has joined #openstack-infra17:58
*** johnsom has joined #openstack-infra17:58
*** SotK has joined #openstack-infra17:58
*** yonglihe has joined #openstack-infra17:58
*** philroche has joined #openstack-infra17:58
*** vdrok has joined #openstack-infra17:58
*** PrinzElvis has joined #openstack-infra17:58
*** knikolla has joined #openstack-infra17:58
*** clarkb has joined #openstack-infra17:58
*** seongsoocho has joined #openstack-infra17:58
*** dtantsur has joined #openstack-infra17:58
*** verdurin has joined #openstack-infra17:58
*** mrmartin has joined #openstack-infra17:58
*** erbarr has joined #openstack-infra17:58
*** donnyd has joined #openstack-infra17:58
*** jmccrory has joined #openstack-infra17:58
*** rpioso has joined #openstack-infra17:58
*** rajinir has joined #openstack-infra17:58
*** rpittau|afk has joined #openstack-infra17:58
*** srwilkers has joined #openstack-infra17:58
*** jrosser has joined #openstack-infra17:58
*** walshh_ has joined #openstack-infra17:58
*** mattmceuen has joined #openstack-infra17:58
*** Nebby has joined #openstack-infra17:58
*** coreycb has joined #openstack-infra17:58
*** jpena|off has joined #openstack-infra17:58
*** jamespage has joined #openstack-infra17:58
*** JayF has joined #openstack-infra17:58
*** jistr has joined #openstack-infra17:58
zbr|roverin case one of you would find it useful: https://github.com/tox-dev/tox-extra -- to report failure if git reports dirty of untracked files.17:59
*** eharney has joined #openstack-infra17:59
*** fungi has joined #openstack-infra17:59
zbr|roveri observed that is a common error around, and now we can address it with a one-liner17:59
*** openstack has joined #openstack-infra18:02
*** ChanServ sets mode: +o openstack18:02
zbr|roveri observed that not everyone is careful to avoid leftovers after running tests18:03
*** SpamapS has quit IRC18:03
*** verdurin has quit IRC18:03
*** clarkb has quit IRC18:03
*** philroche has quit IRC18:03
*** hashar has quit IRC18:03
*** jmorgan has quit IRC18:03
*** akahat has quit IRC18:03
*** trident has quit IRC18:03
*** HenryG has quit IRC18:03
*** dmellado has quit IRC18:03
*** markmcclain has quit IRC18:03
*** csatari has quit IRC18:03
*** mnasiadka has quit IRC18:03
*** ofosos has quit IRC18:03
*** guilhermesp has quit IRC18:03
*** mrmartin has quit IRC18:03
*** erbarr has quit IRC18:03
*** donnyd has quit IRC18:03
*** dtantsur has quit IRC18:03
*** yonglihe has quit IRC18:03
*** jrosser has quit IRC18:03
*** hogepodge has quit IRC18:03
*** nicolasbock has quit IRC18:03
*** mwhahaha has quit IRC18:03
*** parallax has quit IRC18:03
*** Wasaac has quit IRC18:03
*** mgoddard has quit IRC18:03
*** dasm has quit IRC18:03
*** persia has quit IRC18:03
*** markmcclain has joined #openstack-infra18:04
zbr|roverfunny stuff is that once you install the plugin, you don't even need to add it to requires section, it will apply to any tox execution you do, for better or the worse.18:04
*** jmorgan has joined #openstack-infra18:04
*** guilhermesp has joined #openstack-infra18:04
*** verdurin has joined #openstack-infra18:04
*** hashar has joined #openstack-infra18:04
*** csatari has joined #openstack-infra18:04
*** clarkb has joined #openstack-infra18:04
*** jrosser has joined #openstack-infra18:04
*** donnyd has joined #openstack-infra18:04
*** erbarr has joined #openstack-infra18:04
*** mrmartin has joined #openstack-infra18:04
*** dtantsur has joined #openstack-infra18:04
*** yonglihe has joined #openstack-infra18:04
*** nicolasbock has joined #openstack-infra18:04
*** hogepodge has joined #openstack-infra18:04
*** mwhahaha has joined #openstack-infra18:04
*** parallax has joined #openstack-infra18:04
*** Wasaac has joined #openstack-infra18:04
*** mgoddard has joined #openstack-infra18:04
*** dasm has joined #openstack-infra18:04
*** persia has joined #openstack-infra18:04
*** mnasiadka has joined #openstack-infra18:04
*** philroche has joined #openstack-infra18:04
*** dmellado has joined #openstack-infra18:04
*** ofosos has joined #openstack-infra18:04
*** openstackstatus has joined #openstack-infra18:04
*** ChanServ sets mode: +v openstackstatus18:04
*** donnyd has quit IRC18:05
*** erbarr has quit IRC18:05
*** mrmartin has quit IRC18:05
*** dtantsur has quit IRC18:05
*** yonglihe has quit IRC18:05
*** jrosser has quit IRC18:05
*** hogepodge has quit IRC18:05
*** nicolasbock has quit IRC18:05
*** mwhahaha has quit IRC18:05
*** parallax has quit IRC18:05
*** Wasaac has quit IRC18:05
*** mgoddard has quit IRC18:05
*** dasm has quit IRC18:05
*** persia has quit IRC18:05
*** persia has joined #openstack-infra18:05
*** mrmartin has joined #openstack-infra18:05
*** hogepodge has joined #openstack-infra18:05
*** yonglihe has joined #openstack-infra18:05
*** dasm has joined #openstack-infra18:05
*** mgoddard has joined #openstack-infra18:05
*** Wasaac has joined #openstack-infra18:06
*** trident has joined #openstack-infra18:06
*** mordred has quit IRC18:06
*** cyberpear has quit IRC18:06
*** frickler has quit IRC18:06
*** abhishekk has quit IRC18:06
*** tkajinam has quit IRC18:06
*** gmann has quit IRC18:06
*** fnordahl has quit IRC18:06
*** Adri2000 has quit IRC18:06
*** lxkong has quit IRC18:06
*** Jeffrey4l has quit IRC18:06
*** gagehugo has quit IRC18:06
*** aluria has quit IRC18:06
*** Jeffrey4l has joined #openstack-infra18:06
*** frickler_ has joined #openstack-infra18:06
*** gagehugo has joined #openstack-infra18:06
fungiyeah, that looks neat. helps remind folks to stick stuff in .gitignore i suppose18:06
*** donnyd has joined #openstack-infra18:06
*** tkajinam has joined #openstack-infra18:06
*** lxkong has joined #openstack-infra18:07
*** Guest76033 has joined #openstack-infra18:07
*** gmann has joined #openstack-infra18:07
*** Adri2000 has joined #openstack-infra18:07
*** erbarr has joined #openstack-infra18:07
*** mwhahaha has joined #openstack-infra18:07
zbr|roverfungi: yep. I made a tox-bindep which runs bindep before but now I am considering merging the bindep one into extra one.18:07
*** cyberpear has joined #openstack-infra18:08
fungicorvus: you may miss this in the flurry of netsplits, but i suppose i can try an nl04 restart first, that should be reasonably nondisruptive right?18:08
*** jrosser has joined #openstack-infra18:08
corvusfungi: yep18:08
*** auristor has joined #openstack-infra18:08
*** andrewbonney has quit IRC18:08
*** cyberpear has quit IRC18:08
*** nicolasbock has joined #openstack-infra18:08
fungii'll give that a shot, before falling back on promoting or dequeuing18:09
*** dtantsur has joined #openstack-infra18:09
*** abhishekk has joined #openstack-infra18:10
*** akahat has joined #openstack-infra18:10
*** cyberpear has joined #openstack-infra18:10
*** aluria has joined #openstack-infra18:10
*** 7YUAAAAAD is now known as weshay|ruck18:11
corvusfungi: sorry i'm not much help today18:11
*** mordred has joined #openstack-infra18:12
fungicorvus: no worries! i'm usually not either ;)18:12
*** ralonsoh has joined #openstack-infra18:13
*** parallax has joined #openstack-infra18:13
*** dtantsur is now known as dtantsur|afk18:13
fungicorvus: weshay|ruck: looks like that cleared the blockage18:13
fungii see those builds starting now18:14
fungithanks!18:14
funginow if only i had some idea how it got into that state18:14
weshay|ruckwoooooot18:14
* weshay|ruck looks18:14
weshay|ruckfungi++18:15
weshay|ruckthank you!!18:15
*** ralonsoh has quit IRC18:15
corvusfungi: it must either be a nodepool issue/bug, or possible zuul overwriting the record; i'd suspect nodepool, could confirm that if there's supposed to be something like a "request complete" log msg that did not appear18:15
*** ttx has quit IRC18:16
*** eandersson has quit IRC18:16
*** dciabrin has quit IRC18:16
*** tomaw has quit IRC18:16
*** otherwiseguy has quit IRC18:16
*** kota_ has quit IRC18:16
*** valleedelisle has quit IRC18:16
*** dhill has quit IRC18:16
*** systemd has quit IRC18:16
*** amorin has quit IRC18:16
*** mugsie has quit IRC18:16
*** gnuoy has quit IRC18:16
*** elod has quit IRC18:16
*** benj_ has quit IRC18:16
*** priteau has quit IRC18:16
*** hberaud has quit IRC18:16
*** gryf has quit IRC18:16
*** kmarc has quit IRC18:16
*** mtreinish has quit IRC18:16
*** bauzas has quit IRC18:16
*** amoralej|off has quit IRC18:16
*** TerianceVi has quit IRC18:16
*** tristanC has quit IRC18:16
*** amotoki has quit IRC18:16
*** zer0c00l has quit IRC18:16
*** odyssey4me has quit IRC18:16
*** hamalq has quit IRC18:16
*** logan- has quit IRC18:16
*** sorrison has quit IRC18:16
*** osmanlicilegi has quit IRC18:16
*** smcginnis has quit IRC18:16
*** janders has quit IRC18:16
*** calbers has quit IRC18:16
*** soniya29 has quit IRC18:16
*** tobberydberg has quit IRC18:16
*** jamesdenton has quit IRC18:16
*** tosky has quit IRC18:16
*** hemna has quit IRC18:16
*** slaweq has quit IRC18:16
*** jaicaa has quit IRC18:16
*** ajitha has quit IRC18:16
*** d34dh0r53 has quit IRC18:16
*** irclogbot_3 has quit IRC18:16
*** dougwig has quit IRC18:16
*** beisner has quit IRC18:16
*** hrybacki has quit IRC18:16
*** jbryce has quit IRC18:16
*** vkmc has quit IRC18:16
*** gnuoy has joined #openstack-infra18:16
*** kmarc has joined #openstack-infra18:16
*** ajitha has joined #openstack-infra18:16
*** vkmc has joined #openstack-infra18:16
*** ttx has joined #openstack-infra18:16
*** beisner has joined #openstack-infra18:16
fungii'll try to compare the launcher logs for the actually completed request against one of the nearly-but-not-quite-completed ones18:16
*** mugsie has joined #openstack-infra18:16
*** tomaw has joined #openstack-infra18:16
*** odyssey4me has joined #openstack-infra18:16
*** calbers has joined #openstack-infra18:17
*** valleedelisle has joined #openstack-infra18:17
*** soniya29 has joined #openstack-infra18:17
*** dciabrin has joined #openstack-infra18:17
*** kota_ has joined #openstack-infra18:17
*** amotoki has joined #openstack-infra18:17
*** jaicaa has joined #openstack-infra18:17
*** osmanlicilegi has joined #openstack-infra18:17
*** hamalq has joined #openstack-infra18:17
*** hemna has joined #openstack-infra18:17
*** tosky has joined #openstack-infra18:17
*** jamesdenton has joined #openstack-infra18:17
*** otherwiseguy has joined #openstack-infra18:18
*** benj_ has joined #openstack-infra18:18
*** sorrison has joined #openstack-infra18:18
*** tobberydberg has joined #openstack-infra18:18
*** mtreinish has joined #openstack-infra18:18
*** bauzas has joined #openstack-infra18:18
*** logan- has joined #openstack-infra18:18
*** iurygregory has joined #openstack-infra18:20
*** TViernion has joined #openstack-infra18:20
*** frickler_ is now known as frickler18:20
*** cgoncalves has joined #openstack-infra18:21
*** tristanC has joined #openstack-infra18:21
*** smcginnis has joined #openstack-infra18:21
*** irclogbot_1 has joined #openstack-infra18:21
*** carloss has joined #openstack-infra18:25
*** SpamapS has joined #openstack-infra18:26
*** eandersson has joined #openstack-infra18:27
*** mordred has quit IRC18:29
*** dwilde has joined #openstack-infra18:31
*** gryf has joined #openstack-infra18:34
*** ajitha has quit IRC18:35
*** elod has joined #openstack-infra18:37
*** zer0c00l has joined #openstack-infra18:54
*** mordred has joined #openstack-infra18:55
fungiapparently the other stuck request was for a build on a openstack/tripleo-quickstart item in periodic18:56
*** EmilienM has joined #openstack-infra18:56
*** wolsen has joined #openstack-infra19:00
*** rcernin has joined #openstack-infra19:32
*** rlandy has quit IRC19:34
*** rlandy_ has joined #openstack-infra19:34
*** rcernin has quit IRC19:36
*** stevebaker has joined #openstack-infra19:58
*** nweinber has quit IRC20:00
*** zigo has joined #openstack-infra20:04
*** pleia2 has joined #openstack-infra20:19
*** rcernin has joined #openstack-infra20:30
*** priteau has joined #openstack-infra20:37
*** rcernin has quit IRC20:47
*** rcernin has joined #openstack-infra20:47
*** slaweq has joined #openstack-infra20:49
*** rlandy_ is now known as rlandy21:02
*** rcernin has quit IRC21:06
*** hashar has quit IRC21:11
*** jamesdenton has quit IRC21:48
*** jamesdenton has joined #openstack-infra21:49
*** rcernin has joined #openstack-infra21:50
*** rcernin has quit IRC21:55
*** rcernin has joined #openstack-infra22:06
*** bradm has joined #openstack-infra22:09
*** ralonsoh has joined #openstack-infra22:10
*** larsks has quit IRC22:13
*** larsks has joined #openstack-infra22:20
*** rcernin has quit IRC22:21
*** sboyron has quit IRC22:26
*** rcernin has joined #openstack-infra22:31
*** rcernin has quit IRC22:32
*** rcernin has joined #openstack-infra22:32
*** fnordahl has joined #openstack-infra22:37
*** janders has joined #openstack-infra22:54
*** amorin has joined #openstack-infra23:05
*** larsks has quit IRC23:34
*** larsks has joined #openstack-infra23:34
*** rlandy has quit IRC23:45
*** larsks has quit IRC23:52
*** larsks has joined #openstack-infra23:54

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!