Tuesday, 2021-12-14

-@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 821505: Report overall duration when a build set is finished https://review.opendev.org/c/zuul/zuul/+/82150502:26
-@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 821505: Report overall duration when a build set is finished https://review.opendev.org/c/zuul/zuul/+/82150502:29
-@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 821505: Report overall duration when a build set is finished https://review.opendev.org/c/zuul/zuul/+/82150502:32
@jim:acmegating.comzuul-maint: i think opendev is pretty stable... how does this look for a zuul release?  commit 9a2fd432f1ee87c8b9cb9fbb89e15590c544b844 (HEAD -> master, tag: 4.11.0, origin/master, refs/changes/75/821275/2)14:55
@avass:vassast.orgWhat's left for a V5 release after that?15:12
@jim:acmegating.comi think we'll get the docs reorg and the gearmanectomy in15:15
@ashleybullock:matrix.org      Heya, I've been looking into an odd issue with how nodepool marks an AWS ec2 as "Ready", I was wondering if anyone has encountered something similar.15:17
The ec2 is a windows 2019 server, and I've configured winrm certificate authentication. The issue is nodepool marks the instance ready for a job
before the ec2 has fully initialised (Status checks are still not intitialising) and this returns a "Device is not ready" error.
If I set min-ready to 1 and wait a little longer before scheduling a job it works fine, I've tried playing around with boot-timeout (set in nodepool config), and
winrm settings for timeouts in the executor configs but I don't think they apply as the connection is being made yet failing.
Has anyone experienced anything similar? Thanks in advance
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 821711: Add IBM Cloud VPC driver https://review.opendev.org/c/zuul/nodepool/+/82171115:31
@fungicide:matrix.orgcorvus: 9a2fd43 merged two days after the last scheduler restart in opendev, though it doesn't look like we logged that restart on tuesday15:43
@fungicide:matrix.orgi have a feeling we're running on 85dc6d6, so only the readiness/liveness probe for prometheus and the upgrade release note about delete-state merged after that15:45
@fungicide:matrix.orgseems reasonable to make the current master head 4.11.0 in that case15:46
@jim:acmegating.comfungi: that seems likely... and i'm not sure if we would exercise the prom change in opendev either...15:47
@fungicide:matrix.orgthe only potential problem we're currently aware of in opendev is management event backlogs from bulk branch manipulation (create and possibly delete) events15:47
@jim:acmegating.comyeah, and if that's a problem, it's not a recent one15:47
@jim:acmegating.com(ie, is almost certainly in 4.10.x)15:48
@fungicide:matrix.orgwe've seen it happen twice with bulk branch creation across repos twice last week, and i'm presently monitoring to see if a bulk branch deletion is going to cause the same here in a few minutes15:48
@fungicide:matrix.orgopenstack is about to delete some retired branches from repos, about 50 deletions in total, but going to try some small batches first and space them out15:49
@avass:vassast.orgAshley Bullock: i think nodepool marks the node ready as soon as it can connect to the ec2 instance on the winrm socket16:26
@avass:vassast.orgSo if you can delay the server from exposing that port until it's ready then it could work a bit better. Maybe you can block the port with the windows firewall until the server is ready could work16:29
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 821711: Add IBM Cloud VPC driver https://review.opendev.org/c/zuul/nodepool/+/82171117:09
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 821740: Update zuul-changes for use with zuul-client https://review.opendev.org/c/zuul/zuul/+/82174017:18
@jim:acmegating.comi think there's no reason not to go ahead and merge that ^ now (even prior to the gearmanectomy); it will help folks prepare.  i believe opendev should be able to use that script unmodified.17:20
@clarkb:matrix.orgIf it isn't landed by this afternoon I should be able to take a look alter today after meetings17:38
@jim:acmegating.comfungi: i reproduced the event-merge error and found the bug... i'm going to work on a fix and then double check my assertion that it was present before 4.10.  if so, i'll make the 4.11 release as planned; if not, let's regroup.18:03
@fungicide:matrix.org> <@jim:acmegating.com> fungi: i reproduced the event-merge error and found the bug... i'm going to work on a fix and then double check my assertion that it was present before 4.10.  if so, i'll make the 4.11 release as planned; if not, let's regroup.18:29
oh, awesome. thanks! i agree if it's a regression after 4.10 then fixing in 4.11 would be preferable. if it was already a problem in 4.10 then fixing after 4.11 is fine
@jim:acmegating.comfungi: i was wrong.  the bug is not yet released, so i think we should delay 4.11 for the fix.18:59
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 821771: Fix merging reconfiguration events https://review.opendev.org/c/zuul/zuul/+/82177119:00
@jim:acmegating.comzuul-maint: ^ let's hold 4.11 for that bugfix19:01
@maheshbarai1234:matrix.orgHi Team, I am trying to add Windows node via SSH key to the zuul executor. I am getting below error:19:06
- Openssh server is working fine on Windows node and we are able to ssh from zuul executor into Windows machine.
Any Clue?
2021-12-09 15:27:30,074 DEBUG zuul.AnsibleJob.output: [e: fc63d780-58fb-11ec-9cbc-6f4f0d4c3d87] [build: f652bbddd8914af5a4425d443a79dd5c] Ansible output: b'[WARNING]: Platform unknown on host windows-vm-stark is using the discovered'
2021-12-09 15:27:30,074 DEBUG zuul.AnsibleJob.output: [e: fc63d780-58fb-11ec-9cbc-6f4f0d4c3d87] [build: f652bbddd8914af5a4425d443a79dd5c] Ansible output: b'Python interpreter at /usr/bin/python, but future installation of another'
2021-12-09 15:27:30,074 DEBUG zuul.AnsibleJob.output: [e: fc63d780-58fb-11ec-9cbc-6f4f0d4c3d87] [build: f652bbddd8914af5a4425d443a79dd5c] Ansible output: b'Python interpreter could change this. See https://docs.ansible.com/ansible/2.9/'
2021-12-09 15:27:30,074 DEBUG zuul.AnsibleJob.output: [e: fc63d780-58fb-11ec-9cbc-6f4f0d4c3d87] [build: f652bbddd8914af5a4425d443a79dd5c] Ansible output: b'reference_appendices/interpreter_discovery.html for more information.'
@clarkb:matrix.orgcorvus: noted, and that is now high on my list of todos after the infra meeting19:09
@avass:vassast.orgMahesh Barai: have you configured https://zuul-ci.org/docs/nodepool/aws.html#attr-providers.[aws].cloud-images.shell-type ?19:15
-@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 821740: Update zuul-changes for use with zuul-client https://review.opendev.org/c/zuul/zuul/+/82174019:21
@tobias.henkel:matrix.orgcorvus: +2 with q on 82177119:24
@clarkb:matrix.orgtobiash: I attempted to answer19:31
@jim:acmegating.comyep correct19:58
@clarkb:matrix.orgI'ev approved that fix just now20:00
-@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 821771: Fix merging reconfiguration events https://review.opendev.org/c/zuul/zuul/+/82177122:14
@jim:acmegating.comwe just rolling-restarted opendev with that.  in doing so, Clark spotted another bug relating to semaphores.  i believe i understand the issue and will work on a test/fix.23:04

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!