| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 944175: Fix pause event verification test race https://review.opendev.org/c/zuul/zuul/+/944175 | 00:14 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 00:31 | |
| - [zuul/zuul] 944158: Add AZ support to AWS driver https://review.opendev.org/c/zuul/zuul/+/944158 | ||
| - [zuul/zuul] 944159: Remove unused provider methods https://review.opendev.org/c/zuul/zuul/+/944159 | ||
| - [zuul/zuul] 944160: AWS: add security-group-id support https://review.opendev.org/c/zuul/zuul/+/944160 | ||
| - [zuul/zuul] 944161: AWS: Add imdsv2 support https://review.opendev.org/c/zuul/zuul/+/944161 | ||
| - [zuul/zuul] 944162: AWS: Add subnet-id https://review.opendev.org/c/zuul/zuul/+/944162 | ||
| - [zuul/zuul] 944163: AWS: Add ipv6 support https://review.opendev.org/c/zuul/zuul/+/944163 | ||
| - [zuul/zuul] 944164: AWS: Add userdata support https://review.opendev.org/c/zuul/zuul/+/944164 | ||
| - [zuul/zuul] 944165: AWS: add iam-instance-profile support https://review.opendev.org/c/zuul/zuul/+/944165 | ||
| - [zuul/zuul] 944177: AWS: add image-format option https://review.opendev.org/c/zuul/zuul/+/944177 | ||
| - [zuul/zuul] 944178: Openstack: add AZ support https://review.opendev.org/c/zuul/zuul/+/944178 | ||
| - [zuul/zuul] 944179: OpenStack: move schema attributes to match aws https://review.opendev.org/c/zuul/zuul/+/944179 | ||
| - [zuul/zuul] 944180: OpenStack: remove some unimplemented functionality https://review.opendev.org/c/zuul/zuul/+/944180 | ||
| - [zuul/zuul] 944181: OpenStack: remove key-name comments https://review.opendev.org/c/zuul/zuul/+/944181 | ||
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 938078: Fix test_supercedent_github_circular_deps_closed race (2) https://review.opendev.org/c/zuul/zuul/+/938078 | 00:34 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 00:34 | |
| - [zuul/zuul] 944166: Revert "Pin boto and friends" https://review.opendev.org/c/zuul/zuul/+/944166 | ||
| - [zuul/zuul] 944167: AWS: Remove block device mapping from template https://review.opendev.org/c/zuul/zuul/+/944167 | ||
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 944183: WIP: Implement pre-timeout for pre-run playbooks https://review.opendev.org/c/zuul/zuul/+/944183 | 01:31 | |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 943341: Allow ConnectionClosedError in unit test tracebacks https://review.opendev.org/c/zuul/zuul/+/943341 | 07:56 | |
| @joao15130:matrix.org | Hello guys, in my env I can't get nodepool creating instances required by min-ready. It seems the request is declined but I have no information. No specific quota or limit is configured in openstack. | 07:56 |
|---|---|---|
| The only thing I see is from nodepool: | ||
| r | ||
| ``` | ||
| oot@99e009966acf:/# nodepool request-list | ||
| +----------------+----------+-----------+--------------------+----------------+-------+-------------------------------------------------------------------------------------+----------------------------------+ | ||
| | Request ID | Priority | State | Requestor | Node Types | Nodes | Declined By | Event ID | | ||
| +----------------+----------+-----------+--------------------+----------------+-------+-------------------------------------------------------------------------------------+----------------------------------+ | ||
| | 900-0000005883 | 0 | requested | NodePool:min-ready | nodepool-jammy | | 99e009966acf-PoolWorker.ci-cloud-dell-openstack-ci-e619131349624fbd9c82627a92f0d300 | b936c2717b3243068689987f78a13bc2 | | ||
| | 900-0000005884 | 0 | requested | NodePool:min-ready | nodepool-jammy | | 99e009966acf-PoolWorker.ci-cloud-dell-openstack-ci-e619131349624fbd9c82627a92f0d300 | 0dfd95b19300432ab7b4dc6becdd81d1 | | ||
| | 900-0000005885 | 0 | requested | NodePool:min-ready | nodepool-jammy | | 99e009966acf-PoolWorker.ci-cloud-dell-openstack-ci-e619131349624fbd9c82627a92f0d300 | 0768ff8cb6b54655b9df9ee4ee1580c1 | | ||
| +----------------+----------+-----------+--------------------+----------------+-------+-------------------------------------------------------------------------------------+----------------------------------+ | ||
| ``` | ||
| Is there any to increase verbosity to understand the reason of that denial? | ||
| -@gerrit:opendev.org- Dong Zhang marked as active: [zuul/zuul] 943677: Support custom claim and print meta info in zuul-admin create-auth-token https://review.opendev.org/c/zuul/zuul/+/943677 | 09:04 | |
| -@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 944131: WIP: Refactor KeyStorage to utilize ZKObject https://review.opendev.org/c/zuul/zuul/+/944131 | 09:11 | |
| @joao15130:matrix.org | * Hello guys, in my env I can't get nodepool creating instances required by min-ready. It seems the request is declined but I have no information. No specific quota or limit is configured in openstack. | 11:23 |
| The only thing I see is from nodepool: | ||
| r | ||
| ``` | ||
| oot@99e009966acf:/# nodepool request-list | ||
| +----------------+----------+-----------+--------------------+----------------+-------+-------------------------------------------------------------------------------------+----------------------------------+ | ||
| | Request ID | Priority | State | Requestor | Node Types | Nodes | Declined By | Event ID | | ||
| +----------------+----------+-----------+--------------------+----------------+-------+-------------------------------------------------------------------------------------+----------------------------------+ | ||
| | 900-0000005883 | 0 | requested | NodePool:min-ready | nodepool-jammy | | 99e009966acf-PoolWorker.ci-cloud-dell-openstack-ci-e619131349624fbd9c82627a92f0d300 | b936c2717b3243068689987f78a13bc2 | | ||
| | 900-0000005884 | 0 | requested | NodePool:min-ready | nodepool-jammy | | 99e009966acf-PoolWorker.ci-cloud-dell-openstack-ci-e619131349624fbd9c82627a92f0d300 | 0dfd95b19300432ab7b4dc6becdd81d1 | | ||
| | 900-0000005885 | 0 | requested | NodePool:min-ready | nodepool-jammy | | 99e009966acf-PoolWorker.ci-cloud-dell-openstack-ci-e619131349624fbd9c82627a92f0d300 | 0768ff8cb6b54655b9df9ee4ee1580c1 | | ||
| +----------------+----------+-----------+--------------------+----------------+-------+-------------------------------------------------------------------------------------+----------------------------------+ | ||
| ``` | ||
| Is there any way to increase verbosity to understand the reason of that denial? | ||
| @mhuin:matrix.org | have you tried checking nodepool's or zuul-scheduler's logs by grepping one request id or an event id? this might hold more info | 11:31 |
| @fungicide:matrix.org | yeah, the nodepool-launcher debug logging should be very thorough about logging when it declines or yields a node request | 13:04 |
| @fungicide:matrix.org | s/logging when/why/ | 13:04 |
| @joao15130:matrix.org | I don't see anything from the scheduler logs. And I only have one line not really useful from nodepool | 13:57 |
| `2025-03-12 19:50:10,851 INFO nodepool.PoolWorker.ci-cloud-dell-openstack-ci: [e: b936c2717b3243068689987f78a13bc2] [node_request: 900-0000005883] Assigning node request <NodeRequest {'state': 'requested', 'state_time': 1741809009.3423603, 'declined_by': [], 'node_types': ['nodepool-jammy'], 'nodes': [], 'reuse': False, 'requestor': 'NodePool:min-ready', 'requestor_data': None, 'provider': None, 'relative_priority': 0, 'event_id': 'b936c2717b3243068689987f78a13bc2', 'created_time': None, 'tenant_name': None, 'id': '900-0000005883', 'stat': ZnodeStat(czxid=131757, mzxid=131757, ctime=1741809009343, mtime=1741809009343, version=0, cversion=0, aversion=0, ephemeralOwner=72071174208618514, dataLength=330, numChildren=0, pzxid=131757)}>` | ||
| @fungicide:matrix.org | your referring to min-ready may indicate a misunderstanding. nodepool should normally pre-create servers to satisfy the min-ready count when there's available quota to do so. when a node request comes in, it will use an available ready nodes or start booting a sufficient number of new nodes if there are none/not enough ready yet | 14:01 |
| @fungicide:matrix.org | if you have min-ready set to a nonzero value and nodepool isn't keeping ready nodes at idle, then something is wrong. similarly, if it receives a node request and doesn't satisfy it either with available ready nodes or by booting more, something's also wrong. might be the same something in both cases though | 14:02 |
| @joao15130:matrix.org | that was my understanding of min-ready, and we are not reaching any quota limitations. For example the default quota is 10 instances, and we have nodepool instance locked by a job. If min-ready is 2, there should be 2 instances ready. | 14:04 |
| @jim:acmegating.com | Jean Pierre Roquesalane: run with `-d` to enable debug logs. https://zuul-ci.org/docs/nodepool/latest/operation.html#daemon-usage | 14:04 |
| @fungicide:matrix.org | re-reading what you wrote earlier, it seems like you're focusing specifically on min-ready, so yeah there won't be anything in the scheduler log about those, nodepool handles them directly | 14:04 |
| @joao15130:matrix.org | oh yeah I got you | 14:05 |
| @joao15130:matrix.org | corvus: the process runs in a container | 14:05 |
| @fungicide:matrix.org | and yeah, you'll want debug level logging enabled for the nodepool-launcher specifically | 14:05 |
| @jim:acmegating.com | Jean Pierre Roquesalane: ok, so you'll need to configure whatever container runtime you use to add `-d` to the command line. that should be possible with all the container runtimes i'm familiar with. | 14:06 |
| @joao15130:matrix.org | ok | 14:06 |
| @fungicide:matrix.org | optionally you can set DEBUG=1 in the environment, here's where opendev's doing it in a compose file: https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/nodepool-launcher/templates/docker-compose.yaml.j2#L11 | 14:08 |
| @jim:acmegating.com | (there is another option of creating a logging config file, but -- you still need to tell the daemon to use it, so it's still an adjustment to the container runtime, so it doesn't save us any effort in this case -- plus then you have to maintain a logging config file, which i think is silly and we should stop doing it) | 14:08 |
| @jim:acmegating.com | yeah, the env var might be easier, but is still a container runtime adjustment :) | 14:09 |
| @fungicide:matrix.org | right, odds are you'll be setting it in the same file one way or the other | 14:09 |
| @fungicide:matrix.org | and yeah, the logging config can be useful for complex cases like opendev's, here's the one that compose file is referring to for example: https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/nodepool-launcher/files/logging.conf | 14:10 |
| @newbie23:matrix.org | Hi guys,we have 2 config projects foo and bar and 1 untrusted project baz. | 14:10 |
| All baz jobs has A as a parent. | ||
| I want to move a base job A from foo to bar without disruption or config errors. | ||
| The shadow attribute (see https://zuul-ci.org/docs/zuul/latest/tenants.html#attr-tenant.untrusted-projects.%3Cproject%3E.shadow) only works for untrusted projects. | ||
| What would be the best way to achieve this? | ||
| @fungicide:matrix.org | i've never tried to do that non-disruptively, always used an identical job under a different name in the new location and then switched over the depending jobs over to it, cleaning up the original once there are no remaining references | 14:12 |
| @fungicide:matrix.org | s/non-disruptively/transparently/ | 14:13 |
| @joao15130:matrix.org | corvus and fungi I have added -d to the exec run command `sh -c '/var/playbooks/wait-to-start-certs.sh && exec nodepool-launcher -f -d'` . It should add verbosity, thank you | 14:13 |
| @fungicide:matrix.org | (migrating to a different name isn't necessarily disruptive if you do it carefully, just a bit of work) | 14:13 |
| @newbie23:matrix.org | Fair enough, thanks for your input 🙏 | 14:17 |
| Just out of curiosity, assuming the change is merged, will the "is not permitted to shadow" exception prevent from the rest of configuration to be correctly loaded? | ||
| From some tests I did in a dev instance, everything looks fine, but I wonder -- as said, I will probably go for the safer approach of using different names, but I am curious about it | ||
| @newbie23:matrix.org | Any suggestion for pipelines? | 14:25 |
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 944274: Only consider live items on dequeue event https://review.opendev.org/c/zuul/zuul/+/944274 | 15:08 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Simon Westphahl: [zuul/zuul] 943581: Enable node request fallback with zuul-launcher https://review.opendev.org/c/zuul/zuul/+/943581 | 15:27 | |
| @fungicide:matrix.org | newbie2024: changes to trusted config projects aren't speculatively tested, for security reasons, so i think you'll just end up with zuul reporting errors once it merges and any jobs or projects it affects not being able to run until the error is resolved (e.g. by merging a revert) | 15:36 |
| @fungicide:matrix.org | i should say changes to zuul configs for trusted config projects aren't speculatively tested, other content can be | 15:37 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 16:41 | |
| - [zuul/zuul] 944155: Add AWS spot support https://review.opendev.org/c/zuul/zuul/+/944155 | ||
| - [zuul/zuul] 944156: Add a postConfig hook to endpoints https://review.opendev.org/c/zuul/zuul/+/944156 | ||
| - [zuul/zuul] 944157: Add fleet support to aws driver https://review.opendev.org/c/zuul/zuul/+/944157 | ||
| - [zuul/zuul] 944158: Add AZ support to AWS driver https://review.opendev.org/c/zuul/zuul/+/944158 | ||
| - [zuul/zuul] 944159: Remove unused provider methods https://review.opendev.org/c/zuul/zuul/+/944159 | ||
| - [zuul/zuul] 944160: AWS: add security-group-id support https://review.opendev.org/c/zuul/zuul/+/944160 | ||
| - [zuul/zuul] 944161: AWS: Add imdsv2 support https://review.opendev.org/c/zuul/zuul/+/944161 | ||
| - [zuul/zuul] 944162: AWS: Add subnet-id https://review.opendev.org/c/zuul/zuul/+/944162 | ||
| - [zuul/zuul] 944163: AWS: Add ipv6 support https://review.opendev.org/c/zuul/zuul/+/944163 | ||
| - [zuul/zuul] 944164: AWS: Add userdata support https://review.opendev.org/c/zuul/zuul/+/944164 | ||
| - [zuul/zuul] 944165: AWS: add iam-instance-profile support https://review.opendev.org/c/zuul/zuul/+/944165 | ||
| - [zuul/zuul] 944177: AWS: add image-format option https://review.opendev.org/c/zuul/zuul/+/944177 | ||
| - [zuul/zuul] 944166: Revert "Pin boto and friends" https://review.opendev.org/c/zuul/zuul/+/944166 | ||
| - [zuul/zuul] 944167: AWS: Remove block device mapping from template https://review.opendev.org/c/zuul/zuul/+/944167 | ||
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 16:41 | |
| - [zuul/zuul] 944178: Openstack: add AZ support https://review.opendev.org/c/zuul/zuul/+/944178 | ||
| - [zuul/zuul] 944179: OpenStack: move schema attributes to match aws https://review.opendev.org/c/zuul/zuul/+/944179 | ||
| - [zuul/zuul] 944180: OpenStack: remove some unimplemented functionality https://review.opendev.org/c/zuul/zuul/+/944180 | ||
| - [zuul/zuul] 944181: OpenStack: remove key-name comments https://review.opendev.org/c/zuul/zuul/+/944181 | ||
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 19:02 | |
| - [zuul/zuul] 944155: Add AWS spot support https://review.opendev.org/c/zuul/zuul/+/944155 | ||
| - [zuul/zuul] 944156: Add a postConfig hook to endpoints https://review.opendev.org/c/zuul/zuul/+/944156 | ||
| - [zuul/zuul] 944157: Add fleet support to aws driver https://review.opendev.org/c/zuul/zuul/+/944157 | ||
| - [zuul/zuul] 944158: Add AZ support to AWS driver https://review.opendev.org/c/zuul/zuul/+/944158 | ||
| - [zuul/zuul] 944159: Remove unused provider methods https://review.opendev.org/c/zuul/zuul/+/944159 | ||
| - [zuul/zuul] 944160: AWS: add security-group-id support https://review.opendev.org/c/zuul/zuul/+/944160 | ||
| - [zuul/zuul] 944161: AWS: Add imdsv2 support https://review.opendev.org/c/zuul/zuul/+/944161 | ||
| - [zuul/zuul] 944162: AWS: Add subnet-id https://review.opendev.org/c/zuul/zuul/+/944162 | ||
| - [zuul/zuul] 944163: AWS: Add ipv6 support https://review.opendev.org/c/zuul/zuul/+/944163 | ||
| - [zuul/zuul] 944164: AWS: Add userdata support https://review.opendev.org/c/zuul/zuul/+/944164 | ||
| - [zuul/zuul] 944165: AWS: add iam-instance-profile support https://review.opendev.org/c/zuul/zuul/+/944165 | ||
| - [zuul/zuul] 944177: AWS: add image-format option https://review.opendev.org/c/zuul/zuul/+/944177 | ||
| - [zuul/zuul] 944166: Revert "Pin boto and friends" https://review.opendev.org/c/zuul/zuul/+/944166 | ||
| - [zuul/zuul] 944167: AWS: Remove block device mapping from template https://review.opendev.org/c/zuul/zuul/+/944167 | ||
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 19:03 | |
| - [zuul/zuul] 944178: Openstack: add AZ support https://review.opendev.org/c/zuul/zuul/+/944178 | ||
| - [zuul/zuul] 944179: OpenStack: move schema attributes to match aws https://review.opendev.org/c/zuul/zuul/+/944179 | ||
| - [zuul/zuul] 944180: OpenStack: remove some unimplemented functionality https://review.opendev.org/c/zuul/zuul/+/944180 | ||
| - [zuul/zuul] 944181: OpenStack: remove key-name comments https://review.opendev.org/c/zuul/zuul/+/944181 | ||
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!