Wednesday, 2024-04-17

« Tuesday, 2024-04-16 Index Thursday, 2024-04-18 »
-@gerrit:opendev.org- Dr. Jens Harbott proposed: [zuul/nodepool] 916053: Make debootstrap support noble https://review.opendev.org/c/zuul/nodepool/+/91605309:33
@newbie23:matrix.orgHI guys, can you help me to verify my understanding of "branches:" and "override-checkout:"?10:51
I reread the documentation multiple times, but I am not sure I fully grasp them.
jobs variants: jobs with the same name, they can be in the same branch or in different branches, but always in the same repo (project).
"branches:": for a given job variant, if the keyworkd is present, the variant will be used only for items that match the "branches:" value.
"override-checkout:": docs says that if a jobs has this setting, the "override-checkout:" branch will be considered regardless of the item branch.
But what if there are multiple jobs with the same name, some of them with "override-checkout" and other without the setting?
How can Zuul know which job definition to read before actually knowing the branch (that is supposed to be set by override-checkout)?
-@gerrit:opendev.org- Nils Gondermann proposed: [zuul/zuul] 915326: Add inherit-files attr to evaluate parent file matchers https://review.opendev.org/c/zuul/zuul/+/91532612:29
@clarkb:matrix.org> <@newbie23:matrix.org> HI guys, can you help me to verify my understanding of "branches:" and "override-checkout:"?15:34
> I reread the documentation multiple times, but I am not sure I fully grasp them.
>
> jobs variants: jobs with the same name, they can be in the same branch or in different branches, but always in the same repo (project).
>
> "branches:": for a given job variant, if the keyworkd is present, the variant will be used only for items that match the "branches:" value.
>
> "override-checkout:": docs says that if a jobs has this setting, the "override-checkout:" branch will be considered regardless of the item branch.
> But what if there are multiple jobs with the same name, some of them with "override-checkout" and other without the setting?
> How can Zuul know which job definition to read before actually knowing the branch (that is supposed to be set by override-checkout)?
override-checkout only affects the repo state in the running job not the job config that determines what to run
@clarkb:matrix.orgthe branches: specifier does affect which job configs apply to changes for specific branches. Typically if you repo has branches and needs different job configs for each branch you would define those specific job configs in each branch and let the implicit branch matcher (config on this branch applies to this branch) do its thing. This way when you create a new branch from an old branch you don't end up with broken/confusing configs. Then only in centralized single branch config repos would you use the branch: specifier15:35
@clarkb:matrix.orgso that your central config can have job A apply to branch foo and job A' apply to branch bar15:35
@newbie23:matrix.orgAbout override-checkout, what you say it would make sense, but from https://zuul-ci.org/docs/zuul/latest/config/job.html#attr-job.override-checkout :15:39
> This value is also used to help select which variants of a job to run. If override-checkout is set, then Zuul will use this value instead of the branch of the item being tested when collecting jobs to run.
>
That's the part I don't understand.
@newbie23:matrix.org> <@clarkb:matrix.org> override-checkout only affects the repo state in the running job not the job config that determines what to run15:40
About override-checkout, what you say it would make sense, but from https://zuul-ci.org/docs/zuul/latest/config/job.html#attr-job.override-checkout :
This value is also used to help select which variants of a job to run. If override-checkout is set, then Zuul will use this value instead of the branch of the item being tested when collecting jobs to run.
That's the part I don't understand.
@clarkb:matrix.orgnewbie23: what that means is the value to match any implicit or explicit branch values is the one in override-checkout15:43
@clarkb:matrix.orgon the `branch:` side whether implicit or explicit you are saying "this job configuration applies to changes from this branch or branches". On the override-checkout side you're saying checkout this other branch when using this repo and apply job configs for that repo as if they were from that branch.15:45
@clarkb:matrix.orgAlso you shouldn't erally need either option except in very special circumstances15:47
@clarkb:matrix.orgthe default behavior of the job config on my current branch applies to changes to this branch should work for most stuff15:47
@newbie23:matrix.orgSo "override-checkout" actually does affect which job config that determines what to run...15:51
And that is the part that bothers me 🤔
@newbie23:matrix.org> <@clarkb:matrix.org> on the `branch:` side whether implicit or explicit you are saying "this job configuration applies to changes from this branch or branches". On the override-checkout side you're saying checkout this other branch when using this repo and apply job configs for that repo as if they were from that branch.15:51
So "override-checkout" actually does affect which job config that determines what to run...
And that is the part that bothers me 🤔
-@gerrit:opendev.org- Joseph Kostreva proposed: [zuul/zuul] 915334: Refresh git-dependent changes https://review.opendev.org/c/zuul/zuul/+/91533415:54
@sjal:matrix.orgI'm setting up another Zuul environment and I'm having some trouble with envs since newer API schema version, I'll try to be brief and precise because I'm not sure if it's a bug:16:10
So, apiextensions.k8s.io/v1beta1 was deprecated in v1.16 in favor of apiextensions.k8s.io/v1, back then my CRD looked something like this:
```yaml
spec:
env:
- name: OS_CLIENT_CONFIG_FILE
value: /etc/openstack/clouds.yaml
- name: ANYTHING_REALLY
value: samehere
```
Now, that I'm using the operator again I couldn't go with the above because the schema requires an object, while the above is an array:
```yaml
env:
type: object
x-kubernetes-preserve-unknown-fields: true
````
I switched my CR to:
```yaml
env:
KEY: value
OS_CLIENT_CONFIG_FILE: /etc/openstack/clouds.yaml
```
While it passes the validation, setting it up results in:
```
[2024-04-17 16:04:50,460] kopf.objects [INFO ] [zuul/zuul] Create zuul zuul/zuul
[2024-04-17 16:04:50,463] kopf.objects [ERROR ] [zuul/zuul] Handler 'create_fn' failed with an exception. Will retry.
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/kopf/reactor/handling.py", line 266, in execute_handler_once
result = await invoke_handler(
File "/usr/local/lib/python3.8/site-packages/kopf/reactor/handling.py", line 368, in invoke_handler
result = await invocation.invoke(
File "/usr/local/lib/python3.8/site-packages/kopf/reactor/invocation.py", line 151, in invoke
await asyncio.shield(future) # slightly expensive: creates tasks
File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.8/site-packages/zuul_operator/operator.py", line 96, in create_fn
zuul = Zuul(namespace, name, logger, spec)
File "/usr/local/lib/python3.8/site-packages/zuul_operator/zuul.py", line 100, in __init__
if item.get('name') == default_key:
AttributeError: 'str' object has no attribute 'get'
[2024-04-17 16:04:50,480] kopf.objects [WARNING ] [zuul/zuul] Patching failed with inconsistencies: (('remove', ('status',), {'kopf': {'progress': {'create_fn': {'started': '2024-04-12T11:40:33.556258', 'stopped': None, 'delayed': '2024-04-17T16:05:00.463648', 'purpose': 'create', 'retries': 44214, 'success': False, 'failure': False, 'message': "'str' object has no attribute 'get'", 'subrefs': None}}}}, None),)
[2024-04-17 16:05:00,481] kopf.objects [WARNING ] [zuul/zuul] Patching failed with inconsistencies: (('remove', ('status',), {'kopf': {'dummy': '2024-04-17T16:05:00.464812'}}, None),)
```
@sjal:matrix.org * I'm setting up another Zuul environment and I'm having some trouble with envs since newer API schema version, I'll try to be brief and precise because I'm not sure if it's a bug:16:10
So, apiextensions.k8s.io/v1beta1 was deprecated in v1.16 in favor of apiextensions.k8s.io/v1, back then my CRD looked something like this:
```yaml
spec:
env:
- name: OS_CLIENT_CONFIG_FILE
value: /etc/openstack/clouds.yaml
- name: ANYTHING_REALLY
value: samehere
```
Now, that I'm using the operator again I couldn't go with the above because the schema requires an object, while the above is an array:
```yaml
env:
type: object
x-kubernetes-preserve-unknown-fields: true
```
I switched my CR to:
```yaml
env:
KEY: value
OS_CLIENT_CONFIG_FILE: /etc/openstack/clouds.yaml
```
While it passes the validation, setting it up results in:
```
[2024-04-17 16:04:50,460] kopf.objects [INFO ] [zuul/zuul] Create zuul zuul/zuul
[2024-04-17 16:04:50,463] kopf.objects [ERROR ] [zuul/zuul] Handler 'create_fn' failed with an exception. Will retry.
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/kopf/reactor/handling.py", line 266, in execute_handler_once
result = await invoke_handler(
File "/usr/local/lib/python3.8/site-packages/kopf/reactor/handling.py", line 368, in invoke_handler
result = await invocation.invoke(
File "/usr/local/lib/python3.8/site-packages/kopf/reactor/invocation.py", line 151, in invoke
await asyncio.shield(future) # slightly expensive: creates tasks
File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.8/site-packages/zuul_operator/operator.py", line 96, in create_fn
zuul = Zuul(namespace, name, logger, spec)
File "/usr/local/lib/python3.8/site-packages/zuul_operator/zuul.py", line 100, in __init__
if item.get('name') == default_key:
AttributeError: 'str' object has no attribute 'get'
[2024-04-17 16:04:50,480] kopf.objects [WARNING ] [zuul/zuul] Patching failed with inconsistencies: (('remove', ('status',), {'kopf': {'progress': {'create_fn': {'started': '2024-04-12T11:40:33.556258', 'stopped': None, 'delayed': '2024-04-17T16:05:00.463648', 'purpose': 'create', 'retries': 44214, 'success': False, 'failure': False, 'message': "'str' object has no attribute 'get'", 'subrefs': None}}}}, None),)
[2024-04-17 16:05:00,481] kopf.objects [WARNING ] [zuul/zuul] Patching failed with inconsistencies: (('remove', ('status',), {'kopf': {'dummy': '2024-04-17T16:05:00.464812'}}, None),)
```
@sjal:matrix.org * EDITED: now that I posted it I need to check something before I assume it's a bug16:12
@sjal:matrix.orgI'm setting up another Zuul environment and I'm having some trouble with envs since newer API schema version, I'll try to be brief and precise because I'm not sure if it's a bug:16:20
So, apiextensions.k8s.io/v1beta1 was deprecated in v1.16 in favor of apiextensions.k8s.io/v1, back then my CRD looked something like this:
```yaml
spec:
env:
- name: OS_CLIENT_CONFIG_FILE
value: /etc/openstack/clouds.yaml
- name: ANYTHING_REALLY
value: samehere
```
Now, that I'm using the operator again I couldn't go with the above because the schema requires an object, while the above is an array:
```yaml
env:
type: object
x-kubernetes-preserve-unknown-fields: true
```
I switched my CR to:
```yaml
env:
KEY: value
OS_CLIENT_CONFIG_FILE: /etc/openstack/clouds.yaml
```
While it passes the validation, setting it up results in:
```
[2024-04-17 16:04:50,460] kopf.objects [INFO ] [zuul/zuul] Create zuul zuul/zuul
[2024-04-17 16:04:50,463] kopf.objects [ERROR ] [zuul/zuul] Handler 'create_fn' failed with an exception. Will retry.
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/kopf/reactor/handling.py", line 266, in execute_handler_once
result = await invoke_handler(
File "/usr/local/lib/python3.8/site-packages/kopf/reactor/handling.py", line 368, in invoke_handler
result = await invocation.invoke(
File "/usr/local/lib/python3.8/site-packages/kopf/reactor/invocation.py", line 151, in invoke
await asyncio.shield(future) # slightly expensive: creates tasks
File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.8/site-packages/zuul_operator/operator.py", line 96, in create_fn
zuul = Zuul(namespace, name, logger, spec)
File "/usr/local/lib/python3.8/site-packages/zuul_operator/zuul.py", line 100, in __init__
if item.get('name') == default_key:
AttributeError: 'str' object has no attribute 'get'
[2024-04-17 16:04:50,480] kopf.objects [WARNING ] [zuul/zuul] Patching failed with inconsistencies: (('remove', ('status',), {'kopf': {'progress': {'create_fn': {'started': '2024-04-12T11:40:33.556258', 'stopped': None, 'delayed': '2024-04-17T16:05:00.463648', 'purpose': 'create', 'retries': 44214, 'success': False, 'failure': False, 'message': "'str' object has no attribute 'get'", 'subrefs': None}}}}, None),)
[2024-04-17 16:05:00,481] kopf.objects [WARNING ] [zuul/zuul] Patching failed with inconsistencies: (('remove', ('status',), {'kopf': {'dummy': '2024-04-17T16:05:00.464812'}}, None),)
```
I think the schema is wrong, am I missing something here?
@jim:acmegating.comnewbie23: Clark  the idea is if you say you want to checkout branch foo, you probably want to run the variant defined on branch foo16:20
@clarkb:matrix.orgsjal: Looks like tristanC wrote the update that changed the schema. Maybe tristanC knows17:19
@sjal:matrix.orgI think he just wrote a general schema upgrade, it's just probably noone else uses everything in the operator and it's a honest mistake17:50
@tristanc_:matrix.orgsjal: looking at the code, it seems like the operator does expect a list of {name,value} object: https://opendev.org/zuul/zuul-operator/src/branch/master/zuul_operator/zuul.py#L9917:53
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed:17:57
- [zuul/zuul] 916020: WIP fix upgrade test https://review.opendev.org/c/zuul/zuul/+/916020
- [zuul/zuul] 915514: Refactor branch cache to support more queries https://review.opendev.org/c/zuul/zuul/+/915514
- [zuul/zuul] 915515: Add a github graphql query for branch protection https://review.opendev.org/c/zuul/zuul/+/915515
- [zuul/zuul] 915516: Add support for excluding locked branches https://review.opendev.org/c/zuul/zuul/+/915516
@sjal:matrix.org> <@tristanc_:matrix.org> sjal: looking at the code, it seems like the operator does expect a list of {name,value} object: https://opendev.org/zuul/zuul-operator/src/branch/master/zuul_operator/zuul.py#L9918:00
yeah it needs a dict hence `.get()`
@tristanc_:matrix.orgSo I guess the schema is wrong, it should reflect that18:00
@sjal:matrix.orgI'll try fixing it tomorrow if you don't mind18:01
@jkkadgar:matrix.orgI updated https://review.opendev.org/c/zuul/zuul/+/915334 to solve the git-dependency issues in merge-mode: cherry-pick by simply just using refresh if a change isn't in the history. Does anyone think this will still incur a large performance problem or should I go ahead and wrap up the change by adding tests/fixing tests?18:04
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 916008: Demote launch/delete timeeouts to warnings https://review.opendev.org/c/zuul/nodepool/+/91600818:09
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed:18:42
- [zuul/zuul] 914562: Add an upgrade test https://review.opendev.org/c/zuul/zuul/+/914562
- [zuul/zuul] 915095: Run the upgrade test job https://review.opendev.org/c/zuul/zuul/+/915095
- [zuul/zuul] 915514: Refactor branch cache to support more queries https://review.opendev.org/c/zuul/zuul/+/915514
- [zuul/zuul] 915515: Add a github graphql query for branch protection https://review.opendev.org/c/zuul/zuul/+/915515
- [zuul/zuul] 915516: Add support for excluding locked branches https://review.opendev.org/c/zuul/zuul/+/915516
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed:18:53
- [zuul/zuul] 913727: Record merger operations https://review.opendev.org/c/zuul/zuul/+/913727
- [zuul/zuul] 913938: Store a repo state file in the log directory https://review.opendev.org/c/zuul/zuul/+/913938
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed:21:36
- [zuul/zuul] 916141: Support negated regexes in files/irrelevant-files https://review.opendev.org/c/zuul/zuul/+/916141
- [zuul/zuul] 916142: Ignore /COMMIT_MSG in files matchers even more https://review.opendev.org/c/zuul/zuul/+/916142
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 916008: Demote launch/delete timeeouts to warnings https://review.opendev.org/c/zuul/nodepool/+/91600821:46
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed:23:16
- [zuul/zuul] 914561: Make the test change database serializable https://review.opendev.org/c/zuul/zuul/+/914561
- [zuul/zuul] 914562: Add an upgrade test https://review.opendev.org/c/zuul/zuul/+/914562
- [zuul/zuul] 915095: Run the upgrade test job https://review.opendev.org/c/zuul/zuul/+/915095
- [zuul/zuul] 915514: Refactor branch cache to support more queries https://review.opendev.org/c/zuul/zuul/+/915514
- [zuul/zuul] 915515: Add a github graphql query for branch protection https://review.opendev.org/c/zuul/zuul/+/915515
- [zuul/zuul] 915516: Add support for excluding locked branches https://review.opendev.org/c/zuul/zuul/+/915516
« Tuesday, 2024-04-16 Index Thursday, 2024-04-18 »

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!