| -@gerrit:opendev.org- Zuul merged on behalf of lotorev vitaly: [zuul/zuul] 857065: Clarify gerrit trigger approval can be dict or list of dicts https://review.opendev.org/c/zuul/zuul/+/857065 | 00:54 | |
| -@gerrit:opendev.org- Artem Goncharov proposed: | 07:14 | |
| - [zuul/zuul] 859939: Implement Gitea driver https://review.opendev.org/c/zuul/zuul/+/859939 | ||
| - [zuul/zuul] 859940: Implement tests of the gitea driver https://review.opendev.org/c/zuul/zuul/+/859940 | ||
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul-jobs] 859943: Allow overriding of Bazel installer checksum https://review.opendev.org/c/zuul/zuul-jobs/+/859943 | 08:13 | |
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul-jobs] 859943: Allow overriding of Bazel installer checksum https://review.opendev.org/c/zuul/zuul-jobs/+/859943 | 08:31 | |
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul-jobs] 859943: Allow overriding of Bazel installer checksum https://review.opendev.org/c/zuul/zuul-jobs/+/859943 | 08:45 | |
| -@gerrit:opendev.org- Simon Westphahl proposed: | 08:47 | |
| - [zuul/zuul] 859066: Link span of queue item to trigger event span https://review.opendev.org/c/zuul/zuul/+/859066 | ||
| - [zuul/zuul] 859067: Trace received Github events https://review.opendev.org/c/zuul/zuul/+/859067 | ||
| - [zuul/zuul] 859280: Trace Gerrit connection events https://review.opendev.org/c/zuul/zuul/+/859280 | ||
| - [zuul/zuul] 859281: Trace Gitlab connection events https://review.opendev.org/c/zuul/zuul/+/859281 | ||
| - [zuul/zuul] 859282: Trace Pagure connection events https://review.opendev.org/c/zuul/zuul/+/859282 | ||
| - [zuul/zuul] 859283: Trace Git connection events https://review.opendev.org/c/zuul/zuul/+/859283 | ||
| - [zuul/zuul] 859284: Trace timer trigger events https://review.opendev.org/c/zuul/zuul/+/859284 | ||
| - [zuul/zuul] 859285: Trace Zuul trigger events https://review.opendev.org/c/zuul/zuul/+/859285 | ||
| -@gerrit:opendev.org- Simon Westphahl proposed: | 09:19 | |
| - [zuul/zuul] 859282: Trace Pagure connection events https://review.opendev.org/c/zuul/zuul/+/859282 | ||
| - [zuul/zuul] 859283: Trace Git connection events https://review.opendev.org/c/zuul/zuul/+/859283 | ||
| - [zuul/zuul] 859284: Trace timer trigger events https://review.opendev.org/c/zuul/zuul/+/859284 | ||
| - [zuul/zuul] 859285: Trace Zuul trigger events https://review.opendev.org/c/zuul/zuul/+/859285 | ||
| @jpew:matrix.org | Are there any tips for debugging `POST_FAILURES` on jobs? We have a pretty busy server and rarely it gets one which means no logs | 12:46 |
|---|---|---|
| @jpew:matrix.org | I can turn on logging in the executor, but because of our volume of builds, I only have a very short window to look at the logs | 12:46 |
| @clarkb:matrix.org | jpew: normal debug logs are often sufficient without holding the entire job workspace. Another approach is to catch it in the streaming log. Generally if you have no logs it is something directly impacting the log retrieval or upload process. | 14:03 |
| @fungicide:matrix.org | it might be worth revisiting the option we discussed a while back to split console log/json and zuul-info file collection into a separate post task which runs before the rest of the bulk log collection, in order to increase the chances that there will be useful logs for a post_failure build | 14:07 |
| @jpew:matrix.org | Looks like I can maybe temporarily redirect the logs to try and catch it with | 14:13 |
| ``` | ||
| handlers: | ||
| buildlogs: | ||
| class: logging.handlers.TimedRotatingFileHandler | ||
| level: DEBUG | ||
| formatter: simple | ||
| backupCount: 1 | ||
| when: D | ||
| filename: /var/lib/zuul/build-logs.txt | ||
| loggers: | ||
| zuul.AnsibleJob.output: | ||
| level: DEBUG | ||
| handlers: [buildlogs] | ||
| ``` | ||
| I think that will allow me to debug my problem, but it would be nice to have something more robust | ||
| @fungicide:matrix.org | though that was for handling timeouts in post-run... i guess for catching upload failures in post-run you may want a task which unconditionally runs last to upload those instead? regardless, if it's log uploading that's breaking, then it's not going to help either way | 14:13 |
| @jim:acmegating.com | note that if the final post playbook fails, all of the job-output.yaml from it is logged at debug level in the executor. | 14:15 |
| @jim:acmegating.com | * note that if the final post playbook fails, all of the job-output.json from it is logged at debug level in the executor (as yaml). | 14:16 |
| @jpew:matrix.org | @corvus what logger is that? | 14:16 |
| @jim:acmegating.com | should be `zuul.AnsibleJob` | 14:17 |
| @jim:acmegating.com | `Final playbook failed` is a key message to identify that sequence | 14:18 |
| @jpew:matrix.org | corvus: Hmm, but if I set the logging level to DEBUG I get all the ansible output anyway? | 14:19 |
| @jim:acmegating.com | jpew: yes, you'll get the ansible output *and* the json which has even more info | 14:19 |
| @jpew:matrix.org | I guess I would want `zuul.AnsibleJob` but not `zuul.AnsibleJob.outpu` | 14:19 |
| @jpew:matrix.org | * I guess I would want `zuul.AnsibleJob` but not `zuul.AnsibleJob.output` | 14:19 |
| @jpew:matrix.org | Ah | 14:19 |
| @jim:acmegating.com | sometimes the ansible output is as helpful as "oops" :) | 14:20 |
| @jim:acmegating.com | and sometimes the json is more helpful because it has input parameter values, etc | 14:20 |
| @jpew:matrix.org | Does the JSON have the output also? | 14:20 |
| @jim:acmegating.com | it should have the equivalent information. | 14:21 |
| @jim:acmegating.com | it'll be missing things like the task summary lines that are generated by the streaming output. it's basically what's in the console tab vs the streaming log output in zuul. | 14:22 |
| @jpew:matrix.org | Maybe the exporter could have an option to save the JSON file off somewhere? | 14:37 |
| @jim:acmegating.com | you mean the executor? i think it would be a better approach to just promote that log line to info | 14:41 |
| @jpew:matrix.org | Or if it had a dedicated logging domain it could be controlled by the structured logging | 14:43 |
| @jim:acmegating.com | (though if you would like to save the json output to a local site when the playbook fails, that is something that you could do in ansible) | 14:43 |
| @jpew:matrix.org | > <@jim:acmegating.com> (though if you would like to save the json output to a local site when the playbook fails, that is something that you could do in ansible) | 14:44 |
| How? | ||
| @jim:acmegating.com | jpew: with a cleanup playbook that copies the job-output.json to a trusted_rw_paths location if the log upload failed. | 14:48 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 20:32 | |
| - [zuul/zuul] 858987: Rename admin-rule to authorization-rule https://review.opendev.org/c/zuul/zuul/+/858987 | ||
| - [zuul/zuul] 859353: Remove unused /api/user/authorizations REST endpoint https://review.opendev.org/c/zuul/zuul/+/859353 | ||
| - [zuul/zuul] 859354: Simplify tenant_authorizatons check https://review.opendev.org/c/zuul/zuul/+/859354 | ||
| - [zuul/zuul] 859355: Add check_auth tool to zuul-web https://review.opendev.org/c/zuul/zuul/+/859355 | ||
| - [zuul/zuul] 859356: Hide components in whitelabel tenants https://review.opendev.org/c/zuul/zuul/+/859356 | ||
| - [zuul/zuul] 859357: Add api-root tenant config object https://review.opendev.org/c/zuul/zuul/+/859357 | ||
| - [zuul/zuul] 859466: Linger on auth_callback page until login is complete https://review.opendev.org/c/zuul/zuul/+/859466 | ||
| - [zuul/zuul] 859481: Web: always set redux auth and wait for it https://review.opendev.org/c/zuul/zuul/+/859481 | ||
| - [zuul/zuul] 860032: Show login button any time auth is available https://review.opendev.org/c/zuul/zuul/+/860032 | ||
| - [zuul/zuul] 860033: Add access-rules configuration and documentation https://review.opendev.org/c/zuul/zuul/+/860033 | ||
| - [zuul/zuul] 860034: Set Access-Control-Allow-Origin headers in check_auth tool https://review.opendev.org/c/zuul/zuul/+/860034 | ||
| - [zuul/zuul] 860035: Support authz for read-only web access https://review.opendev.org/c/zuul/zuul/+/860035 | ||
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!