| openstackgerrit | Clark Boylan proposed zuul/zuul-jobs master: Add build shard log path docs to upload-logs(-swift) https://review.opendev.org/680240 | 00:50 |
|---|---|---|
| *** mattw4 has joined #zuul | 01:08 | |
| *** mattw4 has quit IRC | 01:21 | |
| *** bhavikdbavishi has joined #zuul | 02:40 | |
| *** bhavikdbavishi1 has joined #zuul | 02:43 | |
| *** bhavikdbavishi has quit IRC | 02:44 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 02:44 | |
| *** jamesmcarthur has quit IRC | 02:49 | |
| *** jamesmcarthur has joined #zuul | 02:50 | |
| *** jamesmcarthur has quit IRC | 02:55 | |
| *** noorul has joined #zuul | 02:58 | |
| noorul | corvus: hi | 02:59 |
| noorul | corvus: I found solution for nginx | 02:59 |
| noorul | corvus: but still I don't see the console tab | 02:59 |
| *** noorul has quit IRC | 03:02 | |
| *** jamesmcarthur has joined #zuul | 03:20 | |
| *** jamesmcarthur has quit IRC | 04:47 | |
| *** raukadah is now known as chandankumar | 04:49 | |
| *** swest has joined #zuul | 04:59 | |
| *** swest has quit IRC | 05:05 | |
| *** swest has joined #zuul | 05:05 | |
| *** bjackman_ has joined #zuul | 05:08 | |
| *** jamesmcarthur has joined #zuul | 05:17 | |
| *** jamesmcarthur has quit IRC | 05:22 | |
| *** fdegir has quit IRC | 05:40 | |
| openstackgerrit | Andreas Jaeger proposed zuul/zuul-jobs master: Add build shard log path docs to upload-logs(-swift) https://review.opendev.org/680240 | 05:40 |
| *** fdegir has joined #zuul | 05:40 | |
| *** bjackman_ has quit IRC | 05:56 | |
| *** bjackman_ has joined #zuul | 06:13 | |
| *** NBorg has joined #zuul | 06:15 | |
| *** jamesmcarthur has joined #zuul | 06:19 | |
| *** jamesmcarthur has quit IRC | 06:23 | |
| *** gtema_ has joined #zuul | 06:36 | |
| *** hashar has joined #zuul | 06:38 | |
| *** gtema_ has quit IRC | 06:43 | |
| *** openstackgerrit has quit IRC | 06:51 | |
| *** sanjayu_ has quit IRC | 06:58 | |
| *** jamesmcarthur has joined #zuul | 06:59 | |
| *** saneax has joined #zuul | 07:01 | |
| *** jamesmcarthur has quit IRC | 07:06 | |
| *** themroc has joined #zuul | 07:08 | |
| *** hashar has quit IRC | 07:13 | |
| *** tosky has joined #zuul | 07:23 | |
| *** hashar has joined #zuul | 07:32 | |
| *** threestrands has quit IRC | 07:32 | |
| *** jpena|off is now known as jpena | 07:33 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 07:43 | |
| *** openstackgerrit has joined #zuul | 08:01 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure - handle initial comment change event https://review.opendev.org/680310 | 08:01 |
| *** jamesmcarthur has joined #zuul | 08:02 | |
| *** jamesmcarthur has quit IRC | 08:07 | |
| *** jank has joined #zuul | 08:21 | |
| *** bogdando has joined #zuul | 09:04 | |
| *** jamesmcarthur has joined #zuul | 09:04 | |
| bogdando | hi folks. How can I debug zuul executor cannot connect nodes with SSH? it refers to non-existing ansible env vars in /var/lib/zuul/builds/xxxxxx. So I cannot reply the failed command as that /var/lib/zuul/builds is empty... Somehow possible to retain the contents after a failure maybe? | 09:05 |
| *** jamesmcarthur has quit IRC | 09:08 | |
| *** hashar has quit IRC | 09:09 | |
| *** bjackman_ has quit IRC | 09:12 | |
| *** bjackman_ has joined #zuul | 09:12 | |
| *** jamesmcarthur has joined #zuul | 09:40 | |
| *** jamesmcarthur has quit IRC | 09:45 | |
| *** bhavikdbavishi has quit IRC | 09:52 | |
| bjackman_ | Out of curiosity, anyone got a Zuul job that runs a Python linter/code style checker and reports the results as inline Gerrit comments? | 10:14 |
| bjackman_ | I wrote one for the Linux kernel style checker and it's quite ergonomic | 10:17 |
| *** jamesmcarthur has joined #zuul | 10:41 | |
| *** jamesmcarthur has quit IRC | 10:46 | |
| AJaeger | bjackman_: Zuul reports inline gerrit comments for config error, not sure whether other jobs use that yet. mordred was working on it at some time... | 10:52 |
| AJaeger | bjackman_: see https://review.opendev.org/#/c/610744/ and dependency | 10:53 |
| AJaeger | bjackman_: so, pep8 could work - check it ;) | 10:54 |
| *** jank has quit IRC | 10:59 | |
| *** jamesmcarthur has joined #zuul | 11:18 | |
| *** jpena is now known as jpena|lunch | 11:21 | |
| bjackman_ | Ah yeah nice that looks like exactly what I was imagining | 11:21 |
| *** jamesmcarthur has quit IRC | 11:23 | |
| *** jamesmcarthur has joined #zuul | 12:00 | |
| *** jank has joined #zuul | 12:03 | |
| *** jank has quit IRC | 12:06 | |
| *** jank has joined #zuul | 12:06 | |
| *** bhavikdbavishi has joined #zuul | 12:10 | |
| *** bjackman_ has quit IRC | 12:20 | |
| *** gtema_ has joined #zuul | 12:30 | |
| *** bhavikdbavishi has quit IRC | 12:56 | |
| pabelanger | just seen this on twitter | 12:58 |
| pabelanger | https://model-engineers.com/en/company/references/success-stories/volvo-cars/ | 12:58 |
| pabelanger | that is really cool | 12:58 |
| *** gtema_ has quit IRC | 12:58 | |
| *** hashar has joined #zuul | 12:59 | |
| *** jpena|lunch is now known as jpena | 13:01 | |
| *** jamesmcarthur has quit IRC | 13:04 | |
| *** bhavikdbavishi has joined #zuul | 13:04 | |
| *** pcaruana has quit IRC | 13:10 | |
| *** jank has quit IRC | 13:17 | |
| *** bogdando has left #zuul | 13:18 | |
| *** bhavikdbavishi has quit IRC | 13:28 | |
| *** pcaruana has joined #zuul | 13:39 | |
| *** mailingsam97 has joined #zuul | 13:51 | |
| mailingsam97 | Hello, this build is failing, https://review.opendev.org/#/c/663404/ could you let me know where I can see the PEP error? | 13:52 |
| *** mailingsam97 has quit IRC | 14:03 | |
| *** pcaruana has quit IRC | 14:12 | |
| fungi | maxamillion: opendev was having some log publication issues, and the log for that build looks like it may have been a related casualty. you can leave a review comment which starts with the word "recheck" to get it rerun | 14:16 |
| maxamillion | fungi: I assume that was for mailingsam97 but they disconnected :) | 14:16 |
| fungi | oh, oops yes, sorry maxamillion! | 14:18 |
| maxamillion | no worries :) | 14:18 |
| maxamillion | tab complete is a blessing and a curse ;) | 14:18 |
| * fungi gets frustrated by people who ask questions and then disappear in 10 minutes | 14:18 | |
| *** jamesmcarthur has joined #zuul | 14:23 | |
| *** swest has quit IRC | 14:24 | |
| *** hashar has quit IRC | 14:33 | |
| *** jamesmcarthur has quit IRC | 14:43 | |
| *** mattw4 has joined #zuul | 14:46 | |
| *** jamesmcarthur has joined #zuul | 15:02 | |
| *** tosky has quit IRC | 15:15 | |
| *** mattw4 has quit IRC | 15:17 | |
| *** jamesmcarthur has quit IRC | 15:21 | |
| clarkb | bogdando also left but there is a flag you can set via the zuul cli to tell the executor to hold build dirs for debugging purposes | 15:21 |
| *** jamesmcarthur has joined #zuul | 15:22 | |
| Shrews | maybe we need a @zuul-ci-support twitter handle for async help :) | 15:26 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-jobs master: Add build shard log path docs to upload-logs(-swift) https://review.opendev.org/680240 | 15:27 |
| *** jamesmcarthur has quit IRC | 15:27 | |
| clarkb | AJaeger: ^ I missed a text substition in my copy pasting | 15:27 |
| clarkb | zuulians ^ is a straightforward docs update followon to the log path role update I did yesterday (I realized two other roles call into that role so documenting how you can set flags on them to affect the log path role is useful) | 15:28 |
| clarkb | Shrews: we can play perl golf on support answers. What could possibly go wrong :) | 15:28 |
| *** jamesmcarthur has joined #zuul | 15:31 | |
| clarkb | zuul build dashboard ideas: Can we make the change number entry a change link instead? Also if we can show what executor a build ran on that would be helpful for debugging too | 15:39 |
| pabelanger | yah, I think that would be useful | 15:42 |
| pabelanger | haven't had that issue before | 15:43 |
| clarkb | pabelanger: means you only have to grep logs on a single host instead of all 12 :) | 15:47 |
| clarkb | I may take a look at that once I've got swift things sorted | 15:48 |
| *** saneax has quit IRC | 15:48 | |
| pabelanger | indeed | 15:49 |
| jamesmcarthur | Hi Zuul team. I'd love to respond to this via our Twitter handle: https://twitter.com/GerritReview/status/1169489856113848320?s=20 | 15:50 |
| jamesmcarthur | I know Zuul and Gerrit have a long relationship | 15:50 |
| jamesmcarthur | So any thoughts/comments I could include in a tweet or two? | 15:50 |
| jamesmcarthur | Also, congrats! Very cool :) | 15:50 |
| clarkb | jamesmcarthur: There are two related things. First is Gerrit itself investigating using Zuul for their CI. Second is Improving Zuul to use Gerrit's new check api | 15:51 |
| clarkb | jamesmcarthur: https://bugs.chromium.org/p/gerrit/issues/detail?id=11418 and https://gerrit-review.googlesource.com/q/topic:%2522subchecks%2522 are related bugs/specs in gerrit | 15:51 |
| jamesmcarthur | OK. | 15:52 |
| clarkb | jamesmcarthur: maybe soemthing like what Shrews said yseterday? it would be great to have gerrit built on zuul because zuul is already built on gerrit | 15:52 |
| clarkb | Shrews: ^ you probably said ie better :) | 15:52 |
| *** themroc has quit IRC | 15:52 | |
| jamesmcarthur | oh, that's great :) | 15:53 |
| jamesmcarthur | Thanks y'all! | 15:53 |
| Shrews | yeah, that was the gist | 15:53 |
| pabelanger | jamesmcarthur: that is zbr ! | 15:54 |
| clarkb | re dashboard ideas: I'm silly on the change link it is already there in another row | 15:54 |
| corvus | clarkb: executor idea is legit but would require a schema change | 15:56 |
| jamesmcarthur | crowd sourcing twitter is fun | 15:59 |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 16:00 | |
| *** altlogbot_1 has quit IRC | 16:01 | |
| *** irclogbot_2 has quit IRC | 16:02 | |
| *** altlogbot_1 has joined #zuul | 16:04 | |
| zbr | pabelanger: do i did not do anything wrong? when I was mentioned I believed I broke something :D | 16:04 |
| openstackgerrit | Paul Belanger proposed zuul/zuul master: Remove support for ansible 2.5 https://review.opendev.org/650431 | 16:05 |
| openstackgerrit | Paul Belanger proposed zuul/zuul master: Switch ansible_default to 2.8 https://review.opendev.org/676695 | 16:05 |
| openstackgerrit | Paul Belanger proposed zuul/zuul master: WIP: Support Ansible 2.9 https://review.opendev.org/674854 | 16:05 |
| *** irclogbot_1 has joined #zuul | 16:06 | |
| *** irclogbot_1 has quit IRC | 16:07 | |
| *** irclogbot_0 has joined #zuul | 16:09 | |
| zbr | pabelanger: re 2.8 we are going to change ansibile to 2.8 on rdo using base job approach, tomorrow (finger crossed) | 16:09 |
| pabelanger | nice | 16:13 |
| *** mattw4 has joined #zuul | 16:15 | |
| AJaeger | zuul-main, want to merge these two zuul-jobs changes: https://review.opendev.org/676430 (use fetch-sphinx-tarball for docs), https://review.opendev.org/678573 (testing for manifest) | 16:23 |
| openstackgerrit | Jimmy McArthur proposed zuul/zuul-website master: Update to page titles and Users https://review.opendev.org/680459 | 16:31 |
| *** chandankumar is now known as raukadah | 16:33 | |
| pabelanger | before I go down the wrong path, for zuul.a.c, I want us to start running elastic-recheck however today it is very openstack specific. I like the idea of trying to classify failures in elasticsearch (or your logs specifically) because these days, I am manually chasing test results to look for patterns. right now we don't have ELK running, so we need to set that up or something other (was looking at graylog). | 16:36 |
| pabelanger | But figured I'd ask here, how do we as zuul operators want to solve the issue of grouping failures together for patterns. | 16:36 |
| pabelanger | keep in mind, I also want to keep the infra needed minimal too. But really like a dashboard view of current issues we are tracking and commenting on review / PR when one is matched | 16:36 |
| *** hashar has joined #zuul | 16:37 | |
| pabelanger | because right now, I don't really know the success rate of jobs in zuul. I can guess, but would love to have actually data about it | 16:37 |
| clarkb | fwiw indexing logs is not a minimal infra activity | 16:37 |
| clarkb | if that is a strong requirement you probably need anoyher approach | 16:37 |
| pabelanger | yah, understand there will be some ops needed for that | 16:37 |
| pabelanger | was mostly, not wanting to duplicate scale of opendev | 16:38 |
| pabelanger | but, this is also not an issue limited to my install of zuul | 16:38 |
| fungi | i wouldn't be surprised if the elastic-recheck implementation in opendev consumes somwhere between 25-50% of the total service provider resources used by our control plane | 16:39 |
| clarkb | one approach may be to filter upfront using the stuff tristanC and dirk have worked on so you onlu need to store a small subset of data | 16:39 |
| clarkb | you potentially miss stuff that way but should be good most of the time | 16:39 |
| pabelanger | greylog does seem to have some sort of UI for helping with the matching stuff. | 16:40 |
| pabelanger | but admit I haven't really looked at it | 16:40 |
| pabelanger | but, I'm really interested in the reporting back to PR when a match is found | 16:40 |
| openstackgerrit | Jimmy McArthur proposed zuul/zuul-website master: CSS fix for ul/li in FAQ https://review.opendev.org/680465 | 16:41 |
| clarkb | if infrastructure is less of a concern updating e-r to be more generic shouldnt be too hard and mriedem would likely appreciate the help on the project too | 16:41 |
| openstackgerrit | Merged zuul/zuul-jobs master: Add build shard log path docs to upload-logs(-swift) https://review.opendev.org/680240 | 16:41 |
| clarkb | its mostly in the theming ui bits that need updates as well as specofying dofferent ruleset locations | 16:42 |
| pabelanger | clarkb: yah, that's was my first thought, see how to move our queries into say project-config for openstack | 16:42 |
| pabelanger | then maybe try to have zuul tenant also consume it | 16:42 |
| clarkb | It doesnt even do anything special with bug links either iirc so any bug tracker could in theory work | 16:43 |
| pabelanger | as for ELK stack, maybe flaper87 has ideas on infra :D | 16:43 |
| clarkb | though I may be wrong about that | 16:43 |
| pabelanger | or I just need to get budget approval | 16:43 |
| clarkb | pabelanger: maybe we can convince them to open source authentication >_> | 16:43 |
| corvus | the volvo folks would like to use elk too | 16:44 |
| pabelanger | I do remember being in a machine learning talk, maybe in vancouver, and was suggested that zuul jobs could do some of the heavy lifting on rendering models. I'm unsure if indexing of logs could work that way too | 16:44 |
| clarkb | the e-r toolchain intentionally decouples zuul jobs from the indexing because indexing is 1) slow 2) often unreliable due to scale | 16:45 |
| clarkb | it is intentionally best effort on the backend to avoid problems in that toolchain from affecting job results | 16:45 |
| pabelanger | yah, that makes sense | 16:46 |
| clarkb | keeping that behavior regardless of tooling is likely to be useful | 16:46 |
| pabelanger | that's kinda why I was poking at graylog, they seem to wrapped up the index / classify into one. Felt much like elastic-recheck | 16:47 |
| pabelanger | but haven't tested | 16:47 |
| *** spsurya has quit IRC | 16:48 | |
| clarkb | in the past we avoided graylog because their licensing was weird iirc | 16:48 |
| pabelanger | yah, I haven't look at that part either | 16:48 |
| clarkb | I think it is AGPL (not sure how that affects zuul communicating with it as an integral component) with important bits in the enterprise version | 16:49 |
| *** igordc has joined #zuul | 16:49 | |
| clarkb | elasticsearch (at the time) didn't have an enterprise version whcih is why I went with it but they ended up becoming more like graylog in that regard over time | 16:49 |
| openstackgerrit | Jimmy McArthur proposed zuul/zuul-website master: CSS fix for ul/li in FAQ https://review.opendev.org/680465 | 16:49 |
| clarkb | so that is probably a wash between those two choices at this point | 16:49 |
| corvus | i don't expect integration between any log system and zuul to be tight enough for licensing to matter. | 16:50 |
| pabelanger | yah, think graylog might be open core | 16:50 |
| corvus | zuul will run ansible which will run whatever log ingestor you tell it to :) | 16:51 |
| clarkb | corvus: thats a good point we already are almost completely decoupled form that perspective | 16:51 |
| clarkb | (with coupling being some helper roles that you can choose to use) | 16:51 |
| corvus | yep. i mean, we have roles that interact with proprietary services. that's a more restrictive license than agpl. | 16:52 |
| pabelanger | let me refresh myself if elastic-recheck, and the services is needs. See how minimal of an installed I need | 16:52 |
| pabelanger | s/if/with | 16:52 |
| pabelanger | I mean, it is kinda exciting too starting to deal with day 2 things for zuul.a.c. really working well so far | 16:53 |
| clarkb | pabelanger: you need an elasticsearch cluster (this is the biggest cost infra wise as it needs much memory and disk and cpu) and something to put data into elasticsearch | 16:53 |
| clarkb | then you run elastic-recheck on top of that | 16:53 |
| pabelanger | clarkb: what about logstash-workers? | 16:53 |
| clarkb | pabelanger: only if you want to use logstash to put data in elasticsearch | 16:53 |
| corvus | pabelanger: while you're looking into it -- you may want to look into whether there's a k8s operator for it. given our experience with it, that may be beneficial (do you agree clarkb?) | 16:53 |
| clarkb | but you could use anything else that works like fluentd | 16:54 |
| clarkb | we may haev to reduce assumptions about keys in the data depending on how those other tools annotate things but that shouldn't be too hard (basically stick with what you can expect from a zuul job like is this gating or not etc) | 16:54 |
| clarkb | corvus: elasticsearhc has one (monty mentioned that when we were running into gitea indexing problems iirc) | 16:54 |
| openstackgerrit | Jimmy McArthur proposed zuul/zuul-website master: CSS fix for ul/li in FAQ https://review.opendev.org/680465 | 16:55 |
| pabelanger | corvus: ack | 16:55 |
| corvus | (and yeah, sorry, "it"=="elasticsearch" in my last) | 16:55 |
| clarkb | pabelanger: in openstack's case logstash made a lot of sense because we had many many input file formats that we needed to converge to some consistent state | 16:56 |
| clarkb | pabelanger: for other systems that may not be the case and logs could be indexed much more simply (in fact with systemd/journald format logs you might even get away with just indexing those as long as everything stuck to that common format) | 16:56 |
| clarkb | pabelanger: personally I would avoid logstash if I could | 16:57 |
| clarkb | it can do a lot for you but as a result its pretty big and slow | 16:57 |
| pabelanger | yah, I need to read up more on it TBH. As long as systemd / journald / ansible logs could be indexed, that would be a lot for us I think | 16:57 |
| *** noorul has joined #zuul | 17:01 | |
| noorul | corvus: I managed to make nginx work | 17:01 |
| noorul | corvus: But I am not seeing console tab | 17:02 |
| *** igordc has quit IRC | 17:04 | |
| clarkb | pabelanger: graylog's correlation engine is enterprise only | 17:07 |
| clarkb | fyi | 17:07 |
| *** jpena is now known as jpena|off | 17:07 | |
| *** igordc has joined #zuul | 17:15 | |
| noorul | ofosos: hi | 17:17 |
| paladox | awesome to see corvus and mordred leading the way to bring zuul ci to gerrit-review! | 17:19 |
| *** igordc has quit IRC | 17:26 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul-jobs master: Flip the order of the emit-job-header tests https://review.opendev.org/680477 | 17:28 |
| corvus | noorul: do you see more info under the results heading in the summary tab? | 17:29 |
| corvus | paladox: \o/ i've started work on https://gerrit-review.googlesource.com/c/homepage/+/235693/2 (see also parent change) | 17:30 |
| paladox | :) | 17:30 |
| noorul | corvus: nothing under results heading | 17:34 |
| corvus | noorul: then i'd open developer tools in the web browser and see if there are errors in the network or console tabs | 17:34 |
| noorul | corvus: Only a JS error https://pasteboard.co/Iw21ACq.png | 17:38 |
| noorul | I was wondering, how come in independent queue there is a link between two reviews? http://zuul.opendev.org/t/zuul/status | 17:38 |
| noorul | https://pasteboard.co/Iw22fU3.png | 17:39 |
| clarkb | noorul: that represents the git graph relationship between those changes | 17:39 |
| clarkb | 674854 is a child of 676695 is a child of 650431 | 17:40 |
| noorul | clarkb: specific to gerrit? | 17:40 |
| noorul | 17:40 | |
| clarkb | no, specific to git | 17:41 |
| fungi | if you hover over the icon next to the upper one it'll say something like "dependent change required for testing" | 17:41 |
| fungi | the icon should be a grey "i" with a circle around it | 17:41 |
| noorul | How does this work in the case of github | 17:41 |
| corvus | noorul: 2 things -- first, you are not running the latest version of the javascript app; that error is now handled and reported as a cors error. second -- that's a cors error, so even though job-output.json.gz can be fetched, it does not seem to have the correct 'access-control-allow-origin' header. how that can be, but the manifest works, i do not understand. | 17:42 |
| clarkb | github doesn't allow for dependencies between pull requests. But if you used zuul-depends on to express those dependencies you would get the same behavior as gerrit | 17:43 |
| noorul | corvus: It has the header https://pasteboard.co/Iw24cdE.png | 17:44 |
| noorul | clarkb: I got it | 17:44 |
| noorul | clarkb: Thanks | 17:45 |
| corvus | noorul: something in the browser is causing that to fail; you might try a different browser, or disabling content blocking extensions. but only do that after upgrading the zuul js dashboard app | 17:45 |
| noorul | corvus: Zuul version: 3.10.2.dev26 745f4737 | 17:46 |
| corvus | 745f4737 is not a commit in zuul's history | 17:47 |
| noorul | corvus: I have some local changes | 17:47 |
| fungi | i'm assuming this is built from the bitbucket driver series | 17:47 |
| corvus | noorul: you should be able to rebase those on current master | 17:47 |
| noorul | 19fe0e9083a815ccc1a9617c5d5f9ad3dd21b6d4 | 17:47 |
| noorul | corvus: Is running on master safe? | 17:48 |
| corvus | noorul: yes, that's what we do | 17:48 |
| noorul | Oh cool! | 17:48 |
| noorul | Let me re-deploy | 17:48 |
| fungi | the point of zuul is to make it safe to run projects on master ;) | 17:48 |
| fungi | would be a shame not to be able to run zuul itself from master | 17:49 |
| corvus | i believe ofosos only hasn't rebased the bitbucket driver series because it isn't necessary, and rebasing a change series like that can be distracting for review. but since you're *also* doing bleeding edge javascript web app development, it is appropriate for you to rebase that in your local deployment :) | 17:49 |
| noorul | fungi: nice one | 17:49 |
| noorul | fungi: he he | 17:50 |
| noorul | This is my second attempt to bring Zuul into the company | 17:50 |
| noorul | Hopefully this time it works out | 17:50 |
| *** jamesmcarthur has quit IRC | 17:51 | |
| clarkb | noorul: out of curiousity can you say which company? | 17:54 |
| noorul | Aruba Networks (now owned by HPE) | 17:55 |
| noorul | ofosos: ping | 17:57 |
| mnaser | hi all: i've been working with my team on gathering more docs and informative content on "how gating works". i did explain it to them and they came up with this, however they'd love community feedback on this sort of thing -- https://vexxhost.com/wp-content/uploads/2019/09/howzuulworks.svg | 17:59 |
| mnaser | yes there are typos and its' rough, no.1 and no.2 would probably be check/gate | 17:59 |
| mnaser | but i'd be happy to hear if anyone thinks that there is something that could be more accurately represented (or easier for folks to understand gating) | 17:59 |
| clarkb | mnaser: the Project Gating no 1 block is not really gating necessarily. Zuul for example does not enforce clean check | 18:01 |
| clarkb | in the openstack case that is how it works though | 18:01 |
| mnaser | right, i think the idea was to try and model the fact that zuul can provide feedback to the user/dev before any code review happens | 18:01 |
| mnaser | but thats' really what any CI tool does | 18:01 |
| noorul | mnaser: I found this https://docs.openstack.org/infra/manual/developers.html useful | 18:01 |
| * noorul is running yarn build | 18:02 | |
| Shrews | mnaser: i'd be tempted to differentiate between zuul and gerrit in there, but maybe that adds more complexity than you're looking for? | 18:02 |
| clarkb | mnaser: my other peice of feedback would be that it doesn't capture the serialized queue of changes that are tested in parallel which is another important piece of zuul's gating | 18:03 |
| *** hashar has quit IRC | 18:03 | |
| mnaser | thats a good point, maybe "scm" or whatever we want to call that | 18:03 |
| mnaser | clarkb: thats true, i think at least for this we're trying to help get the gating point across with this i think, it'd be an even more complex graph then i think :p | 18:03 |
| *** jamesmcarthur has joined #zuul | 18:04 | |
| noorul | I also found it difficult to come up with a diagram that explains speculative testing | 18:04 |
| *** mattw4 has quit IRC | 18:04 | |
| clarkb | the video on the zuul website runs through it | 18:04 |
| clarkb | I think animations/video are likely the best way to capture that | 18:05 |
| clarkb | Shrews: did you want to rereview https://review.opendev.org/#/c/679670/ I addressed an english problem that jamesmcarthur pointed out | 18:05 |
| *** mattw4 has joined #zuul | 18:05 | |
| Shrews | mnaser: also https://docs.openstack.org/infra/publications/zuul/#(18) might have ideas for you | 18:06 |
| Shrews | clarkb: looking | 18:06 |
| mnaser | Shrews: neat ill share that too | 18:07 |
| *** jamesmcarthur has quit IRC | 18:07 | |
| noorul | corvus: I am still getting the same error https://pasteboard.co/Iw2edxH.png | 18:08 |
| noorul | corvus: This is in chrome incognito mode | 18:10 |
| noorul | corvus: In firefox I don't see any errors but not console tab | 18:11 |
| noorul | corvus: Nothing after results section too | 18:11 |
| corvus | noorul: the line numbers match the current source now, so that's an improvement | 18:12 |
| *** armstrongs has joined #zuul | 18:12 | |
| corvus | noorul: here's where the error happens: https://opendev.org/zuul/zuul/src/branch/master/web/src/actions/build.js#L288 | 18:13 |
| corvus | the actual exception is a coding error in our error handling -- we can and should fix that, but we can look past that for now and i'm pretty sure that what it's telling us is that there is some error in retrieving the job-output.json.gz file | 18:17 |
| corvus | but it's not a server error, it's something in the browser | 18:17 |
| noorul | corvus: It is retrieving it has binary | 18:17 |
| noorul | corvus: Why is it looking for .gz ? | 18:18 |
| noorul | logs tab has http://10.29.12.161:8000/60/60/4a8c02c1e16329db1be9a5f6bc497558b7848af8/gate/testjob/d0a6146/job-output.json and | 18:19 |
| noorul | http://10.29.12.161:9000/t/demo/build/d0a6146e99e346afb19d1d2004f527ab/log/job-output.txt | 18:19 |
| noorul | Not sure why is it looking for job-output.json.gz | 18:19 |
| paladox | corvus i'm hopping upstream will use my zuul-status plugin with zuul :) | 18:20 |
| corvus | noorul: because it tries both .json.gz and .json in case there's a compressed version. it's old, and should be changed to use the manifest, but it doesn't right now. at any rate, your server does serve a .gz file | 18:20 |
| corvus | noorul: so i think for it to work, you'll need to convince nginx to send a "content-type: application/json" and "content-encoding: gzip" headers on that file. | 18:21 |
| corvus | noorul: that's what our static log server did. see for example this file: http://logs.openstack.org/0f/0f8e5c44faec43a56d08184b5e60e5ac3e8a0234/post/publish-openstack-tox-docs/ef11dcd/job-output.json.gz | 18:21 |
| corvus | pabelanger: did you unwip that change and send a notice to zuul-announce yet? | 18:22 |
| *** jamesmcarthur has joined #zuul | 18:23 | |
| noorul | This gives me without any encoding http://10.29.12.161:9000/t/demo/build/d0a6146e99e346afb19d1d2004f527ab/log/job-output.txt.gz | 18:23 |
| noorul | and this http://10.29.12.161:9000/t/demo/build/d0a6146e99e346afb19d1d2004f527ab/log/job-output.txt gives me with encoding | 18:23 |
| corvus | noorul: no one can access that url except you because it's at an rfc1918 address | 18:23 |
| noorul | But there is only one file on the server job-output.txt.gz | 18:23 |
| noorul | nginx takes care of that | 18:24 |
| *** armstrongs has quit IRC | 18:25 | |
| corvus | noorul: here is how we configure apache to do this: http://paste.openstack.org/show/771463/ | 18:26 |
| noorul | Shouldn't that line dispatch(receiveBuildOutput(buildId, response.data) fail and fall back to non gz file? | 18:26 |
| corvus | noorul: your choices are: 1) figure out how to do the equivalent with nginx. 2) use apache and use that code we already wrote. 3) wait for https://review.opendev.org/567696 to land. 4) ignore the issue for now | 18:27 |
| corvus | noorul: no, because fetching the .gz file *succeeded* | 18:27 |
| corvus | it just got unusable data because it was expecting a gzip content encoding | 18:27 |
| *** igordc has joined #zuul | 18:36 | |
| *** mattw4 has quit IRC | 18:41 | |
| noorul | he he | 18:50 |
| noorul | I got the console | 18:50 |
| noorul | corvus clarkb pabelanger fungi Thank you all for the help | 18:55 |
| pabelanger | welcome to the fun! | 18:55 |
| noorul | off to bed | 18:55 |
| *** noorul has quit IRC | 18:55 | |
| openstackgerrit | Jimmy McArthur proposed zuul/zuul-website master: Removing erroneous og images https://review.opendev.org/680488 | 19:00 |
| openstackgerrit | Jimmy McArthur proposed zuul/zuul-website master: Replacing OG images with Zuul icon https://review.opendev.org/680490 | 19:10 |
| AJaeger | pabelanger: could you WIP https://review.opendev.org/#/c/567696/ another two weeks, please? We wait for changes at least two weeks - and you might want to do a followup to your email and give that date as information out. | 19:30 |
| *** gtema has joined #zuul | 19:30 | |
| AJaeger | corvus, clarkb, what are we going to do with 567696 in OpenDev? Follow the change to not compress - or keep status quo? Status quo means a change to our base jobs before we merge 567696 | 19:32 |
| pabelanger | AJaeger: yah, we need 2 weeks, but was asked to unWIP by corvus | 19:32 |
| clarkb | AJaeger: we use upload-logs-swift now so should be unaffected | 19:33 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Web: rely on new attributes when determining task failure https://review.opendev.org/680498 | 19:34 |
| pabelanger | dmsimard: tristanC: will want to confirm for rdo / sf / other tenants they have | 19:34 |
| pabelanger | they still use a log server | 19:34 |
| AJaeger | clarkb: Lucky us ;) | 19:34 |
| corvus | yeah, i suspect that software factory will want to set the value to keep compression. it's safe to do that now, since the option doesn't yet exist | 19:35 |
| corvus | then when the changes merges it'll be a noop | 19:35 |
| AJaeger | yep | 19:35 |
| corvus | basically, it's extra work to configure a static logserver to deal with compressed files. if you've already done it, great, but it's a lot to ask of new users | 19:36 |
| *** jamesmcarthur has quit IRC | 19:37 | |
| *** dtroyer has joined #zuul | 19:37 | |
| *** jamesmcarthur has joined #zuul | 19:38 | |
| AJaeger | agreed | 19:38 |
| *** jamesmcarthur has quit IRC | 19:42 | |
| tristanC | pabelanger: no worries, we'll apply that: https://softwarefactory-project.io/r/16176 | 19:48 |
| tristanC | corvus: well, we are looking forward using upload-logs-swift when user provides swift endpoints, but the current ara-report noise is going to be an issue... | 19:50 |
| corvus | tristanC: yeah, and i'm guessing you see even more and larger nested ara's than we do upstream | 19:52 |
| tristanC | some jobs do have multiple ara-report indeed, one for zuul, one from the undercloud, and some from other playbooks ran later down the job. so we really rely on ara-report's being one file | 19:53 |
| *** gtema has quit IRC | 20:00 | |
| openstackgerrit | Paul Belanger proposed zuul/zuul master: WIP: Support Ansible 2.9 https://review.opendev.org/674854 | 20:04 |
| pabelanger | 2.9.0b1 is out | 20:04 |
| pabelanger | for ansible | 20:04 |
| *** jamesmcarthur has joined #zuul | 20:09 | |
| pabelanger | jlk: is there a process to opt into github feature? for example triage role? | 20:31 |
| openstackgerrit | Paul Belanger proposed zuul/zuul master: Switch ansible_default to 2.8 https://review.opendev.org/676695 | 20:38 |
| openstackgerrit | Paul Belanger proposed zuul/zuul master: WIP: Support Ansible 2.9 https://review.opendev.org/674854 | 20:38 |
| *** jamesmcarthur has quit IRC | 20:48 | |
| *** jamesmcarthur has joined #zuul | 20:50 | |
| clarkb | anyone know when ansiblefest hotel confirmations will go out? | 20:54 |
| clarkb | mine is still pending | 20:54 |
| jlk | pabelanger: hrm. Public beta stuff, not sure. | 20:55 |
| pabelanger | clarkb: asking | 20:56 |
| pabelanger | clarkb: you are attendee right? | 20:56 |
| clarkb | pabelanger: yes | 20:56 |
| pabelanger | not speaker? | 20:56 |
| pabelanger | kk | 20:56 |
| clarkb | correct | 20:56 |
| jlk | I need to make those | 20:56 |
| pabelanger | clarkb: k, asking in internal chat. will reply once somebody has answer | 20:58 |
| clarkb | pabelanger: thank you! | 20:58 |
| jlk | pabelanger: so it says triage and maintain roles are public beta, maybe you just have to go to the admin page of a org to turn that feature on for the org? | 20:58 |
| pabelanger | jlk: maybe, but i can't seem to see them. But also, not really sure where to look | 20:58 |
| jlk | yeah hrm | 20:58 |
| pabelanger | looking in all the settings, I don't see any reference to them | 20:59 |
| pabelanger | which makes me thing, our account doesn't have it enabled | 20:59 |
| jlk | Foudn it sortof | 21:02 |
| jlk | In the _repo_ settings for a repository within an org | 21:02 |
| jlk | I added my bot as a collaborator in the Collaborators and Teams page | 21:02 |
| jlk | and for permission I could select Triage or Maintainer | 21:02 |
| pabelanger | Ooh | 21:03 |
| pabelanger | i see it | 21:03 |
| pabelanger | however, it doesn't seem exposed at org level | 21:03 |
| pabelanger | for teams | 21:04 |
| jlk | yeah, it might be a per-repo setting | 21:04 |
| pabelanger | but, that gets me started | 21:04 |
| pabelanger | kk | 21:04 |
| pabelanger | can build some automation around thta | 21:04 |
| fungi | jamesmcarthur: i've commented on your series of zuul-website changes, when you have a chance to take a look | 21:04 |
| pabelanger | jlk: thank you! | 21:05 |
| *** mattw4 has joined #zuul | 21:10 | |
| jlk | ah | 21:10 |
| jlk | pabelanger: you can add people to a team within the org. Then on the REPO level you can grant a team triage or whatever | 21:11 |
| jlk | the default is just read access to any public repo within an org | 21:11 |
| jamesmcarthur | fungi: on it :) | 21:11 |
| pabelanger | clarkb: sounds like after 9/9 is when block opens | 21:15 |
| pabelanger | jlk: perfect, thanks | 21:15 |
| -openstackstatus- NOTICE: Gerrit is being restarted to pick up configuration changes. Should be quick. Sorry for the interruption. | 21:16 | |
| clarkb | pabelanger: ok so people that requested a room a while back should get confirmation sometime after the 9th? | 21:16 |
| openstackgerrit | Jimmy McArthur proposed zuul/zuul-website master: OK, trying again to update with the correct og image tags, removing the erroneous gitlab tags. https://review.opendev.org/680520 | 21:16 |
| openstackgerrit | Jimmy McArthur proposed zuul/zuul-website master: OK, trying again to update with the correct og image tags, removing the erroneous gitlab tags. https://review.opendev.org/680520 | 21:19 |
| fungi | jamesmcarthur: from context on that commit message i get the impression what you actually want to do is amend https://review.opendev.org/680459 rather than creating a separate change | 21:19 |
| openstackgerrit | Jimmy McArthur proposed zuul/zuul-website master: OK, trying again to update with the correct og image tags, removing the erroneous gitlab tags. https://review.opendev.org/680520 | 21:20 |
| fungi | jamesmcarthur: https://docs.openstack.org/infra/manual/developers.html#updating-a-change might be helpful | 21:21 |
| jamesmcarthur | Yeah, I thought I was doing a follow-up commit | 21:21 |
| jamesmcarthur | I _think_ I fixed it | 21:21 |
| jamesmcarthur | this should now be the correct change: https://review.opendev.org/#/c/680520/3/www/index.html | 21:22 |
| pabelanger | clarkb: asking | 21:29 |
| pabelanger | clarkb: how long ago did you request? | 21:29 |
| *** jamesmcarthur has quit IRC | 21:29 | |
| clarkb | doesn't show up in my request, let me check other logs where I wrote stuff down | 21:30 |
| pabelanger | clarkb: and, because I don't know, this was all via the ansiblefest site right? | 21:30 |
| clarkb | yes it was via the ansiblefest site and my logs say around august 6 | 21:30 |
| fungi | i tried to request a room around then and the site told me none were available, so i just booked directly with the hotel instead | 21:32 |
| pabelanger | yah | 21:32 |
| pabelanger | so, fungi is right. If you didn't get out of rooms, you got a room | 21:32 |
| pabelanger | so, clarkb you should see something soon about the room | 21:32 |
| clarkb | ok thanks for checking | 21:32 |
| fungi | though i think mine may have been due to trying to also book thursday night | 21:32 |
| clarkb | and ya ansiblefest site shows the request its just 'pending' | 21:32 |
| pabelanger | yah, humans say should come out 2 weeks before event | 21:33 |
| fungi | (i booked to check in sunday and check out friday) | 21:33 |
| pabelanger | confirmation | 21:33 |
| clarkb | pabelanger: cool I'll keep an eye out for it in a week or two then | 21:33 |
| fungi | i've still been unable to get the event interface to show me any options for the thursday workshops though | 21:33 |
| clarkb | related to ansiblefest https://review.opendev.org/#/c/679670/ was inspired directly by some of the question jimmy was asking me while doing prep for ansiblefest | 21:34 |
| clarkb | corvus: ^ maybe you want to take a look at that? | 21:34 |
| clarkb | pabelanger: are you going to ansiblefest? | 21:36 |
| pabelanger | clarkb: yes, need to book flights today | 21:38 |
| pabelanger | also have a talk about zuul.a.c and network automation testing | 21:39 |
| clarkb | exciting | 21:40 |
| pabelanger | there is also https://etherpad.openstack.org/p/ansible-summit-atlanta-2019 for the monday | 21:41 |
| pabelanger | depending on how many zuul folks are going, we might be able to have a slot | 21:41 |
| clarkb | yup I registered for that too. I was thinking it might be a good venue to bring up the python fork per ansible task performance regression? | 21:41 |
| clarkb | making that quicker will make zuul quicker too :) | 21:42 |
| pabelanger | ++ | 21:42 |
| pabelanger | I expect it to be all things collections | 21:42 |
| pabelanger | and how to deal with everything getting moved out from ansible/ansible | 21:42 |
| clarkb | the great reorg | 21:42 |
| pabelanger | that could be a good story for zuul cross project testing | 21:42 |
| pabelanger | and maybe opendev to front end it, via gerrit | 21:42 |
| pabelanger | I'm told, they are not enforcing github | 21:43 |
| pabelanger | for community | 21:43 |
| pabelanger | which also means, we can move zuul_console / zuul_stream into zuul collection and install it properly | 21:43 |
| *** NBorg has quit IRC | 21:47 | |
| *** jamesmcarthur has joined #zuul | 21:50 | |
| *** jamesmcarthur has quit IRC | 21:55 | |
| clarkb | pabelanger: fyi http://lists.openstack.org/pipermail/openstack-discuss/2019-September/009141.html | 22:00 |
| pabelanger | clarkb: ah, yes that is true. Should fix that | 22:04 |
| pabelanger | I can reply shortly | 22:04 |
| fungi | http://lists.openstack.org/pipermail/openstack-discuss/2019-September/009141.html reminds me that we don't have much in the way of documentation on how to contribute to zuul | 22:09 |
| *** AJaeger_ has joined #zuul | 22:10 | |
| corvus | fungi: i hope we never need much in the way of documentation on how to contribute to zuul | 22:10 |
| corvus | fungi: there's a web page on the different ways to get involved: https://zuul-ci.org/community.html | 22:11 |
| fungi | d'oh! i should have linked that page | 22:11 |
| fungi | i forgot we had that content | 22:12 |
| corvus | linked? | 22:12 |
| fungi | in my reply | 22:12 |
| corvus | oh. i think maybe clarkb and pabelanger were assuming the author meant windmill. no one has said the word "windmill" until now though, so i'm really not sure. | 22:13 |
| fungi | i just replied again | 22:13 |
| fungi | aha, i missed the windmill context | 22:13 |
| corvus | just a supposition on my part | 22:13 |
| clarkb | ya I assumed windmill | 22:13 |
| *** AJaeger has quit IRC | 22:14 | |
| fungi | i assumed they meant the zuul-jobs repo | 22:14 |
| fungi | i guess that's the problem with using ansible both within and to deploy zuul | 22:14 |
| fungi | hard to know which someone means without clarification ;) | 22:14 |
| clarkb | ya the context for me was zuul AND nodepool | 22:14 |
| clarkb | which to me implied the things to deploy them both (since nodepool doesn't have roles otherwise) | 22:15 |
| corvus | i only got there because i wondered why clarkb only pinged pabelanger. still took me like 2 minutes to figure that out. how clarkb deduced that is impressive :) | 22:15 |
| fungi | i figured they simply didn't know that nodepool doesn't use ansible | 22:15 |
| pabelanger | yah, I assumed roles for windmill too | 22:16 |
| *** jamesmcarthur has joined #zuul | 22:16 | |
| corvus | clarkb: we need to return the zuul website preview build as an artifact | 22:16 |
| clarkb | corvus: it works to click through to the logs and go into the html fwiw | 22:16 |
| pabelanger | I'll follow up about zuul-ci docs too | 22:16 |
| clarkb | but ya artifact would make it more prominent on the main dashboard | 22:16 |
| corvus | clarkb: yeah, just so much clicky | 22:16 |
| clarkb | if the faq change looks good I'll push a followup to add pagure but if there are things I need to change I'll incorporate into a new patchset | 22:17 |
| corvus | clarkb: the pagure thing is the only -1 worthy comment i have | 22:19 |
| clarkb | k I'll push up a followup change now then | 22:19 |
| clarkb | corvus: are you ok with a followup or would you prefer a new ps? | 22:20 |
| corvus | clarkb: if you push a new ps i'll carryover the +2 and +3 it | 22:20 |
| clarkb | k | 22:21 |
| *** jamesmcarthur has quit IRC | 22:21 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Add Zuul FAQ page https://review.opendev.org/679670 | 22:22 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: CSS fix for ul/li in FAQ https://review.opendev.org/680465 | 22:23 |
| clarkb | rebased jimmy's change too | 22:23 |
| *** jamesmcarthur has joined #zuul | 22:25 | |
| *** jamesmcarthur has quit IRC | 22:30 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Flip the order of the emit-job-header tests https://review.opendev.org/680477 | 22:37 |
| *** igordc has quit IRC | 22:48 | |
| clarkb | corvus: https://review.opendev.org/#/c/679670/ is +1 from zuul now after new ps if you want to +3 | 23:05 |
| pabelanger | re: ara-report removal from base jobs. Are we looking to do zuul release for web fixes? | 23:06 |
| pabelanger | Hmm | 23:17 |
| corvus | pabelanger: in general, i don't see why not. the fix for that was both a web and executor fix. it's also not critical -- it mostly worked beforehand. | 23:17 |
| pabelanger | is install-docker role broken? https://zuul.opendev.org/t/zuul/build/856582ac29484063883ce9dbf0c9112b/log/job-output.txt#283 | 23:17 |
| pabelanger | corvus: okay cool, also happy to make the change to help swift preformance | 23:18 |
| pabelanger | oh | 23:18 |
| pabelanger | https://zuul.opendev.org/t/zuul/build/856582ac29484063883ce9dbf0c9112b/log/job-output.txt#249 | 23:18 |
| pabelanger | mirror issue with rackspace | 23:18 |
| clarkb | corvus: we may want a release for the fix of zuul not testing the correct commit in some corner cases | 23:18 |
| corvus | hrm | 23:19 |
| corvus | i don't see the final task in the pre playbook on https://zuul.opendev.org/t/zuul/build/856582ac29484063883ce9dbf0c9112b/console | 23:20 |
| corvus | pabelanger linked to https://zuul.opendev.org/t/zuul/build/856582ac29484063883ce9dbf0c9112b/log/job-output.txt#249 | 23:20 |
| corvus | even weirder https://zuul.opendev.org/t/zuul/build/856582ac29484063883ce9dbf0c9112b/console#0/3/20/ubuntu-bionic has the output for the final task | 23:21 |
| pabelanger | also, I would have expected that task to fail | 23:22 |
| pabelanger | but returned okay | 23:22 |
| clarkb | so the data is there but not rendered? | 23:22 |
| pabelanger | maybe, because apt says a warning | 23:22 |
| pabelanger | yah, I don't see configure-mirrors : Update apt cache in json file | 23:23 |
| corvus | it won't have the " configure-mirrors :" part | 23:24 |
| corvus | that's 2 fields concat together in the web ui | 23:24 |
| corvus | but "Update apt cache in json file" should certainly be greppable | 23:24 |
| pabelanger | that isn't but I do see task now | 23:25 |
| pabelanger | well, maybe not | 23:25 |
| corvus | i'm trying to figure out if all the tasks are shifted or something | 23:25 |
| pabelanger | Install Ubuntu repository files looks to be | 23:26 |
| corvus | it's not shifted -- all the configure-mirror tasks before that are correct. it's just that the last 2 real tasks got combined into a single task in the json file, with the output of the last task and the name of the previous task | 23:29 |
| corvus | that is really weird | 23:29 |
| *** threestrands has joined #zuul | 23:30 | |
| corvus | this holds true for other builds too | 23:30 |
| corvus | now, is it only that playbook, or others as well? | 23:30 |
| corvus | the other playbooks in that build look fine | 23:31 |
| corvus | this is a strange bug; i'll have to dig into it tomorrow | 23:39 |
| ianw | (from prior discussion in #openstack-infra) ... zuul is not picking up https://github.com/philpep/testinfra/pull/494, no doubt related to the config error @ http://zuul.openstack.org/config-errors | 23:52 |
| ianw | but it *has* worked, the last one was 5 days ago @ http://zuul.openstack.org/config-errors | 23:52 |
| ianw | bah, https://github.com/philpep/testinfra/pull/491#issuecomment-526821697 | 23:52 |
| *** jamesmcarthur has joined #zuul | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
