| flaper87 | SpamapS: roger that! I think I'll follow your approach | 00:08 |
|---|---|---|
| *** rlandy|ruck is now known as rlandy|ruck|bbl | 00:11 | |
| *** stewie925 has joined #zuul | 00:14 | |
| *** threestrands has joined #zuul | 00:24 | |
| SpamapS | Shrews: Oh, btw, I agree with you that the Dockerfile probably isn't an API to be carefully adjusted. I just wanted to bring up that it broke us and that I'm still looking for a way to run Zuul and Nodepool without every upgrade being a slog through dependencies, release notes, and trial&error. Not complaining.. just asking for help in figuring out how I can follow closely without feeling like I"m on a | 00:49 |
| SpamapS | treadmill. | 00:49 |
| fungi | sometimes i'm tempted to take up goat farming | 00:53 |
| * fungi sympathizes | 00:53 | |
| SpamapS | Every time I rotate API keys on our 30+ 3rd party SaaS API's I finish by checking real estate prices in the Highlands of Scotland. | 00:54 |
| Shrews | SpamapS: i have developed more thoughts around that | 00:54 |
| SpamapS | "Maybe we can just buy a plot of grass and watch the sheep roam." | 00:54 |
| *** michael-beaver has quit IRC | 00:55 | |
| Shrews | I think if the Dockerfile that we've created for our testing/demonstration purposes is going to be promoted for others to use in their systems (and we develop a promise around that to not break you) then it should probably be separated into a total "zuul docker-based packaging" unit, separate from nodepool, that can then be tested in various ways we are not yet doing. | 00:56 |
| SpamapS | Yeah, I really am hesitant to make any changes to what I'm doing until the Kubernetes operator happens. I may even find some cycles for working on that next month. | 00:57 |
| Shrews | SpamapS: because I don't think we've ever suggested others to use that (unless I've missed that) | 00:57 |
| SpamapS | Shrews:to be clear, this breaks me *with the official images* | 00:58 |
| SpamapS | I don't use the Dockerfile, I just pull and push the nodepool images into my local registry. | 00:58 |
| Shrews | SpamapS: ah | 00:58 |
| SpamapS | The default user changed, and thus, my k8s objects must be adjusted. | 00:58 |
| SpamapS | I think the thing that's going wrong, is that we're not being explicit about what the support level is for those docker images. | 00:59 |
| SpamapS | They're damned useful for the quickstart. | 00:59 |
| Shrews | Yeah | 00:59 |
| Shrews | If we are going to be explicit about a higher support level, then yeah... it needs to be a real packaging thing. But we've explicitly tried to stay out of that area. But seems we still have a foot lingering in it... | 01:00 |
| SpamapS | IMO it will make a lot more sense when there's an operator, and we can test the whole thing in minikube in the gate. | 01:00 |
| Shrews | The operator probably makes all this moot | 01:00 |
| Shrews | yeah, what you said | 01:00 |
| Shrews | SpamapS: in the meantime, i'll poke you if I see any more Dockerfile changes put up | 01:01 |
| Shrews | with a sharp, pointy stick | 01:02 |
| fungi | and now i have kraftwerk's "pocket calculator" stuck in my head | 01:02 |
| fungi | thanks | 01:02 |
| fungi | (i'm the operator with my pocket calculator) | 01:02 |
| Shrews | you kids and your new fangled music | 01:03 |
| * fungi shakes his cane at Shrews | 01:03 | |
| SpamapS | Shrews:mighty kind of you | 01:05 |
| SpamapS | I think the real problem is just that I've charged off into the bold unknown and now I'm annoyed that they don't have wifi or ice cubes. | 01:05 |
| Shrews | SpamapS: they do have b****face at the bar | 01:06 |
| * Shrews grabs 2 straws | 01:06 | |
| SpamapS | damn straight | 01:06 |
| *** calebb has quit IRC | 01:25 | |
| clarkb | zuulians I've been talking with stewie925 over in #openstack-qa about a weird thing opendev.org/x/ranger-agent has noticed for jobs against https://review.opendev.org/#/c/677258/5 that change is proposed against the python3 branch but some jobs check out master instead of python3. This leads them to testing the wrong commit | 01:29 |
| clarkb | What I've found is that on the executors that run the jobs against master there is no python3 branch in the /var/lib/zuul/executor-git repo for that project | 01:30 |
| clarkb | the jobs that run on executors with a python3 branch succeed | 01:30 |
| clarkb | Initially I thought maybe a problem with our opendev replication but thost executor git repos all have remotes for review.opendev.org which is the canonical source (gerrit) | 01:31 |
| *** bhavikdbavishi has joined #zuul | 01:31 | |
| fungi | any idea how long ago that branch was created? | 01:32 |
| clarkb | I don't know | 01:33 |
| clarkb | it was well before these jobs ran because an earlier patchset was pushed on the 19th | 01:33 |
| fungi | just wondering if we could find logs from when the executors should have fetched that branch | 01:34 |
| clarkb | well they should do an update on every job run | 01:35 |
| *** bhavikdbavishi1 has joined #zuul | 01:35 | |
| *** bhavikdbavishi has quit IRC | 01:35 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 01:35 | |
| fungi | can probably take a snapshot of the git tree from an affected executor and then try a remote update in it | 01:36 |
| clarkb | 2019-08-26 16:35:41,406 INFO zuul.Merger: [e: 9e01fe2c653a4c7ea9d121e41c2ed5f7] [build: b405f247cf3a441b924bce4c35b2f635] Skipping updating local repository gerrit/x/ranger-agent | 01:39 |
| clarkb | that may be the problem "skipping" | 01:39 |
| fungi | huh... why ever for, zuul? | 01:39 |
| clarkb | if I'm reading the code correctly bceause the rev/commit already exists in the repo | 01:40 |
| fungi | oh! so it's a commit present on master too? | 01:40 |
| clarkb | no I think this may be more subtle the commit (e5ef4f738e6d916103c178aedb65e6aa7d2e9dfd) does not exist in master or on ze05's repo | 01:41 |
| fungi | wait, that doesn't make sense | 01:41 |
| fungi | change pushed for a nonexistent parent? | 01:42 |
| clarkb | no the commit is proposed against python3 and has not merged yet | 01:42 |
| clarkb | so does not exist anywhere near master | 01:42 |
| fungi | and the parent is the tip of that branch? | 01:43 |
| clarkb | yes | 01:43 |
| clarkb | which happens to also be master | 01:43 |
| fungi | er | 01:43 |
| fungi | okay, so the master and python3 branches have the same head commit currently | 01:43 |
| fungi | yeah, that's probably a corner case we didn't think about but will happen any time a branch is created from the tip of another existing branch i guess | 01:44 |
| clarkb | what is odd is some executors have updated | 01:44 |
| clarkb | but ya I guess if we check the parent commit then we could think the branch is there and not update | 01:44 |
| clarkb | so the needs update check should also check if the ref as well as the rev is present | 01:45 |
| fungi | the difference in result is merely cosmetic though, right? because it's testing the same git history in either case? | 01:45 |
| clarkb | no it isn't that is the problem | 01:46 |
| clarkb | it is testing a checkout of master which is the parent commit | 01:46 |
| fungi | oh, but i guess for cross-repo dependencies it'll compare the wrong refname | 01:46 |
| clarkb | this is for the repo under test specifically | 01:46 |
| clarkb | on the test node it checks out master | 01:46 |
| fungi | oh, it doesn't actually checkout the change? | 01:46 |
| clarkb | master has the wrong ref because we pushed the correct ref to python3 | 01:46 |
| clarkb | correct it checks out master | 01:46 |
| fungi | yeah that's broken then | 01:46 |
| fungi | so a fallback behvior gone awry | 01:47 |
| fungi | i guess ff merges between branches could also result in this scenario | 01:48 |
| fungi | maybe that's the difference? the branch appeared on some mergers when there was an outstanding parent which had not merged to the python3 branch and had not then been subsequently merged from python3 to master? | 01:49 |
| fungi | er i mean which had merged to the python3 branch | 01:49 |
| clarkb | possible those other executors had never handled jobs for that repo too | 01:50 |
| clarkb | and so when they get one they started from scratch | 01:51 |
| clarkb | executors that had handled the repo in the past did the skip? | 01:51 |
| fungi | so python3 branch created from tip of master, changes a,b proposed in series to python3, change a merges and now python3 and master differ, change b is tested and the executor fetches the (differing) python3 branch | 01:51 |
| fungi | later python3 is ff merged into master | 01:51 |
| fungi | now the branch tips are equivalent again | 01:51 |
| fungi | ahh, yeah, executor cloning repo for the first time could also explain it | 01:52 |
| fungi | or executor's merger handling a cat job on scheduler restart? | 01:52 |
| openstackgerrit | Clark Boylan proposed zuul/zuul master: This is a stub for a bug to help us find and fix it https://review.opendev.org/678710 | 01:53 |
| clarkb | fungi: ya I think a cat on restart might do it too | 01:53 |
| clarkb | in any case I think ^ that calls out the code that needs updating. What I've written is more psuedo code than anything becase it is late and I don't understand the datastructures that are currently passed in | 01:53 |
| clarkb | chances are we need to pass more data to do this check properly | 01:53 |
| clarkb | I was just handed a plate of dinner so eating that next | 01:54 |
| clarkb | then probably won't get to this until tomorrow | 01:54 |
| *** stewie925 has quit IRC | 02:16 | |
| *** rlandy|ruck|bbl is now known as rlandy|ruck | 02:29 | |
| *** rlandy|ruck has quit IRC | 02:29 | |
| *** noorul has joined #zuul | 02:35 | |
| *** dolpher has quit IRC | 02:36 | |
| *** noorul has quit IRC | 02:52 | |
| *** rfolco has quit IRC | 04:03 | |
| *** raukadah is now known as chkumar|rover | 04:26 | |
| * SpamapS kinda needs a whiteboard to understand that conversation | 04:34 | |
| AJaeger | zuul-maint, do you know why target_dir is not passed on from https://opendev.org/openstack/project-config/src/branch/master/playbooks/static-docs/promote.yaml#L40 to https://opendev.org/openstack/project-config/src/branch/master/playbooks/static-docs/promote.yaml#L55 ? See https://zuul.opendev.org/t/openstack/build/6e42137688154d8fa6759ec5dcf03651/console which failed with | 04:52 |
| AJaeger | https://zuul.opendev.org/t/openstack/build/6e42137688154d8fa6759ec5dcf03651/console#1/1/0/static.openstack.org | 04:52 |
| *** swest has joined #zuul | 05:10 | |
| *** swest has quit IRC | 05:16 | |
| *** swest has joined #zuul | 05:17 | |
| *** sanjayu_ has joined #zuul | 05:42 | |
| AJaeger | ianw found a solution and explained it to me on #openstack-infra - thanks! | 05:54 |
| *** badboy has joined #zuul | 06:53 | |
| badboy | hi | 06:54 |
| mordred | Shrews, SpamapS: the dockerfiles and containers produced from them are totally intended for production usage - so I think that breakage indicates an area where our testing is insufficient and needs to be improved (sorry SpamapS) | 07:00 |
| *** fdegir has quit IRC | 07:07 | |
| *** fdegir has joined #zuul | 07:08 | |
| *** gtema_ has joined #zuul | 07:28 | |
| *** threestrands has quit IRC | 07:32 | |
| *** jpena|off is now known as jpena | 07:40 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Make json outfile seeking clearer https://review.opendev.org/678740 | 07:43 |
| *** panda has quit IRC | 08:02 | |
| *** panda has joined #zuul | 08:02 | |
| badboy | how can I filter vote-deleted event based on the content? | 08:22 |
| *** jangutter has quit IRC | 08:37 | |
| *** badboy has quit IRC | 08:48 | |
| *** badboy has joined #zuul | 09:00 | |
| *** jangutter has joined #zuul | 09:01 | |
| *** zbr has joined #zuul | 09:12 | |
| badboy | tristanC: as I understand Zuul only accepts the following events: https://opendev.org/zuul/zuul/src/branch/master/zuul/driver/gerrit/gerritconnection.py#L119 | 09:16 |
| badboy | tristanC: so vote-deleted is not one of them | 09:16 |
| *** hashar has joined #zuul | 09:39 | |
| *** sshnaidm is now known as sshnaidm|afk | 09:41 | |
| *** bjackman has joined #zuul | 09:42 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Zuul Web: add /api/user/authorizations endpoint https://review.opendev.org/641099 | 09:48 |
| *** mhu has joined #zuul | 09:56 | |
| zbr | does anyone have a CLI script that I can use to query status of zuul jobs on current change? like listing jobs and their status: queued, running, finished. | 10:02 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Write errors from ansible execution into json log https://review.opendev.org/676723 | 10:02 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: A Zuul reporter for Elasticsearch https://review.opendev.org/644927 | 10:09 |
| *** jpena is now known as jpena|lunch | 11:30 | |
| openstackgerrit | Merged zuul/zuul master: Apply changes to command module from ansible 2.6 https://review.opendev.org/678594 | 11:42 |
| AJaeger | zuul-maint, everybody, when and how to move to ansible 2.7 for OpenDev? See https://review.opendev.org/676697 | 11:43 |
| tobiash | AJaeger: 2.7 is the default in zuul, is that a preparation to switch the default to 2.8? | 11:44 |
| *** sanjayu_ has quit IRC | 11:46 | |
| AJaeger | let'S ask pabelanger , we switched Zuul tenant already | 11:49 |
| *** AJaeger is now known as AJaeger_ | 11:50 | |
| openstackgerrit | Benedikt Löffler proposed zuul/zuul master: Report retried builds in a build set via mqtt. https://review.opendev.org/632727 | 11:51 |
| openstackgerrit | Benedikt Löffler proposed zuul/zuul master: Report retried builds via sql reporter. https://review.opendev.org/633501 | 11:51 |
| *** gtema_ has quit IRC | 11:52 | |
| *** gtema has joined #zuul | 11:53 | |
| *** themr0c has joined #zuul | 11:55 | |
| *** themroc has quit IRC | 11:55 | |
| *** gtema has quit IRC | 11:59 | |
| *** sshnaidm|afk is now known as sshnaidm | 11:59 | |
| *** badboy has quit IRC | 12:15 | |
| *** rfolco has joined #zuul | 12:22 | |
| *** rlandy has joined #zuul | 12:28 | |
| *** rlandy is now known as rlandy|ruck | 12:29 | |
| *** jpena|lunch is now known as jpena | 12:32 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Zuul Web: add /api/user/authorizations endpoint https://review.opendev.org/641099 | 12:43 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Add reference pipelines file for Gerrit driver https://review.opendev.org/672683 | 12:48 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Add reference pipelines file for Github driver https://review.opendev.org/672712 | 12:57 |
| *** bjackman has quit IRC | 12:59 | |
| *** yolanda has quit IRC | 13:06 | |
| *** yolanda__ has joined #zuul | 13:06 | |
| fbo | hi zuul-maint; who should I ping to ask for review on the Elasticsearch reporter https://review.opendev.org/644927 ? I think the review is ready and seems fine to be merged also we use this reporter w/o issue since some months. | 13:07 |
| tobiash | fbo: pinging zuul-maint is correct ;) | 13:08 |
| *** bjackman has joined #zuul | 13:09 | |
| fbo | tobiash: so that's perfect :) | 13:09 |
| pabelanger | tobiash: AJaeger_: yah, I wanted to flip the switch to ansible 2.8 before 2.9 was released. So we we're 2 releases back. | 13:23 |
| pabelanger | but need to fix up stack at https://review.opendev.org/674854/ first | 13:23 |
| *** bhavikdbavishi has quit IRC | 13:29 | |
| *** bhavikdbavishi has joined #zuul | 13:29 | |
| *** michael-beaver has joined #zuul | 13:55 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: webtrigger: add web route and rpclistener https://review.opendev.org/554839 | 13:56 |
| *** jeliu_ has joined #zuul | 14:02 | |
| *** jeliu_ has quit IRC | 14:02 | |
| *** jeliu_ has joined #zuul | 14:03 | |
| openstackgerrit | Merged zuul/zuul master: Write json log file by seeking and appending https://review.opendev.org/676717 | 14:12 |
| openstackgerrit | Benedikt Löffler proposed zuul/zuul master: Report retried builds in a build set via mqtt. https://review.opendev.org/632727 | 14:16 |
| openstackgerrit | Benedikt Löffler proposed zuul/zuul master: Report retried builds via sql reporter. https://review.opendev.org/633501 | 14:16 |
| Shrews | mhu: oh hi! I could use your help on something. My changes in https://review.opendev.org/#/q/topic:autohold-revamp introduce two new zuul CLI commands: autohold-delete and autohold-info. I'm not very familiar with cherrypy and wondering if you could help in integrating those? Or guide me in what needs to be done. | 14:26 |
| *** sshnaidm_ has joined #zuul | 14:29 | |
| *** sshnaidm has quit IRC | 14:30 | |
| *** sshnaidm__ has joined #zuul | 14:34 | |
| *** spsurya has joined #zuul | 14:35 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Write errors from ansible execution into json log https://review.opendev.org/676723 | 14:35 |
| *** sshnaidm_ has quit IRC | 14:36 | |
| *** sshnaidm__ is now known as sshnaidm | 14:51 | |
| clarkb | corvus: did you see the notes about zuul running tests against the wrong branch in scrollback? | 15:01 |
| clarkb | long story short it seems that zuul may skip repo updates when new branches are created and that results in zuul falling back to a default branch of master | 15:01 |
| mhu | Shrews, happy to help if I can! I'll have a look at your changes | 15:02 |
| corvus | clarkb: i did not see that; i'm pretty sure we thought of that and we're under the impression that works and is tested, so it will be interesting to see what we missed | 15:02 |
| clarkb | corvus: I think it may happen beacuse we check if a commit is present and not necessarily that the refs re present | 15:03 |
| clarkb | then later we check if the branch is in the repo that we did't update because the commit was there? | 15:03 |
| clarkb | I think we need to check for refs and commits before deciding if an update is appropriate | 15:03 |
| *** jamesmcarthur has joined #zuul | 15:05 | |
| Shrews | mhu: Thanks! I started to get confused as to the best approach when I looked at ZuulWebAPI.autohold(). | 15:06 |
| corvus | clarkb: oh, i read scrollback -- that was a later efficiency change that tobiash came up with -- I9364e438d581b068fa19c9dfc24adab60479c385 | 15:06 |
| corvus | clarkb: that's got a test we can probably adapt | 15:07 |
| corvus | clarkb: i just left a comment on that change -- can you take a look and see if that's the sort of thing you had in mind? | 15:08 |
| clarkb | yup I even pushed a psuedocode change that "does" that | 15:09 |
| clarkb | because it was late and wanted a pointer for the future | 15:09 |
| corvus | clarkb: cool, i think that's the right approach | 15:09 |
| corvus | at least, it sounds good to my tired braid | 15:09 |
| corvus | brain | 15:09 |
| clarkb | tobiash: ^ if you hvae a moment can you look at that too since you wrote that optimization | 15:10 |
| tobiash | clarkb, corvus: I'm currently in fire fighting mode in my own deployment, is it ok if I look in about two hours? | 15:11 |
| clarkb | tobiash: yes, no rush. | 15:12 |
| tobiash | clarkb: is there a tldr so I don't have to read the whole scrollback? | 15:13 |
| clarkb | tobiash: basically in https://review.opendev.org/#/c/648229/6/zuul/merger/merger.py isUpdateNeeded only checks if commits are present and not refs. So when a new branch is created with a shared commit of an existing branch we dno't update and pull in that new branch. Later this causes zuul to fall back to the default branch if the change was proposed to a new branch | 15:14 |
| tobiash | clarkb: thanks, I'll check later at home | 15:15 |
| tobiash | however my understanding of the repo state was that it should be complete and re-creates every ref | 15:16 |
| tobiash | is that understanding wrong? | 15:16 |
| fungi | it was the case until that mechanism was changed to skip updating when the ref is already present | 15:16 |
| clarkb | I haven't gotten that far yet. Mostly identifying behavior based on logs so far. I will go through and trace that after meetings and try to understand the specific cause | 15:16 |
| fungi | er, when the commit is already present | 15:16 |
| tobiash | so this check should check if every commit referenced by the refs to be created by the repo state is present (so it can be recreated) | 15:17 |
| tobiash | if not, it updates the repo, if everything is there update is not needed | 15:17 |
| tobiash | that was the idea | 15:17 |
| *** mattw4 has joined #zuul | 15:17 | |
| clarkb | and that every ref is present | 15:17 |
| fungi | problem is the commit can be there because it was present on another branch | 15:17 |
| tobiash | because restoring the repo state throws all refs away and recreates them with their target commits (which has been verified to exist) | 15:18 |
| fungi | when the commit is present on the default branch and the target branch, we're seeing it not fetch the target branch ref and it logs "skipping updating local repository..." | 15:21 |
| fungi | (so hitting line 620 there) | 15:21 |
| *** mattw4 has quit IRC | 15:22 | |
| *** themr0c has quit IRC | 15:29 | |
| *** chkumar|rover is now known as raukadah | 15:31 | |
| openstackgerrit | Merged zuul/zuul master: Improve visualization retry attempts in UI https://review.opendev.org/678147 | 15:32 |
| *** jangutter has quit IRC | 15:46 | |
| *** sanjayu_ has joined #zuul | 15:51 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed zuul/zuul-jobs master: Add tests for manifest generation for missing files https://review.opendev.org/678573 | 15:52 |
| *** sshnaidm is now known as sshnaidm|afk | 15:54 | |
| *** jpena is now known as jpena|off | 16:04 | |
| *** noorul has joined #zuul | 16:06 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul master: Check refs and revs for repo needing updates https://review.opendev.org/678895 | 16:09 |
| clarkb | corvus: fungi tobiash ^ I think something like that is much closer to the fix we want. Now to figure out testing for it | 16:09 |
| clarkb | (my psuedocode is now real code that may or may not work) | 16:09 |
| *** noorul has quit IRC | 16:22 | |
| *** jamesmcarthur has quit IRC | 16:22 | |
| *** jamesmcarthur has joined #zuul | 16:27 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul master: Check refs and revs for repo needing updates https://review.opendev.org/678895 | 16:40 |
| clarkb | corvus: tobiash fungi ^ I think that is ready for review now | 16:40 |
| *** amotoki is now known as amotoki_ | 16:42 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed zuul/zuul-jobs master: Add tests for manifest generation for missing files https://review.opendev.org/678573 | 16:44 |
| clarkb | Can someone explain to me how to manage zuul/web/static/.keep properly? | 16:44 |
| clarkb | seems like every time I push a change that gets modified | 16:44 |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: Add PerconaXDB Cluster to Zuul-Operator https://review.opendev.org/677315 | 16:44 |
| clarkb | maybe I should stop using git commit -a and instead explicitly git add files? | 16:44 |
| tobiash | clarkb: this needs fixing, then the keep file should be no problem anymore: https://review.opendev.org/663108 | 16:45 |
| tobiash | clarkb: what I don't understand on the ref fix is, why isn't that branch part of the repo state? | 16:46 |
| clarkb | tobiash: it is part of the repo state but its rev value matches master so the rev commit is present | 16:46 |
| tobiash | from my understanding the repo state should be complete and contain every ref needed for the job in that repo | 16:46 |
| clarkb | we have refs/origin/master: '12345' and refs/origin/python3: '12345' beacuse the rev value matches master in the repo we see that as not needing an update | 16:47 |
| clarkb | then the executor local git repo does not update | 16:47 |
| *** jamesmcarthur has quit IRC | 16:47 | |
| fungi | it contains every commit and every named ref, but apparently not necessarily every associated branch ref | 16:47 |
| tobiash | but why do we need an update in this case? | 16:47 |
| tobiash | we're restoring every ref from the repo state | 16:48 |
| clarkb | tobiash: because later zuul checks if the change's proposed branch is a valid branch in the local repo and beacuse we did not update python3 does not exist in the local repo | 16:48 |
| clarkb | then it falls back to the default branch (master) | 16:48 |
| tobiash | is that before we restore the repo state then? | 16:48 |
| tobiash | because after restoring it the branch should be there | 16:48 |
| fungi | which routine does the "restoring"? | 16:49 |
| tobiash | https://opendev.org/zuul/zuul/src/branch/master/zuul/merger/merger.py#L679 | 16:49 |
| tobiash | _restoreRepoState | 16:50 |
| tobiash | it deletes all refs and sets all new | 16:50 |
| clarkb | tobiash: https://opendev.org/zuul/zuul/src/branch/master/zuul/executor/server.py#L1158-L1160 is where we select the default branch | 16:50 |
| tobiash | and the check before is designed to ensure that this works | 16:50 |
| clarkb | tobiash: they are different repos | 16:52 |
| clarkb | resolve branch is looking in executor git, and your restore state is in the build git repo | 16:53 |
| clarkb | and yes self.project_info is populated before doMergeChanges which does the state restore | 16:54 |
| clarkb | (so even if it were the same repo the ordering is wrong) | 16:54 |
| clarkb | I think this ensures both sets of repos will be correct then we don't have to worry about strict ordering so much | 16:55 |
| tobiash | ah, got it | 16:57 |
| openstackgerrit | Clark Boylan proposed zuul/zuul master: Check refs and revs for repo needing updates https://review.opendev.org/678895 | 16:57 |
| clarkb | fixed the .keep file so unittests can run | 16:57 |
| tobiash | resolvebranch only resolved the branch name in the other repo and the correct revision is done later | 16:57 |
| tobiash | actually this might even fix some weird issue which one of my users reported | 16:59 |
| tobiash | he complained that zuul did use a wrong commit somewhere but I hadn't time for an analysis yet | 16:59 |
| tobiash | It's very likely that this is the issue | 16:59 |
| tobiash | clarkb: fix lgtm, thanks! | 17:01 |
| clarkb | thank you! | 17:01 |
| Shrews | mhu: thx for the hints/feedback. I'll take a stab at making those changes and ping you if I need more help. | 17:08 |
| *** jamesmcarthur has joined #zuul | 17:27 | |
| clarkb | ianw: I'm reading corvus' comments on https://review.opendev.org/#/c/678049/ to imply he would prefer we restart the log streamer instead of killing the executor entirely. I think I'll hold off on incorporating any further code changes until we have agreement on the best high level approach there. But I do think it is implementing the pipe read on both sides (and maybe as a followup look at multiprocessing) | 17:30 |
| clarkb | jeliu_: are you using the install-kubernetes role from zuul-jobs in your operator testing? | 17:31 |
| clarkb | jeliu_: if so have you run into any problems with dns resolution? I'm curious if https://review.opendev.org/#/c/677787/ would be helpful (based on other people's jobs I think it will be but you are probably doing the most with that currently) | 17:31 |
| *** hashar has quit IRC | 17:32 | |
| *** jamesmcarthur has quit IRC | 17:52 | |
| *** bhavikdbavishi has quit IRC | 17:52 | |
| *** bhavikdbavishi has joined #zuul | 17:54 | |
| *** bhavikdbavishi has quit IRC | 18:13 | |
| jeliu_ | clarkb: yes I am using the role. And we haven't ran into dns problems that I know of, the only networking problem we had before was related to firewall | 18:27 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: webtrigger: add web route and rpclistener https://review.opendev.org/554839 | 18:28 |
| *** mhu has quit IRC | 18:30 | |
| *** bjackman has quit IRC | 18:43 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Revert "Revert "Create zuul/web/static on demand"" https://review.opendev.org/663108 | 19:02 |
| *** jamesmcarthur has joined #zuul | 19:03 | |
| *** michael-beaver has quit IRC | 19:04 | |
| *** sanjayu_ has quit IRC | 19:06 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: web: fix connections list github base_url https://review.opendev.org/678950 | 19:47 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Update heuristing of parallel starting builds. https://review.opendev.org/671702 | 20:07 |
| *** weshay is now known as weshay_pto | 20:11 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: web: test trailing slash are removed from renderTree https://review.opendev.org/676824 | 20:34 |
| *** wxy-xiyuan has quit IRC | 20:45 | |
| *** jamesmcarthur has quit IRC | 20:46 | |
| pabelanger | how would I add datetime to builds page? eg: https://dashboard.zuul.ansible.com/t/ansible/buildsets?result=MERGER_FAILURE doesn't say when they happened, which is helpful to know | 21:02 |
| tristanC | pabelanger: perhaps using https://review.opendev.org/#/c/677466/ ? | 21:02 |
| pabelanger | \o/ | 21:03 |
| pabelanger | yah, we are seeing a lot of MERGER failures on github, and want to see if they are all related. By theory is some race condition on github side for when event is sent out, and the ref on remote side is created | 21:04 |
| tristanC | we saw a few recently too, hence the wip patch, though it was for an opendev repo | 21:05 |
| tristanC | which hasn't been updated for a long time, and for some reason the merger job failed once on each salled content | 21:06 |
| *** spsurya has quit IRC | 21:07 | |
| pabelanger | that's about the only issue we see now with github, all other improvements from tobiash have been awesome | 21:07 |
| openstackgerrit | Merged zuul/zuul master: web: extract pure functions from the TaskOutput component https://review.opendev.org/675460 | 21:18 |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: Add PerconaXDB Cluster to Zuul-Operator https://review.opendev.org/677315 | 21:20 |
| openstackgerrit | Merged zuul/nodepool master: Functional testing: add journal-to-console element https://review.opendev.org/669787 | 21:32 |
| openstackgerrit | Merged zuul/zuul master: github: properly exist when failing to get a pull request https://review.opendev.org/676948 | 21:33 |
| openstackgerrit | Merged zuul/zuul master: Hide "root" variable in job web page https://review.opendev.org/675461 | 21:47 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: web: add build button to trigger job https://review.opendev.org/635716 | 21:54 |
| clarkb | ianw corvus had some thoughts on https://review.opendev.org/#/c/678049/ if you want to check them | 21:59 |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: Add PerconaXDB Cluster to Zuul-Operator https://review.opendev.org/677315 | 22:08 |
| clarkb | jeliu_: I think as soon as we try to run workload in the pods that pulls external resources (from say github or gerrit etc) we may need that cahnge | 22:09 |
| clarkb | jeliu_: do you know if you are doing anything like that already (would be useful to know if minikube somehow solves this problem when kubeadm doesn't) | 22:09 |
| jeliu_ | I currently do have external resources in my change from github (will try to remove these dependencies later on) and it seems to be working fine. Is the use case that your patch fixes for minikube or kubeadm? | 22:19 |
| jeliu_ | clarkb: going offline soon feel free to contact me through email (jeliu@redhat.com) | 22:19 |
| clarkb | jeliu_: mnaser ran into problems with dns resolution using kubeadm. Reviewing our use of minikube I beleive we have the same flaw | 22:20 |
| clarkb | but should only be affected if the pods themselves make dns requests for things on the internet | 22:21 |
| *** jeliu_ has quit IRC | 22:25 | |
| *** armstrongs has joined #zuul | 22:33 | |
| *** armstrongs has quit IRC | 22:42 | |
| ianw | clarkb: yeah, i don't have strong opinions; as you say i think systemd restarting the service, or k8s or whatever is, for mine, is just an easier to reason about behaviour | 23:07 |
| ianw | i think moreso though, if we had a test that would be awesome | 23:08 |
| ianw | and then i'd feel more comfortable about a bigger change like jumping to multiprocessing to handle it, if we thought that was a bit of a simplification | 23:08 |
| clarkb | I wonder if it would make sense to test that in the quickstart job? testing forking behavior in the unittest suite seems wrong, but maybe it is ok there | 23:13 |
| clarkb | the unittests have never been real unittests | 23:14 |
| ianw | yeah, it does not seem unit-testy, i agree ... it's a tricky one | 23:14 |
| *** rlandy|ruck has quit IRC | 23:57 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!