| *** bhagyashris_ has joined #openstack-infra | 01:01 | |
| *** bhagyashris has quit IRC | 01:09 | |
| *** rlandy has quit IRC | 02:45 | |
| *** ykarel|away has joined #openstack-infra | 03:48 | |
| *** ykarel_ has joined #openstack-infra | 04:15 | |
| *** ykarel|away has quit IRC | 04:21 | |
| *** ykarel_ is now known as ykarel | 04:25 | |
| *** opendevreview has quit IRC | 04:26 | |
| *** thiagop has joined #openstack-infra | 04:35 | |
| *** outbrito has quit IRC | 04:35 | |
| *** vishalmanchanda has joined #openstack-infra | 04:39 | |
| *** abhishekk has joined #openstack-infra | 04:55 | |
| *** bhagyashris_ is now known as bhagyashris | 06:08 | |
| *** david-lyle has joined #openstack-infra | 06:20 | |
| *** dklyle has quit IRC | 06:20 | |
| *** soniya29 has joined #openstack-infra | 06:24 | |
| *** soniya has joined #openstack-infra | 06:27 | |
| *** soniya29 has quit IRC | 06:33 | |
| *** Adri2000 has joined #openstack-infra | 06:48 | |
| *** david-lyle has quit IRC | 06:50 | |
| *** odyssey4me has joined #openstack-infra | 06:59 | |
| *** odyssey4me has quit IRC | 07:00 | |
| *** odyssey4me has joined #openstack-infra | 07:01 | |
| *** ykarel_ has joined #openstack-infra | 07:01 | |
| *** ykarel has quit IRC | 07:03 | |
| *** andrewbonney has joined #openstack-infra | 07:29 | |
| *** rpittau|afk is now known as rpittau | 07:30 | |
| *** jpena|off is now known as jpena | 07:31 | |
| *** soniya29 has joined #openstack-infra | 07:33 | |
| *** soniya has quit IRC | 07:39 | |
| *** soniya has joined #openstack-infra | 07:42 | |
| *** soniya29 has quit IRC | 07:48 | |
| *** elodilles is now known as elodilles_pto | 07:55 | |
| *** yasufum has joined #openstack-infra | 07:59 | |
| *** ykarel_ is now known as ykarel | 08:00 | |
| *** derekh has joined #openstack-infra | 08:23 | |
| *** lucasagomes has joined #openstack-infra | 08:25 | |
| *** tosky has joined #openstack-infra | 08:39 | |
| *** lucasagomes has quit IRC | 08:41 | |
| *** gfidente has joined #openstack-infra | 08:44 | |
| *** lucasagomes has joined #openstack-infra | 08:53 | |
| *** lucasagomes has quit IRC | 09:03 | |
| *** lucasagomes has joined #openstack-infra | 09:25 | |
| *** abhishekk has quit IRC | 09:27 | |
| *** lucasagomes has quit IRC | 09:38 | |
| *** lucasagomes has joined #openstack-infra | 09:39 | |
| *** soniya29 has joined #openstack-infra | 09:59 | |
| *** lucasagomes has quit IRC | 10:01 | |
| *** soniya has quit IRC | 10:03 | |
| *** yamak16 has quit IRC | 10:08 | |
| *** lucasagomes has joined #openstack-infra | 10:18 | |
| *** lucasagomes has quit IRC | 10:29 | |
| *** lucasagomes has joined #openstack-infra | 11:00 | |
| *** lucasagomes has quit IRC | 11:08 | |
| *** odyssey4me has quit IRC | 11:16 | |
| *** jpena is now known as jpena|lunch | 11:31 | |
| *** rlandy has joined #openstack-infra | 11:45 | |
| *** odyssey4me has joined #openstack-infra | 12:01 | |
| *** jpena|lunch is now known as jpena | 12:31 | |
| *** soniya has joined #openstack-infra | 12:48 | |
| *** soniya29 has quit IRC | 12:54 | |
| *** lucasagomes has joined #openstack-infra | 13:23 | |
| *** abhishekk has joined #openstack-infra | 13:34 | |
| *** elodilles_pto has quit IRC | 13:41 | |
| *** elodilles has joined #openstack-infra | 13:43 | |
| *** lucasagomes has quit IRC | 13:59 | |
| *** lucasagomes has joined #openstack-infra | 14:03 | |
| *** opendevreview has joined #openstack-infra | 14:05 | |
| opendevreview | Dmitrii Shcherbakov proposed openstack/project-config master: Add label-Verified to snaps.config https://review.opendev.org/c/openstack/project-config/+/795341 | 14:05 |
|---|---|---|
| *** lucasagomes has quit IRC | 14:18 | |
| *** vishalmanchanda has quit IRC | 14:18 | |
| *** lucasagomes has joined #openstack-infra | 14:21 | |
| *** philroche has joined #openstack-infra | 14:31 | |
| *** soniya has quit IRC | 14:36 | |
| ade_lee | ykarel, fungi hey -- I'm seeing this on centos-8-stream jobs .. https://zuul.opendev.org/t/openstack/build/8c01115fed714a45ad922f5263e58bc8/console | 14:40 |
| *** ykarel has quit IRC | 14:41 | |
| ade_lee | looks like its failing at trying to connect to mysql? | 14:41 |
| ade_lee | I originally thought it might have something to do with fips -- but its not even getting to that point | 14:42 |
| *** ykarel has joined #openstack-infra | 14:42 | |
| ade_lee | ykarel, did you see above ? | 14:43 |
| ade_lee | looks like its failing at trying to connect to mysql? | 14:43 |
| ade_lee | I originally thought it might have something to do with fips -- but its not even getting to that point | 14:43 |
| ykarel | ade_lee, no /me didn't see, i had some network issues | 14:44 |
| ade_lee | ykarel, ack -- I was asking about https://zuul.opendev.org/t/openstack/build/8c01115fed714a45ad922f5263e58bc8/console | 14:45 |
| ade_lee | ykarel, I'm seeing this on centos-8-stream jobs | 14:45 |
| *** ykarel has quit IRC | 14:46 | |
| fungi | looks like ykarel is having connectivity issues | 14:46 |
| fungi | er, still having i mean | 14:47 |
| *** ykarel has joined #openstack-infra | 14:47 | |
| opendevreview | Dmitrii Shcherbakov proposed openstack/project-config master: Add label-Verified to snaps.config https://review.opendev.org/c/openstack/project-config/+/795341 | 14:48 |
| fungi | ade_lee: the log at least seems to confirm that the mariadb-server package was installed on the system prior to that | 14:50 |
| *** dklyle has joined #openstack-infra | 14:50 | |
| fungi | does something maybe need to start the service? | 14:50 |
| ade_lee | fungi, yeah maybe .. let me see if I can throw up a patch to do that -- I just wanted to check if this was a known issue | 14:52 |
| fungi | ade_lee: also when did the failures start? we had a quota issue on our centos mirror volume which probably resulted in using packages from 2021-06-03 up through the middle of yesterday, so there's some ~4 days of package updates which happened on the mirror around then | 14:52 |
| fungi | wondering if something changed in how mariadb behaved by default in the latest 8-stream | 14:53 |
| ade_lee | fungi, well not sure -- until the last change that fixed the repo problem - ie. moving to centos-stream - the builds had been failing for that first | 14:53 |
| tosky | fungi: re "long while", thanks for clarifying, I was trying to not be too hard as (I think) I was in other parts of that email! | 14:55 |
| ykarel | ade_lee, yes seems like mariadb not running, so worth to check on that aspect | 14:57 |
| ykarel | also doesn't seem that job is collecting system logs | 14:57 |
| ade_lee | ykarel, yeah - not sure how to fix that -- maybe it conks out before that gets set up? | 14:58 |
| ykarel | may be can try attempt local reproducer for this or try node held, that should be more easier i think | 14:58 |
| ade_lee | ykarel, what's node held? | 15:00 |
| ykarel | ade_lee, you can ask infra for node held, means not to delete the node on which job is running, and can access the node and debug | 15:06 |
| ykarel | but for this case it looks simpler that mariadb is not running | 15:06 |
| *** lamt has quit IRC | 15:06 | |
| ykarel | mariadb do not start automatically | 15:06 |
| *** lamt has joined #openstack-infra | 15:06 | |
| ykarel | so u need to start mariadb.service explicitly | 15:06 |
| ykarel | before running tools/test-setup.sh | 15:07 |
| fungi | yeah, basically i'd set an autohold for the openstack-tox-functional-py36-fips with change 790536 in the openstack/glance project of zuul's openstack tenant, and the next time it gets a build failure for that it "holds" the node lock so that nodepool won't garbage collect th at build's nodeset | 15:08 |
| fungi | if you decide you need to manually inspect the failing environmen | 15:08 |
| fungi | t | 15:08 |
| ykarel | or can hack https://review.opendev.org/c/openstack/glance/+/790536/1/tools/test-setup.sh#17 to start mariadb with sudo systemctl status mariadb.service | 15:09 |
| ykarel | just for testing, as not sure what service name is there in ubuntu or other distros | 15:09 |
| ade_lee | ykarel, fungi ok - let me put up a test patch and see if it works | 15:12 |
| ade_lee | should be quick - this usually fails pretty quickly | 15:13 |
| *** tdasilva_ has joined #openstack-infra | 15:17 | |
| *** tdasilva has quit IRC | 15:17 | |
| *** thiago__ has joined #openstack-infra | 15:19 | |
| *** tdasilva_ has quit IRC | 15:19 | |
| opendevreview | Merged openstack/project-config master: Add label-Verified to snaps.config https://review.opendev.org/c/openstack/project-config/+/795341 | 15:21 |
| opendevreview | Merged openstack/project-config master: New Project Request: airship/image-builder https://review.opendev.org/c/openstack/project-config/+/795118 | 15:31 |
| *** tbrito has joined #openstack-infra | 15:57 | |
| *** thiagop has quit IRC | 15:57 | |
| clarkb | fungi: gmann: re the d-g thread do you think that it is worth mentioning we have never required zuul or a specific version of zuul? but in those cases CI systems may need to sort things out from scratch? (I don't want to give the impression we're forcing them to make any particular changes here). Also, what are the risks with handing d-g over to another group? Is it old | 15:59 |
| clarkb | stable branches that still use d-g? should we maybe suggest they fork in that case? | 16:00 |
| *** d34dh0r53 has joined #openstack-infra | 16:00 | |
| clarkb | I mean upgrading to zuulv3 or newer sounds great, but thehy may be running jenkins and not zuul etc | 16:00 |
| gmann | clarkb: for stable branch which is until stable/ussuri (if project has not backported the zuulv3 migration work) we should keep it as it is. i mean under same group | 16:01 |
| clarkb | sounds like a fork would be appropriate if people want to keep it alive for modern development then | 16:01 |
| gmann | clarkb: fungi I am thinking to mark it deprecated and no-fix for master and fix acceptable for stable branch only | 16:02 |
| gmann | clarkb: yeah | 16:02 |
| clarkb | gmann: d-g doesn't have stable branches, all development is on master | 16:02 |
| gmann | i mean support for stable branch | 16:02 |
| clarkb | but I guess we can communicate the changes to master are for stable/old on other repos | 16:02 |
| clarkb | ++ | 16:02 |
| *** ykarel is now known as ykarel|away | 16:02 | |
| gmann | we should do some official warning or mechanism to stop running it for >stable/wallaby or so | 16:03 |
| gmann | do you think that is fine now? | 16:03 |
| gmann | this one especially 'stop running it for >stable/wallaby ' | 16:03 |
| clarkb | yes, why don't we add a warning (don't break it and make the situation for those CIs worse) and update the readme that this is maintenance for old branches only | 16:03 |
| clarkb | then we can also respond to that thread suggesting they fork it if they want to keep using the tool for newer branches | 16:04 |
| gmann | ok, so just warning but no hard error | 16:04 |
| clarkb | that would be my vote | 16:05 |
| gmann | ok but in that case anyone asking for fix and fixes for master is hard to deny. but fine i think | 16:05 |
| clarkb | gmann: agreed. If it works without changing anything we won't intentionally break you, but if you need fixes we will -2 those | 16:06 |
| gmann | sure | 16:06 |
| clarkb | and we are happy for someone else to fork it and maintain it if they decide that is better for them than updating their CI systems | 16:06 |
| clarkb | (has to be a fork to avoid concerns with old stable branch testing) | 16:06 |
| gmann | and how about deprecation in governance ? I think we can do that | 16:07 |
| clarkb | ++ | 16:07 |
| gmann | we would not be able to do in project gate or so as it is branchless but officially marking in governance will help | 16:08 |
| clarkb | because once those old stable branches die we can stop the projcet entirely | 16:08 |
| gmann | exactly | 16:08 |
| clarkb | sounds liek a good plan. Do you want to respond to the list with updates after the changes are pushed or would it help if I do that? | 16:08 |
| fungi | doesn't necessarily even need to wait for the old branches to reach eol, in a number of cases we've opted to reduce test coverage in em branches | 16:09 |
| gmann | I can push the change and then you can reply on email. I will push after nova meeting or so | 16:09 |
| clarkb | gmann: sounds good, thanks! | 16:09 |
| *** ykarel|away has quit IRC | 16:13 | |
| gmann | one more thing, on gate testing. as we will be supporting only stable branch, I am thinking to stop master jobs from there and only have stable branch jobs ? which is actually part of deprecation. what you say? | 16:13 |
| clarkb | gmann: ++ | 16:13 |
| gmann | ok, will do | 16:14 |
| *** lucasagomes has quit IRC | 16:15 | |
| *** rpittau is now known as rpittau|afk | 16:31 | |
| *** jpena is now known as jpena|off | 16:33 | |
| *** akekane_ has joined #openstack-infra | 16:34 | |
| *** akekane_ has quit IRC | 16:35 | |
| opendevreview | Merged openstack/project-config master: Drop logstash filter jobs https://review.opendev.org/c/openstack/project-config/+/792710 | 16:37 |
| *** abhishekk has quit IRC | 16:41 | |
| ade_lee | fungi, are we running postgres too? restarting mariadb succeeded , but then postgres failed .. | 16:51 |
| opendevreview | Ghanshyam proposed openstack/project-config master: Move devstack-gate jobs list to in-tree https://review.opendev.org/c/openstack/project-config/+/795379 | 16:51 |
| opendevreview | Ghanshyam proposed openstack/devstack-gate master: Move devstack-gate jobs list to in-tree https://review.opendev.org/c/openstack/devstack-gate/+/795380 | 16:52 |
| fungi | ade_lee: it's running whatever's in that script in the glance repo | 16:54 |
| fungi | ade_lee: https://opendev.org/openstack/glance/src/branch/master/tools/test-setup.sh | 16:54 |
| fungi | i assume glance has postgres-specific tests so sets it up to support those | 16:55 |
| ade_lee | fungi, ack - trying to restart postgres too now | 16:55 |
| fungi | if glance has dropped their postgres tests, then they should probably clean up references in bindep.txt and tools/test_setup.sh | 16:55 |
| fungi | would make their jobs faster too | 16:56 |
| clarkb | I think oslo.db has standardized migration testing for mysql and postgres | 16:56 |
| ade_lee | fungi, in any case - its seems like restarting mariadb did move us further through the script -- so now we just need to put a restart somewhere | 16:57 |
| ade_lee | and make it non-platform specific | 16:57 |
| clarkb | gmann: will you use topic:"deprecate-devstack-gate" for the governance change too? if so I'll start drafting that email and use a topic lookup link for gerrit links | 16:58 |
| clarkb | ade_lee: fungi: note that typically projects don't run unittests on rhel/fedora/centos this means you may run into all sorts of problems doing that conversion | 16:58 |
| gmann | clarkb: initially for governance yes. but later I need to change that to project-update | 16:58 |
| clarkb | also I'd probably strongly recommend against doing that conversion now because centos8's lifetime is short and centos-8-stream isn't super stable | 16:58 |
| clarkb | ubuntu gives you 5 years to run those jobs on which is useful for stable branch lifetime | 16:59 |
| fungi | though for this particular case, i wonder if ubuntu is as well-suited to running fips-centric deployments | 16:59 |
| clarkb | fungi: it may not be, just pointing out that switching wholesale to centos is likely to be problematic longer term | 17:00 |
| ade_lee | clarkb, yup - I'm seeing that this is indeed the case -- the reason I'm doing this is because I wanted to run fips specific tests - | 17:00 |
| fungi | and i probably underestimate the number of usa federal contracts canonical holds | 17:00 |
| ade_lee | clarkb, oh - I'm not thinking of switching out long term - just making them work for the fips tests | 17:01 |
| ade_lee | clarkb, fungi I'm ok with adding some code somewhere that says "if this is centos, restart mariadb and postgres" | 17:02 |
| ade_lee | just need to figure out where | 17:02 |
| ade_lee | I'm seeing the same problem in glance, cinder, nova .. | 17:03 |
| fungi | ade_lee: yeah, most projects copied nova's tools/test_setup.sh file, which is why we made it into a defacto-standard with a common job role | 17:03 |
| fungi | ade_lee: swift has some platform detection in theirs you could crib from: https://opendev.org/openstack/swift/src/branch/master/tools/test-setup.sh | 17:04 |
| ade_lee | fungi, cool - I can do that then. | 17:05 |
| *** ykarel|away has joined #openstack-infra | 17:06 | |
| ade_lee | fungi, so better to do in each projects test_setup.sh -- or is there somewhere earlier where we install mariadb and postgres where it would be better to put common code? | 17:07 |
| fungi | also comparing and contrasting nova's and swift's test_setup.sh provides a great example of the flexibility of that approach | 17:07 |
| ade_lee | fair enough :) | 17:07 |
| fungi | ade_lee: there is no common code around that. basically the test_setup.sh script was settled on as the stopgap for "my project has particular things which need to be done as root in addition to installing distro packages before some tests can be run" | 17:08 |
| ade_lee | ok | 17:08 |
| fungi | so for swift, it creates an xfs filesystem to enable some of their file attribute functional testing | 17:09 |
| fungi | for nova, it preconfigures some database servers to support database functional testing | 17:09 |
| fungi | but it's intentionally arbitrary and turing-complete | 17:09 |
| fungi | to the job role calling those scripts, they're a black box | 17:09 |
| fungi | the idea being developers can run precisely the same scripts to set up their local development environments | 17:10 |
| fungi | we could probably find some commonalities between them and abstract them into a centralized library, but then you get into some interesting bootstrapping catch-22s | 17:11 |
| ade_lee | fungi, can I get a node hold for https://review.opendev.org/c/openstack/glance/+/790536 ? I tried restarting postgres - but ran into issues http://paste.openstack.org/show/806465/ | 17:18 |
| ade_lee | fungi, ah - it seems there is an init scrpt too -- I'll try run that | 17:23 |
| opendevreview | Ghanshyam proposed openstack/devstack-gate master: Add deprecation warning in README file https://review.opendev.org/c/openstack/devstack-gate/+/795383 | 17:28 |
| clarkb | gmann: I need to step out for a few, but I sent the email already and I'll do reviews when I get back | 17:29 |
| gmann | clarkb: thanks. I will work on job things after my lunch. | 17:29 |
| *** andrewbonney has quit IRC | 17:32 | |
| fungi | ade_lee: do you still want the autohold or are you trying something else first? | 17:38 |
| ade_lee | fungi, yeah - I got further - but I think I'm gonna need to get on a system to actually run things and see why they fail .. | 17:39 |
| ade_lee | http://paste.openstack.org/show/806466/ | 17:40 |
| ade_lee | I suspect I may need to add the zuul user to a postgres group or something maybe | 17:42 |
| fungi | ade_lee: seems you have a lot of broken jobs reporting on that change, any one in particular you want the autohold for? | 17:43 |
| *** ykarel|away has quit IRC | 17:43 | |
| fungi | openstack-tox-functional-py36-fips i guess? | 17:43 |
| ade_lee | fungi, https://review.opendev.org/c/openstack/glance/+/790536 - the glance one | 17:43 |
| ade_lee | yup | 17:43 |
| ade_lee | if I fix that one, I can fix the rest (hopefully) | 17:44 |
| fungi | ade_lee: the autohold is set, also you can trim out any jobs you don't care about in the .zuul.yaml within your wip change to speed up testing | 17:45 |
| ade_lee | fungi, good idea thanks | 17:46 |
| fungi | i do that all the time if i'm trying to isolate a problem with a single job, and then restore the .zuul.yaml after i've worked it out before dropping wip state | 17:46 |
| fungi | ade_lee: also another option, for future reference, is to just locally boot our test images and try stuff out: https://nb01.opendev.org/images/ | 17:47 |
| ade_lee | fungi, ack - thanks | 17:48 |
| ade_lee | bookmarking now | 17:48 |
| fungi | you might compare against what's on nb02 if you want to make sure you have the very latest serial for a given label, the image build load is distributed between them at random | 17:48 |
| fungi | they use glean and expect a configdrive, and expect you to supply sshkey metadata | 17:50 |
| fungi | but other than that nothing too special about booting them | 17:50 |
| fungi | no metadata server needed. can handle dhcp by default or static network configuration in the configdrive | 17:51 |
| clarkb | if you don't want to set up a config drive you can modify the image to include your ssh key and they will dhcp by default | 17:52 |
| *** gfidente is now known as gfidente|afk | 18:02 | |
| opendevreview | Rajath Av proposed openstack/project-config master: Changes to be committed: modified: zuul.d/projects.yaml Change-Id: Ib712abce33b5f552965be874d9965a24eb7ec105 https://review.opendev.org/c/openstack/project-config/+/795396 | 18:29 |
| opendevreview | Rajath Av proposed openstack/project-config master: Changes to be committed: modified: zuul.d/projects.yaml Change-Id: Ib712abce33b5f552965be874d9965a24eb7ec105 Adding this change as networking-infoblox plugin has been migrated to support python3.6 https://review.opendev.org/c/openstack/project-config/+/795396 | 18:30 |
| ade_lee | fungi, ok -- so how do I get to my node -- will zuul tell me when its done? | 18:42 |
| opendevreview | Rajath Av proposed openstack/project-config master: Changes to be committed: modified: zuul.d/projects.yaml https://review.opendev.org/c/openstack/project-config/+/795397 | 18:42 |
| clarkb | ade_lee: zuul will report the ip in its logs when the job fails, but y ouneed one of us to add your ssh key to it before you'll be able to get in. If you can point us to the job lgo when done that is helpful | 18:42 |
| ade_lee | clarkb, ok thanks -- I think its trying three times .. | 18:43 |
| clarkb | oh is it failing in pre-run? zuul will only hold the last one | 18:43 |
| ade_lee | yup | 18:43 |
| opendevreview | Rajath Av proposed openstack/project-config master: Changes to be committed: modified: zuul.d/projects.yaml https://review.opendev.org/c/openstack/project-config/+/795396 | 18:43 |
| opendevreview | Rajath Av proposed openstack/project-config master: Changes to be committed: modified: zuul.d/projects.yaml https://review.opendev.org/c/openstack/project-config/+/795397 | 18:45 |
| fungi | ade_lee: right, need to wait for that RETRY_LIMIT result to show up | 18:45 |
| fungi | seems wasteful in this particular case, but more generally it helps weed out intermittent failures due to things like network connectivity problems from actual job failures | 18:46 |
| ade_lee | fungi, clarkb by the way - if you'll want to review https://review.opendev.org/c/zuul/zuul-jobs/+/788778 - I think it looks pretty good now | 18:47 |
| ade_lee | and we see that one working in the barbican patch -- https://review.opendev.org/c/openstack/barbican/+/760665 | 18:48 |
| ade_lee | fungi, clarkb not sure where the ip is -- https://zuul.opendev.org/t/openstack/build/607fb96cc15a4daa8c2844aa453455f3/log/job-output.txt | 18:49 |
| ade_lee | 158.69.66.94 ? | 18:49 |
| ade_lee | clarkb, fungi -- my key -- https://github.com/vakwetu.keys | 18:50 |
| fungi | yeah, https://zuul.opendev.org/t/openstack/build/607fb96cc15a4daa8c2844aa453455f3/log/zuul-info/inventory.yaml#10 | 18:52 |
| fungi | i can't seem to ssh to it though | 18:54 |
| fungi | aha, for some reason the node which was held is 23.253.236.4 | 18:55 |
| clarkb | oh did it hold the first then? I know it only holds one | 18:57 |
| fungi | trying to figure that out, it's node 0025027287 anyway | 18:57 |
| fungi | zuul debug log says it was build 3b662e15c3c54f43b44dccb9ca569619 which isn't in the db, maybe it hasn't reported yet? | 19:02 |
| clarkb | that sounds like it held the first fail which doesn't get recordred otherwise | 19:03 |
| clarkb | neat | 19:03 |
| fungi | mmm, interestingly the build history for that job does show "retry" result builds which aren't "retry_limit" though | 19:03 |
| fungi | and those seem to be the earlier builds | 19:04 |
| fungi | but they have different build ids than this one | 19:04 |
| fungi | aha, i think i see what happened. based on timestamps i believe this ran against 790536,4 but that never reported because it was aborted by the upload of 790536,5 | 19:12 |
| fungi | anyway, this node *probably* has what you need, if it doesn't we can reset the trap and recheck when there are no new patchsets getting pushed | 19:13 |
| ade_lee | fungi, yeah - it should be fine | 19:14 |
| fungi | ade_lee: ssh root@23.253.236.4 | 19:14 |
| opendevreview | Rajath Av proposed openstack/project-config master: Changes to be committed: modified: zuul.d/projects.yaml https://review.opendev.org/c/openstack/project-config/+/795397 | 19:14 |
| ade_lee | fungi, perfect - I'm in - thanks! | 19:15 |
| ade_lee | fungi, I'll let you know when I'm done with it | 19:15 |
| fungi | much obliged | 19:16 |
| *** zxiiro has quit IRC | 19:23 | |
| fungi | clarkb: so, anyway, to summarize, zuul does report the retry builds, but not aborted builds (which is what ended up being held, i guess because an abort counts as a hard failure for the autohold logic?) | 19:31 |
| clarkb | huh I dind't think it did but your evidecnce seems to support that | 19:31 |
| fungi | or more likely this was one which ended in a retry_limit but the buildset itself hadn't reported yet and got aborted | 19:31 |
| fungi | so by that time it had already triggered the hold | 19:32 |
| *** stevebaker has quit IRC | 19:33 | |
| *** thiago__ is now known as tdasilva | 19:43 | |
| opendevreview | Merged openstack/project-config master: Changes to be committed: modified: zuul.d/projects.yaml https://review.opendev.org/c/openstack/project-config/+/795397 | 19:55 |
| *** stevebaker has joined #openstack-infra | 20:19 | |
| *** derekh has quit IRC | 20:48 | |
| *** derekh has joined #openstack-infra | 21:01 | |
| *** derekh has quit IRC | 21:25 | |
| *** reed has joined #openstack-infra | 21:56 | |
| gmann | clarkb: fungi is there any way to pick the job definition from stable branch (not from master) while running it on master gate? | 22:24 |
| gmann | I was checking if devstack-gate could run the neutron-grenade-multinode job from stable/stein version which is legacy job version not the master one which is zuulv3 native | 22:25 |
| clarkb | I think you may need to use a different job name? and have it fallback to doing the master lookup? | 22:26 |
| clarkb | the pragma stuff won't work because that applies globally in the tenant making the branches equivalent | 22:26 |
| clarkb | you maybe able to make a variant that applies to a specific branch? | 22:26 |
| gmann | but neutron-grenade-multinode job name is same in stable and master in neutron repo | 22:31 |
| gmann | and if i inherit newjob rom 'neutron-grenade-multinode' still I need to add branch variant to master as devstack-gate master gate need to run it so it will pick base job from neutron's master ? | 22:33 |
| clarkb | yes inheritance alone won't do it as it will find the master job | 22:33 |
| clarkb | instead I think you can add a job on master that overrides the branches that are used when the job runs | 22:34 |
| clarkb | then that job will be used and run over the branches that you want | 22:34 |
| gmann | clarkb: but override branch using override-checkout is not I want here. I need devstack-gate run legacy version of job 'neutron-grenade-multinode' so that i test the d-g code path | 22:35 |
| clarkb | yes, I'm saying you need to copy the legacy version of the job to master and give it a new name and set the vars to older branches when you do that | 22:36 |
| clarkb | I don't think there is an easier way than that but I could be wrong | 22:36 |
| clarkb | Zuul is built around the assumption that you basically never want to do this I think | 22:36 |
| gmann | humm, copying is one option but maintaining that when it break is another issue instead of using neutron own one. | 22:37 |
| gmann | clarkb: we have legacy tempest job running in devstackl-gate. I was thinking to add one legacy grenade job also (which is not there currently also) for test coverage. but by seeing the copy-only option I am thinking we continue with the testing we have (legacy tempest based job only). what you say? | 22:39 |
| gmann | if somehow d-g is failed with any legacy grenade job then we can fix from the log/debug where it was failing | 22:39 |
| clarkb | gmann: ya I think the legacy tempest jobs are basically a copy paste since they have different names | 22:39 |
| gmann | otherwise preparing grenade legacy job version seems more work | 22:40 |
| gmann | clarkb: yes | 22:40 |
| clarkb | and ya fixing things with depends on is probably easiest if we run into problems | 22:40 |
| gmann | yeah | 22:40 |
| opendevreview | Ghanshyam proposed openstack/devstack-gate master: Keep only legacy jobs which actually test the d-g code path https://review.opendev.org/c/openstack/devstack-gate/+/795426 | 22:47 |
| gmann | clarkb: ^^ | 22:48 |
| *** derekh has joined #openstack-infra | 22:56 | |
| *** tosky has quit IRC | 23:03 | |
| *** yamak16 has joined #openstack-infra | 23:52 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!