| *** akahat is now known as akahat|ruck | 02:23 | |
| *** rcastillo|rover is now known as rcastillo | 04:29 | |
| *** soniya29 is now known as soniya29|ruck | 04:45 | |
| *** soniya29|ruck is now known as soniya29|rover | 04:45 | |
| * soniya29 will be back in 1 hr | 05:30 | |
| *** soniya29 is now known as soniya29|afk | 05:30 | |
| *** yadnesh|away is now known as yadnesh | 05:58 | |
| marios | o/ | 06:06 |
|---|---|---|
| marios | rcastillo: thanks merging https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/45877 | 06:32 |
| marios | akahat|ruck: soniya29|afk: o/ | 06:55 |
| akahat|ruck | marios, o/ | 07:11 |
| marios | hi can you please update https://hackmd.io/2hB-P772SqyqDs0KKZzZEQ?view | 07:11 |
| marios | akahat|ruck: did you and soniya29|afk already sync earlier? | 07:12 |
| marios | like who is doing what /split the responsibilities etc | 07:12 |
| akahat|ruck | marios, yeah.. we had sync.. I'll be looking in upstream and soniya29|afk will take a look in downstream. | 07:13 |
| * akahat|ruck updating hackmd | 07:13 | |
| marios | ok cool | 07:13 |
| marios | akahat|ruck: fyi https://review.opendev.org/c/openstack/python-tripleoclient/+/864208 for 17.1/9 cix | 07:20 |
| akahat|ruck | marios, ack | 07:22 |
| marios | akahat|ruck: re https://hackmd.io/Vn_zjslARMG5Crs4YwnSqw - 17.1/8 also promoted yesterday 10th | 07:26 |
| akahat|ruck | updated | 07:29 |
| *** soniya29|afk is now known as soniya29 | 07:32 | |
| *** amoralej|off is now known as amoralej | 07:40 | |
| * akahat|ruck lunch | 07:53 | |
| jm1 | good morning folks :) | 07:55 |
| jm1 | pojadhav: hello :) | 07:56 |
| jm1 | pojadhav: your patch had to be reverted again :/ https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/46045/ | 07:57 |
| jpodivin | marios: hi. | 07:57 |
| jpodivin | are you here? | 07:57 |
| jm1 | pojadhav: maybe its due to missing files? | 07:58 |
| marios | jpodivin: o/ | 07:58 |
| jm1 | pojadhav: it seems as if other files have been edited as well which might also be required: ssh centos@10.0.111.235 'cd /home/centos/ci-config && git status' | 07:58 |
| marios | jpodivin: thanks for updates i added info into BZ and tried to sell your patch earlier in tripleo seems folks not around yet | 07:59 |
| jm1 | pojadhav: could you please add all those edited files to a new patch, even if they seem unrelated? | 07:59 |
| jm1 | pojadhav: then i will merge your patch and in worst case we revert again. but hopefully we will get some idea what is going on here | 08:00 |
| marios | akahat|ruck: soniya29: if you need something please ping. since it is quiet please stay ontop of the promotions - check last runs of each periodic line and see what you can chase (and if there are any real issues there that need bug/cix) | 08:01 |
| *** jpena|off is now known as jpena | 08:01 | |
| jpodivin | marios: We are going to take a look at those other errors too. It's possible that the installation isn't working out for some reason, hence the permissions errors. | 08:02 |
| marios | jpodivin: ok | 08:08 |
| marios | soniya29: can you please re-nick | 08:08 |
| marios | bhagyashris: o/ did you get onto the node? | 08:16 |
| marios | bhagyashris: i saw you hit recheck on the test patch but why? | 08:16 |
| marios | was there some udpate | 08:16 |
| bhagyashris | marios, nope i see the issue is different so rechecked the job | 08:16 |
| bhagyashris | once it finished will get into it | 08:16 |
| marios | bhagyashris: can you please give me IP address in pvt (will give you my key sec | 08:17 |
| marios | bhagyashris: what is the different issue? | 08:17 |
| bhagyashris | some netstat issue https://sf.hosted.upshift.rdu2.redhat.com/logs/44/434544/5/check/periodic-tripleo-ci-rhel-9-8-multinode-mixed-os-rhos-17.1/6b69623/job-output.txt | 08:17 |
| marios | bhagyashris: https://sf.hosted.upshift.rdu2.redhat.com/logs/44/434544/5/check/periodic-tripleo-ci-rhel-9-8-multinode-mixed-os-rhos-17.1/6b69623/job-output.txt | 08:17 |
| marios | bhagyashris: no that is ..ignoring | 08:17 |
| marios | bhagyashris: looks fine | 08:18 |
| marios | bhagyashris: it fails because you forced it | 08:18 |
| marios | bhagyashris: 2022-11-10 11:19:31.234096 | primary | +(./toci_quickstart.sh:182): echo 'Quickstart completed with exit code 0 but forcing failure with exit 1' | 08:18 |
| bhagyashris | ok i thought it's diffrent issue | 08:18 |
| marios | bhagyashris: sent you my key in pvt | 08:19 |
| marios | bhagyashris: so we have the node from yesterday right? | 08:19 |
| bhagyashris | marios, can you pass me your ssh pub key | 08:19 |
| marios | bhagyashris: just did | 08:19 |
| bhagyashris | let me ssh | 08:19 |
| marios | thanks | 08:19 |
| bhagyashris | i have logs but let me check i can login to that or not | 08:19 |
| bhagyashris | marios, hey my bad i forgot to hold the node let me hold that | 08:21 |
| bhagyashris | holding node | 08:21 |
| marios | bhagyashris: i was going to say... i cant see your request in the list of holds | 08:21 |
| bhagyashris | holding | 08:21 |
| marios | bhagyashris: k please add a request to hold the one you jut rechecked | 08:21 |
| bhagyashris | marios, apology for not holding node. i thought i hold a node | 08:30 |
| marios | bhagyashris: ack i see we have one now so should be on the node in couple hours | 08:32 |
| soniya29 | marios, sure | 08:53 |
| *** soniya29 is now known as soniya29|rover | 08:53 | |
| marios | soniya29|rover: o/ can you please add the testproject for promotions you are chasing in the hackmd | 08:59 |
| soniya29|rover | marios, ack | 08:59 |
| soniya29|rover | marios, seems 17.1 jenkins jobs are still failing due to https://bugzilla.redhat.com/show_bug.cgi?id=2141701 | 09:00 |
| marios | soniya29|rover: yes try sync with jpodivin about that. they have fix in progress, but we may also want to explore disable those validations | 09:02 |
| marios | soniya29|rover: so dig there/ask jpodivin about how you can disable them and post a patch ... if the issue isn't resolved by end of today we can disable | 09:03 |
| soniya29|rover | marios, okay | 09:03 |
| akahat|ruck | marios, sure. | 09:20 |
| akahat|ruck | marios, soniya29|rover fond one in component jobs: https://bugs.launchpad.net/tripleo/+bug/1996264 | 09:20 |
| * bhagyashris lunch | 09:21 | |
| soniya29|rover | akahat|ruck, ack | 09:21 |
| jm1 | pojadhav: pooooooojaaaaaa where aaart you?? | 09:27 |
| jm1 | ;) | 09:27 |
| pojadhav | jm1, hey I am here | 09:27 |
| pojadhav | :D | 09:27 |
| jm1 | pojadhav: ah there you are, hello :D | 09:27 |
| pojadhav | jm1, hello | 09:27 |
| jm1 | pojadhav: wasnt sure whether you saw my messages :D | 09:27 |
| * marios food biab | 09:28 | |
| pojadhav | jm1, yep I read msgs.. but I am scared to touch prod server.. :D | 09:28 |
| jm1 | pojadhav: no worry. when you prepare the patch, i will merge and thus be responsible;) | 09:28 |
| pojadhav | i need someone more experienced with me :) | 09:29 |
| jm1 | pojadhav: how can i help? | 09:29 |
| pojadhav | jm1, can we sit together and having tmate session and do changes togther.. | 09:29 |
| pojadhav | what you think | 09:29 |
| jm1 | pojadhav: sure! | 09:30 |
| pojadhav | jm1, give me some time.. will be back and ping you | 09:30 |
| jm1 | pojadhav: yep, ping me :) | 09:30 |
| soniya29|rover | marios, had discussion with jpodivin regarding validations..he suggests to not disable the validations since it may disable ALL checks of the undercloud | 09:33 |
| jpodivin | the installation procedure calls `check` function, under that all other checks are invoked in order | 09:34 |
| jpodivin | validations included. | 09:35 |
| jpodivin | `--no-validations` arg, or the equivalent in config file would disable that, but it would disable anything. | 09:35 |
| soniya29|rover | akahat|ruck, marios i have updated hackmd for the downstream | 09:48 |
| jm1 | frenzy_friday: hey ananya :) | 09:49 |
| jm1 | frenzy_friday: you once defined your own ansible inventory for your personal staging env, didnt you? | 09:49 |
| frenzy_friday | jm1, hey, yep | 09:50 |
| jm1 | frenzy_friday: did you upload it to a repo? | 09:51 |
| jm1 | frenzy_friday: or do you have it locally only? | 09:51 |
| frenzy_friday | no, I had it locally | 09:52 |
| jm1 | frenzy_friday: but you still have it? | 09:52 |
| frenzy_friday | This is what I did (I dont have the files anymore) | 09:52 |
| frenzy_friday | ## Incockpit deploy | 09:52 |
| frenzy_friday | - ci-scripts/infra-setup/tenant_vars/infra-tripleo/→ change according to incockpit psi stuff | 09:52 |
| frenzy_friday | - ci-scripts/infra-setup/tenant_vars/infra-tripleo/servers.yml → keep only incockpit | 09:52 |
| frenzy_friday | - source the psi rc file | 09:52 |
| frenzy_friday | - ansible-playbook ci-scripts/infra-setup/provision-all.yml | 09:52 |
| frenzy_friday | oh wait, I have some of them :D | 09:53 |
| jm1 | frenzy_friday: dasm and i overhauled the inventory structure. we now should also be able to test changes/patches before actually deploying them to production. do you want to prepare a patch to add your own env? | 09:54 |
| frenzy_friday | jm1, can we have a quick call? | 09:54 |
| jm1 | frenzy_friday: sure | 09:54 |
| frenzy_friday | https://meet.google.com/hdu-tveg-ngb | 09:54 |
| akahat|ruck | marios, I'm getting "Merge failed" error in testproject. Have you seen this before? | 10:09 |
| akahat|ruck | https://review.rdoproject.org/r/c/testproject/+/46075 | 10:09 |
| akahat|ruck | https://review.rdoproject.org/r/c/testproject/+/41465 | 10:10 |
| marios | akahat|ruck: yes when there was a problem with upstream gerrit on 1 nov | 10:11 |
| marios | akahat|ruck: there https://code.engineering.redhat.com/gerrit/c/testproject/+/427472/9#message-b563a599f65ee6623610058362e1925954b5cc41 | 10:11 |
| marios | akahat|ruck: try asking in rhos-ops to see if there are known issues with rdo zuul/gerrit? | 10:12 |
| akahat|ruck | marios, looks like this issue is back. | 10:13 |
| akahat|ruck | going with #rhos-ops | 10:13 |
| marios | akahat|ruck: add note in the hackmd about this issue for now please | 10:15 |
| akahat|ruck | marios, ack | 10:15 |
| pojadhav | jm1, hey | 10:23 |
| pojadhav | are you free now ? | 10:23 |
| jm1 | pojadhav: 3 mins | 10:23 |
| pojadhav | jm1, sure | 10:24 |
| jm1 | pojadhav: reaady :) | 10:31 |
| pojadhav | jm1, https://meet.google.com/xfh-hiaa-zrp | 10:31 |
| marios | soniya29|rover: i missed your ping soniya29|rover> marios, had discussion with jpodivin regarding validations..he suggests to not disable the validations since it may disable ALL checks of the undercloud | 10:40 |
| marios | soniya29|rover: jpodivin: obviously we dont want to degrade by disabling, but the issue has blocked for few days already | 10:41 |
| marios | soniya29|rover: jpodivin: i think we need to consider it at least prepare for it | 10:41 |
| marios | jpodivin: can you please advise soniya29|rover where/how to disable so she can prepare the patch | 10:41 |
| marios | jpodivin: and we can decide when to merge it/maybe we wont | 10:41 |
| marios | enable_validations | 10:45 |
| marios | undercloud_preflight | 10:45 |
| marios | jpodivin: can we be any more spcific than that? ^^ or all of those | 10:45 |
| marios | undercloud_enable_validations: false | 10:47 |
| marios | jpodivin: so is there really no way to switch off some of those do we have to do all or nothing? | 10:50 |
| pojadhav | frenzy_friday, hey can you please join us https://meet.google.com/xfh-hiaa-zrp | 10:52 |
| jm1 | frenzy_friday: do you have time to join us for a minute? https://meet.google.com/xfh-hiaa-zrp | 10:52 |
| jpodivin | marios: as I've indicated, the `--no-validations` argument is the appropriate way to disable validations. It is rather blunt, it disables everything. | 10:52 |
| marios | jpodivin: where did you indicate that did i miss some earlier chat? scrolling | 10:53 |
| jpodivin | marios: 10:35:34 | 10:53 |
| marios | jpodivin: i see. so we can't say disable only undercloud_preflight? | 10:53 |
| jpodivin | marios: well this does disable all pre-flight checks. So essentially it does what you want | 10:54 |
| marios | k soniya29|rover can you please ask atila in rhos-ops about this - the job definitions are jenkins not zuul (so not in our usual repo). we'd need to understand where to add the --no-validations for those jobs | 10:55 |
| marios | jpodivin: do you think the fix is good or dont know yet too early | 10:56 |
| marios | jpodivin: like you were investigating the other issue with the missing directory? | 10:56 |
| jpodivin | marios: the missing directory should be fine. It's more degradation imho. What kills the preflight is the lack of inventory. That should be addressed https://review.opendev.org/c/openstack/python-tripleoclient/+/863541 | 10:57 |
| jpodivin | which has just merged. | 10:57 |
| jpodivin | no | 10:57 |
| jpodivin | wait, it's still in progress | 10:57 |
| jpodivin | gate is slow or something | 10:58 |
| marios | gate should be OK 3 hour queue right now | 10:59 |
| marios | soniya29|rover: sent you link in private probably the config needs to go directly in jenkins but would be good to ask atila and confirm | 11:06 |
| soniya29|rover | marios, ack | 11:26 |
| frenzy_friday | 0/ folks , jm1 pojadhav and I are working on the upstream cockpit - it may break from time to time today. Pls use the internal one for now :D | 11:33 |
| marios | frenzy_friday: should let the ruck rover know especially for these things soniya29|rover akahat|ruck ^^ | 11:38 |
| soniya29|rover | marios, frenzy_friday, ack | 11:38 |
| akahat|ruck | frenzy_friday, marios thank you for info. | 11:39 |
| marios | frenzy_friday: jm1: please don't leave it broken today though i mean revert before weekend if there are breaking changes | 11:42 |
| *** dviroel|afk is now known as dviroel | 11:43 | |
| marios | bhagyashris: digging but i think i see what/why the issue happens. search for 2022-11-11 10:57:46.035208 in the latest job-output.txt | 12:04 |
| marios | bhagyashris: you will see it twice | 12:04 |
| marios | bhagyashris: that is the problem, like the hash_info.sh is updated correctly but it seems that task is executed 'earlier' somehow | 12:05 |
| marios | bhagyashris: cant find any other hash_info.sh created on the undercloud been digging there | 12:08 |
| marios | or the other nodes | 12:08 |
| bhagyashris | marios, yeah | 12:08 |
| bhagyashris | i saw that | 12:08 |
| jm1 | pojadhav, frenzy_friday: back :) | 12:12 |
| bhagyashris | marios, but here it's showing 8 | 12:14 |
| bhagyashris | 2022-11-11 10:57:55.379279 | TASK [get_hash : print out dlrn_api url] | 12:14 |
| bhagyashris | 2022-11-11 10:57:55.456847 | dlrnapi_url: https://osp-trunk.hosted.upshift.rdu2.redhat.com/api-rhel8-osp17-1 | 12:14 |
| *** yadnesh is now known as yadnesh|away | 12:14 | |
| marios | bhagyashris: get_hash is working fine and it creates the hash_info.sh correctly | 12:15 |
| marios | bhagyashris: problem is ansible limitation https://docs.ansible.com/ansible/latest/user_guide/playbooks_reuse_roles.html#running-a-role-multiple-times-in-one-play it only run dlrn-report once | 12:16 |
| marios | bhagyashris: i am going to post a patch in a bit but we'll have to merge agian, trick is to make sure we only affect this job | 12:16 |
| bhagyashris | ok | 12:18 |
| frenzy_friday | marios, yep, we will revert by EU eod | 12:19 |
| marios | thx frenzy_friday | 12:19 |
| frenzy_friday | jm1, pojadhav ansible pull has reverted the dashboard. I am stopping ansible pull and making a save without any dashboard changes | 12:21 |
| frenzy_friday | jm1, oh you already did that :D | 12:21 |
| frenzy_friday | did you already upload the patch? (I see the commit on the server) | 12:23 |
| jm1 | frenzy_friday: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/46079 | 12:23 |
| jm1 | pojadhav: ^ | 12:23 |
| jm1 | marios: would you mind having a look at this patch? https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/46079 | 12:24 |
| jm1 | marios: its not necessary to read all changes because they were "auto generated" | 12:24 |
| jm1 | marios: but i would like to ask for your approval with the direction of that patch | 12:25 |
| marios | jm1: ack adding to review list but middle sthing right now can it wait or urgent? | 12:26 |
| jm1 | marios: actually we will have to trial and error anyway so i would go for merging it and do a revert if necessary | 12:27 |
| jm1 | marios: changes are already applied in upstream cockpit anyway | 12:27 |
| marios | jm1: it is a scary patch i dont wana | 12:27 |
| marios | jm1: k i will have a look | 12:28 |
| jm1 | marios: yeah its not meant to be read by humans because all those grafana dashboards code is generated by grafana | 12:28 |
| dviroel | jm1: my question is: which environment did you use to get those configurations? | 12:29 |
| jm1 | dviroel: hey :) all those changes where done in our prod upstream grafana | 12:29 |
| jm1 | dviroel: and they are live | 12:29 |
| jm1 | dviroel, marios: so what i did was log into grafana, open each dashboard and just hit save without any changes from my side. | 12:30 |
| jm1 | dviroel, marios: all those changes are because of our upgrade to a recent grafana instance a couple of month ago | 12:30 |
| dviroel | jm1: ok, I just asked because there are some "ids" there, that may be specific for each environment. If you got from prod env, should be fine | 12:31 |
| dviroel | my guess ^ | 12:31 |
| jm1 | dviroel: honestly, i dont like this way of applying changes either but its how its described in our readme and how we always did it :/ | 12:32 |
| jm1 | dviroel, marios: but i will take responsibility for every breakage and fix it if necessary ;) | 12:33 |
| marios | bhagyashris: please help me sanity check remote: SUCCESS | 12:34 |
| marios | remote: | 12:34 |
| marios | remote: https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-config/+/435010 Move mixed OS dlrn re-reporing to a separate playbook [NEW] | 12:34 |
| marios | oops bad paste sorry, but right patch at least :D | 12:34 |
| marios | bhagyashris: once you have a check if you agree we can merge and try it ... main concern is that we don't break anything other than the mixed os job :) wdyt? | 12:34 |
| bhagyashris | marios, yeah ack | 12:35 |
| bhagyashris | looking | 12:35 |
| marios | jm1: thanks... i understand ... i came across this issue when workin on zed. so basically the data format has changed? so anyone tries to check in a new change now breaks the cockpit so we revert | 12:35 |
| marios | jm1: i think your patch is a base upgrade to new format in a working manner, and then we can post stuff ontop? | 12:35 |
| jm1 | marios: exactly | 12:35 |
| jm1 | marios: patches break our prod instances only if not all changes are submitted. but as you see those changes affect even "unrelated" dashboards, so this new base is necessary to at least have a chance of reviewing dashboard changes | 12:37 |
| jm1 | marios, dviroel, dasm, pojadhav, frenzy_friday: we have to remember that each time we upgrade our docker image for grafana, we will have to check whether those updates affect our dashboards. if they do, we have to do the same kind of "upgrade our dashboards without changes" again | 12:38 |
| frenzy_friday | ack | 12:39 |
| marios | jm1: frenzy_friday: fwiw then ack from me https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/46079/1#message-024709575c8e3157c82ae4dc4c5eed5b0c102256 lets try it? | 12:40 |
| dviroel | marios: will merge jm1 patch | 12:41 |
| marios | jm1: do we have enough time to try then revert if needed? | 12:41 |
| marios | otherwise leave to monday ? | 12:41 |
| jm1 | marios: thank you! | 12:41 |
| marios | .. | 12:41 |
| jm1 | dviroel: already done ;) | 12:41 |
| marios | k :) | 12:41 |
| frenzy_friday | marios, yeah revert will be easy | 12:41 |
| marios | frenzy_friday: ack | 12:41 |
| jm1 | marios: no worry, i will monitor that change | 12:41 |
| marios | thanks for working there jm1 frenzy_friday | 12:42 |
| frenzy_friday | jm1, should I remove the skip ansible pull file? | 12:42 |
| frenzy_friday | jm1, should I remove the skip ansible pull file? | 12:42 |
| dviroel | jm1: left my vote there anyway :) | 12:42 |
| jm1 | marios, pojadhav: thats actually the nice thing about our cockpit infra: you cannot break something permanently. if you do, we simply kill all containers, drop all volumes and wait for ansible to restore it | 12:42 |
| jm1 | frenzy_friday: nope, lets wait until patch is merged, then i will kill all containers+volumes and let ansible recreate it from scratch | 12:43 |
| frenzy_friday | cool | 12:43 |
| jm1 | dviroel: thank you :D | 12:43 |
| jm1 | frenzy_friday: can you monitor downstream cockpit? | 12:44 |
| jm1 | frenzy_friday: we might also want to upgrade the downstream dashboards | 12:44 |
| frenzy_friday | jm1, yep | 12:44 |
| dviroel | @team, need to take the kid to the doc, back in 1h-2h | 12:45 |
| *** dviroel is now known as dviroel|doc_appt | 12:45 | |
| dviroel|doc_appt | marios: dasm|off: we should not have the ci podified meeting today | 12:45 |
| marios | dviroel|doc_appt: ack | 12:45 |
| *** frenzy_friday is now known as frenzy_friday|lunch | 12:47 | |
| bhagyashris | marios, agree with your patch | 12:51 |
| bhagyashris | i think we can merge and test | 12:52 |
| marios | bhagyashris: does it make sense? hoping that moving to its own play will mean the dlrn-report role is actually re-executed | 12:53 |
| marios | bhagyashris: ok lets go | 12:53 |
| bhagyashris | marios, yup | 12:53 |
| jm1 | pojadhav: want to apply changes for victoria to our upstream prod cockpit? | 12:54 |
| jm1 | pojadhav: only changes to ui is needed, i will export the changes and prepare a patch for you | 12:54 |
| marios | bhagyashris: k i logged out of the zuul nodes please clear the existing hold unless you want it for someting else? | 12:55 |
| bhagyashris | marios, do you want me to hold node in next test re-run? | 12:55 |
| marios | bhagyashris: and once the patch merges recheck and yeah please new hold in case | 12:55 |
| bhagyashris | ack | 12:55 |
| marios | bhagyashris: but really hoping this one is gonna be good | 12:55 |
| marios | bhagyashris: even though we will force fail | 12:55 |
| bhagyashris | marios, yeah. clearing hold node | 12:56 |
| marios | bhagyashris: will be late for you but I might have time to dig on the node today, otherwise we can rehold again monday | 12:56 |
| bhagyashris | marios, ok np may be i will be around. I will hold node and let's see | 12:57 |
| bhagyashris | marios, patch merged. recheck the testproject. and hold the node Done | 13:00 |
| jm1 | marios, frenzy_friday, pojadhav: looks like upstream cockpit is slowly recovering :) | 13:03 |
| bhagyashris | added you key as well so that you can directly access the node | 13:05 |
| bhagyashris | s/you/your | 13:05 |
| marios | bhagyashris: thanks but i was already on it | 13:06 |
| marios | bhagyashris: oh, did you add mine manually this time? | 13:07 |
| marios | bhagyashris: i could get onto all 3 nodes though, so i think it was already included | 13:07 |
| marios | should not hurt in any case... lets see now would be nice finish for friday | 13:08 |
| * marios prepares sacrifices for zuul | 13:08 | |
| jm1 | marios: fetching data for zed fails when callign this command: ruck_rover.py --influx --release zed --promotion_name current-tripleo-rdo --distro centos-9 | 13:08 |
| jm1 | marios: is this expected? | 13:08 |
| bhagyashris | marios, yeah through patch. i think it will not hurt .... | 13:08 |
| jm1 | marios: do we have any promotions for zed already? if not that would explain why its failing | 13:09 |
| marios | jm1: hmm might be i think we omitted the current-tripleo-rdo promotion criteria ? | 13:09 |
| marios | jm1: we do have promotions for current-tripleo | 13:09 |
| marios | jm1: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/config_environments/rdo/CentOS-9/zed.yaml | 13:09 |
| marios | jm1: so probably we should remove the current-tripleo-rdo from zed queries then my bad | 13:10 |
| marios | jm1: will you post that or should I ? | 13:10 |
| jm1 | marios: will we never get current-tripleo-rdo criteria? | 13:10 |
| marios | jm1: i think we said we will drop those ... if you can make it work for now keep it but if it is breaking can also remove it | 13:10 |
| jm1 | marios: lets fix rr script instead. its perfectly valid to have no current-tripleo-rdo defined, so its a bug in rr script | 13:10 |
| marios | jm1: right now no criteria for current-tripleo-rdo in the promotion criteria file | 13:11 |
| marios | jm1: ok | 13:11 |
| jm1 | marios: ack, thanks for the explaination! | 13:11 |
| marios | thx jm1 | 13:12 |
| jm1 | marios: will zed component dashboard and integration dashboard look exactly like master etc.? | 13:12 |
| marios | jm1: yeah i don't expect/know of any diffs | 13:12 |
| jm1 | marios: ok great! then i will just renamed all stuff in victoria dashboard to zed ;) cc pojadhav | 13:12 |
| marios | jm1: :D ship it | 13:13 |
| pojadhav | jm1, hey sorry | 13:24 |
| pojadhav | i was stepped out to bring some grocery... | 13:24 |
| jm1 | pojadhav: no worry | 13:24 |
| jm1 | pojadhav: i am renaming the victoria stuff to zed | 13:25 |
| marios | jm1: frenzy_friday|lunch: is it normal we have no data? downstream too ? | 13:25 |
| marios | http://tripleo-cockpit.lab4.eng.bos.redhat.com/d/BqAFEazMz/upstream-and-rdo-promotions?orgId=1 | 13:25 |
| jm1 | marios: where is data missing? | 13:25 |
| marios | jm1: ^^ | 13:25 |
| marios | and same upstream one http://dashboard-ci.tripleo.org/d/jwkDMPD4k/upstream-and-rdo-promotions?orgId=1 | 13:26 |
| jm1 | marios: i see data on upstream cockpit. what are you missing? | 13:26 |
| marios | jm1: sorry page refresh was needed | 13:27 |
| marios | my bad false alarm :/ | 13:27 |
| marios | soniya29|rover: akahat|ruck: as discussed earlier would be good to chase hashes a bit today since it is quiet. it will help you on monday. akahat|ruck for example http://promoter.rdoproject.org/promoter_logs/centos9_master.log be5f291ecb22662485815181459d48fa has 3 missing jobs worth a try/testproject? | 13:30 |
| * marios coffee refill brb | 13:30 | |
| pojadhav | jm1, ack | 13:31 |
| pojadhav | jm1, marios : zed started visible on the promotions page :) | 13:39 |
| pojadhav | jm1, thank you | 13:39 |
| marios | cool | 13:41 |
| pojadhav | frenzy_friday|lunch, ^^ | 13:42 |
| jm1 | pojadhav, marios: still fixing bugs. some dashboards still say centos-8 master... | 13:43 |
| pojadhav | jm1, ack | 13:43 |
| soniya29|rover | marios, i tried rekicking but not able to | 13:44 |
| soniya29|rover | https://review.rdoproject.org/r/c/testproject/+/42692 | 13:44 |
| *** frenzy_friday|lunch is now known as frenzy_friday | 13:46 | |
| *** amoralej is now known as amoralej|lunch | 13:47 | |
| frenzy_friday | jm1, pojadhav Upstream cockpit looks good! I am trying the same stuff on internal. Will post tha patch soon | 13:51 |
| jm1 | frenzy_friday, pojadhav, marios: victoria turned into zed :D https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/46081 | 13:54 |
| jm1 | already live in prod upstream cockpit | 13:55 |
| marios | nice one jm1 | 14:00 |
| pojadhav | jm1++ :) | 14:02 |
| jm1 | pojadhav: no thanks to you pojadhav for demostrating how that import, change, export workflow on grafana works :) | 14:04 |
| pojadhav | jm1, :) | 14:05 |
| * pojadhav leaving for the day.. have a great weekend folks..!! | 14:05 | |
| *** dasm|off is now known as dasm | 14:06 | |
| jm1 | pojadhav: thx you too. see you next week :D | 14:06 |
| dasm | o/ | 14:06 |
| dasm | 12:38 <jm1> marios, dviroel, dasm, pojadhav, frenzy_friday: we have to remember that each time we upgrade our docker image for grafana | 14:10 |
| dasm | sounds like at least every major release change | 14:10 |
| dasm | at least | 14:10 |
| jm1 | dasm: yeah | 14:13 |
| jm1 | dasm: hello btw :D | 14:13 |
| jm1 | marios, #oooq: upstream cockpit is reloading once again | 14:16 |
| jm1 | will take a couple of minutes to recover | 14:16 |
| marios | thx | 14:16 |
| * soniya29|rover stepping out for an hour | 14:16 | |
| *** soniya29|rover is now known as soniya29|rover|afk | 14:16 | |
| * jm1 having a break now | 14:18 | |
| *** amoralej|lunch is now known as amoralej | 14:19 | |
| dasm | oooq cockpit recovered. currently it's missing data, but it should be recovered soon | 14:19 |
| dasm | jm1: you fixed the cockpit issue. \o/ cc pojadhav marios | 14:41 |
| marios | yeah nice one | 14:46 |
| *** dviroel|doc_appt is now known as dviroel | 14:49 | |
| frenzy_friday | jm1, hey, I forgot where do I find the .sh file which checks for skip_ansible_pull file? I am checking if we have the same in the internal cockpit | 14:50 |
| dasm | frenzy_friday: the file: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/roles/base/templates/ansible-pull.sh.j2 | 14:51 |
| dasm | its location: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/roles/base/tasks/main.yml#L83 | 14:52 |
| frenzy_friday | dasm, awesome, thanks! | 14:52 |
| marios | folks fyi vexx will be running upgrade so we expect disruption for periodic/rdo jobs | 14:57 |
| marios | akahat|ruck: soniya29|rover|afk: ^^ (rhos-ops) | 14:58 |
| akahat|ruck | marios, ack. Thanks for update. | 14:58 |
| marios | you can follow in #vexxhost | 14:58 |
| marios | akahat|ruck: ^ | 14:58 |
| marios | akahat|ruck: most likely that will happen after we leave/later during the US day | 15:02 |
| marios | fyi dasm dviroel ^^ | 15:02 |
| dasm | ack | 15:02 |
| dviroel | marios: ack, thanks for the heads up | 15:03 |
| akahat|ruck | marios, after few abandon and restore finally testproject is running: https://review.rdoproject.org/r/c/testproject/+/41465 | 15:10 |
| marios | akahat|ruck: k lets see if it makes it before they run the upgrade | 15:12 |
| *** dviroel is now known as dviroel\LUNCH | 15:12 | |
| *** dviroel\LUNCH is now known as dviroel|lunch | 15:12 | |
| akahat|ruck | yeah.. hoping it should get completed before their upgrade. | 15:13 |
| marios | isnt it already late there akahat|ruck soniya29|rover|afk i mean what time does your work day finish? | 15:17 |
| marios | 2047 local? | 15:17 |
| akahat|ruck | marios, i'll be away in 10 mins. | 15:17 |
| akahat|ruck | yeah.. that is correct. | 15:17 |
| marios | k have a good one akahat|ruck o/ | 15:23 |
| dasm | akahat|ruck: o/ take care! | 15:24 |
| akahat|ruck | happy weekend everyone!! | 15:25 |
| * akahat|ruck offline o/ | 15:25 | |
| frenzy_friday | jm1, dasm do you know what would be the easiest way of pushing changes from the prod internal cockpit server? I made some changes there ( a lot of json files) and I do not want to copy them to my local and push the commit. But If I want to git review from the server I have to set up git review with my username/private keys, which might not be the best idea | 15:43 |
| dasm | frenzy_friday: you need to copy them back | 15:44 |
| dasm | that's the easiest | 15:44 |
| dasm | frenzy_friday: the other way is to pass your keys, but it opens you for other issues | 15:45 |
| dasm | *to other issues | 15:45 |
| * marios off in few have a good on oooci o/ | 16:05 | |
| dasm | marios: take care. have a good weekend ;/ | 16:05 |
| dasm | o/ | 16:05 |
| marios | bhagyashris: still waiting https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/stream/5621720b01d04691930b9c04ce4c0d3f?logfile=console.log ;) | 16:06 |
| marios | monday either way but would be nice to know if it worked... | 16:06 |
| marios | you too dasm | 16:06 |
| *** dviroel|lunch is now known as dviroel | 16:08 | |
| marios | so was it a big lunch - looked that way dviroel is now known as dviroel\LUNCH | 16:10 |
| dviroel | lol | 16:10 |
| marios | like i'm not only going for lunch but seriously its gonna be LUNCH | 16:10 |
| dviroel | was hungry | 16:10 |
| marios | yeah it happens | 16:10 |
| *** marios is now known as marios|out | 16:15 | |
| dasm | marios|out: :) | 16:16 |
| frenzy_friday | dasm, jm1 pls take a look at this patch https://code.engineering.redhat.com/gerrit/c/openstack/rrcockpit/+/435064 I have some questions (like how it added 16-full-component-pipeline - I did not add it - was it not comitted already?) etc. Lets discuss on Monday when you guys get some time | 16:29 |
| dasm | frenzy_friday: sure | 16:30 |
| *** soniya29|rover|afk is now known as soniya29|rover | 16:48 | |
| marios|out | bhagyashris: \o/ worked at least reported 8 on second run | 16:52 |
| *** jpena is now known as jpena|off | 16:53 | |
| *** amoralej is now known as amoralej|off | 17:21 | |
| bhagyashris | great! | 17:44 |
| * jm1 out for today, have a nice evening folks o/ | 19:54 | |
| dasm | jm1: o/ | 19:54 |
| jm1 | dasm: o/ | 19:54 |
| * dasm => offline | 22:01 | |
| dasm | o/ | 22:01 |
| *** dasm is now known as dasm|off | 22:01 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!