| *** yadnesh|away is now known as yadnesh | 05:28 | |
| marios | o/ | 05:57 |
|---|---|---|
| *** marios is now known as marios|ruck | 05:57 | |
| bhagyashris | 0/ | 06:37 |
| * bhagyashris lunch brb | 07:08 | |
| bhagyashris | marios|ruck, hey i job is green and nothing failed related to rport to dlrn task but the task skipped due to condition false | 08:06 |
| marios|ruck | bhagyashris: ok i'll have a look - might be what i mentioned during our call (because of config play, so the condition cant come from unmerged/review maybe we need to merge that first the _lower_version thing) | 08:07 |
| bhagyashris | i am not sure but we defined the mixed_os_lower_version in the patch still it skipped | 08:07 |
| bhagyashris | marios|ruck, ahh ok | 08:08 |
| *** yadnesh is now known as yadnesh|afk | 08:09 | |
| jm1 | good morning folks :D | 08:18 |
| arxcruz | moin moin | 08:18 |
| rlandy | hello all | 08:33 |
| marios|ruck | \o | 08:37 |
| chandankumar | o/ | 08:40 |
| arxcruz | chandankumar, if you have time take a look at https://review.opendev.org/c/openstack/tripleo-ci/+/859526 | 08:43 |
| chandankumar | arxcruz: sure will take a look, thank you :-) | 08:44 |
| arxcruz | :* | 08:44 |
| *** yadnesh|afk is now known as yadnesh | 08:53 | |
| * marios|ruck food biab | 09:23 | |
| akahat | o/ | 09:36 |
| *** dviroel_ is now known as dviroel | 09:40 | |
| * dviroel o/ | 09:40 | |
| bhagyashris | marios|ruck, hey should i submit separate patch to add var? | 10:16 |
| marios|ruck | bhagyashris: yeah i think so, i didn't check the logs yet but switching to that in couple mins | 10:20 |
| marios|ruck | bhagyashris: but sure prep patch if you want first we will need to merge it | 10:20 |
| bhagyashris | marios|ruck, fyi https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/434695 | 10:31 |
| marios|ruck | bhagyashris: please check and update that https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/430904/4#message-c0bc1c02e568469a00daf86158943ff4c10332d0 | 10:34 |
| bhagyashris | marios|ruck, ack | 10:36 |
| marios|ruck | bhagyashris: keep both files in that change i mean 8 and 9 | 10:36 |
| bhagyashris | ack | 10:37 |
| marios|ruck | bhagyashris: also i don't think the task was skipped i am still checking https://sf.hosted.upshift.rdu2.redhat.com/logs/44/434544/2/check/periodic-tripleo-ci-rhel-9-8-multinode-mixed-os-rhos-17.1/9c203d5/job-output.txt | 10:39 |
| marios|ruck | bhagyashris: check 2022-11-08 16:02:22.205671 | TASK [Recreate hash_info file for Mixed OS jobs - override distro] | 10:39 |
| bhagyashris | marios|ruck, ohh sorry i was looking at old log | 10:43 |
| bhagyashris | my bad | 10:43 |
| marios|ruck | np trying to see what went wrong there | 10:43 |
| rlandy | bhagyashris: hi - working with Robert on the Jira board now | 10:49 |
| rlandy | bhagyashris: pls can you add as both as board admins | 10:49 |
| rlandy | and remove wes and rafael folco | 10:49 |
| bhagyashris | rlandy, ack | 10:49 |
| rlandy | bhagyashris: probably a good time to add pooja | 10:49 |
| bhagyashris | rlandy, pooja is admin and you too | 10:49 |
| rlandy | can you add robert? | 10:50 |
| rlandy | bhagyashris: https://issues.redhat.com/secure/RapidView.jspa?rapidView=11751 still shows wes and folco | 10:51 |
| rlandy | bhagyashris: I can't configure the board | 10:52 |
| bhagyashris | rlandy, can you please check | 10:54 |
| rlandy | we're there - thanks | 10:54 |
| bhagyashris | rlandy, if you see here https://issues.redhat.com/plugins/servlet/project-config/TRIPLEOCI/roles | 10:55 |
| bhagyashris | you will see jm1 is also admin so not sure the link you share is diffrent settings | 10:56 |
| rlandy | thank you - we cab optimize when I get back | 10:57 |
| pojadhav | bhagyashris, rlandy : looks like sandeep also having admin role | 10:57 |
| rlandy | pojadhav: probably when he was UA | 10:58 |
| pojadhav | rlandy, ahh okay | 10:58 |
| bhagyashris | ack | 10:59 |
| frenzy_friday | pls add to your review lists https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-config/+/434698 | 11:15 |
| reviewbot | I have added your review to the Review list | 11:15 |
| marios|ruck | anyone else problem posting to rdo gerrit? | 11:50 |
| marios|ruck | Problem running 'git remote update gerrit' | 11:50 |
| marios|ruck | Fetching gerrit | 11:50 |
| marios|ruck | ssh: connect to host review.rdoproject.org port 29418: Connection timed out | 11:50 |
| marios|ruck | bhagyashris: please check again like that https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/430904/6#message-033ab32467337550095ff7141fe35270fcd6c7fd | 11:53 |
| bhagyashris | marios|ruck, ok | 11:54 |
| bhagyashris | marios|ruck, recheck the testproject on recent changed in tripleo-env patch https://code.engineering.redhat.com/gerrit/c/testproject/+/434544/2#message-d66e58d9a8f8f78d3f1e641612409a3d5da12860 | 11:57 |
| marios|ruck | bhagyashris: it wont help us for the hash i wanted to test with my changes but now blocked on pushing to ci-config | 11:57 |
| marios|ruck | bhagyashris: can we at least at that one before you rerun https://review.opendev.org/c/openstack/tripleo-ci/+/864105 | 11:58 |
| marios|ruck | bhagyashris: done | 11:58 |
| marios|ruck | bhagyashris: if gerrit comes back i'll update again ... this on ^^^ wil just give us the file might help for debug | 11:58 |
| marios|ruck | bhagyashris: i want to try sthing else too but need ci-config for that | 11:59 |
| bhagyashris | marios|ruck, ack | 12:00 |
| arxcruz | is review.o.o also off for you guys? | 12:04 |
| arxcruz | oh, nevermind, just read marios|ruck message | 12:04 |
| marios|ruck | well looks like upstream gerrit too then not just rdo arxcruz maybe root related | 12:07 |
| marios|ruck | coffee brb | 12:18 |
| marios|ruck | upstream back now arxcruz looks like | 12:26 |
| marios|ruck | but not rdo yet | 12:26 |
| marios|ruck | and rdo back too | 12:42 |
| marios|ruck | bhagyashris: also you moved the _lower var now so this isnt enough by itself https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/430745 | 12:42 |
| marios|ruck | bhagyashris: can you please restore that ^^ and abandon the new one? | 12:43 |
| bhagyashris | 430745 is dependent on new one | 12:45 |
| marios|ruck | bhagyashris: ah you have relation chain... ok but we should abandon this one not needed https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/434695/1 | 12:45 |
| bhagyashris | nw i will abandon new one | 12:45 |
| bhagyashris | ack | 12:45 |
| marios|ruck | bhagyashris: yeah lets keep them together no need for new patch | 12:45 |
| rlandy | dviroel: rcastillo|rover: dasm|off: nhicher has ibm cloud second node ready for testing | 12:45 |
| marios|ruck | thanks, andi can add the ci-config patch for test too | 12:45 |
| rlandy | dviroel: rcastillo|rover: dasm|off: any of you guys available to help him with a quick test of the env? | 12:46 |
| bhagyashris | marios|ruck, should i do now or wait for testproject result which is running currently | 12:46 |
| rlandy | dviroel: rcastillo|rover: dasm|off: on USA time | 12:46 |
| marios|ruck | bhagyashris: now please i want to include test https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/46040 | 12:46 |
| bhagyashris | ack | 12:46 |
| dviroel | rlandy: yeah, I can help | 12:47 |
| bhagyashris | marios|ruck, Done | 12:48 |
| * bhagyashris stepping out for bit | 12:49 | |
| rlandy | dviroel: great - thank you - should be easy - just switching some env params | 12:49 |
| marios|ruck | bhagyashris: thx updated test https://code.engineering.redhat.com/gerrit/c/testproject/+/434544/4#message-62bbd4338afb1022d4cbf26db9d7374c7c31dfc2 | 12:49 |
| marios|ruck | lets see | 12:49 |
| bhagyashris | ack | 12:50 |
| rlandy | dviroel: pls ping nhicher when he is on line | 13:06 |
| dviroel | rlandy: ok, will do | 13:06 |
| *** slaweq_ is now known as slaweq | 14:09 | |
| *** dasm|off is now known as dasm | 14:10 | |
| dasm | o/ | 14:10 |
| jm1 | dasm: o/ | 14:10 |
| * jm1 stepping back for a bit | 14:11 | |
| chandankumar | rlandy: ysandeep|out https://github.com/ceph/ceph-ansible/pull/3576 this is what is possible via zuul and github pr | 14:42 |
| marios|ruck | rcastillo|rover: o/ quick sync in 15 mins ok? | 15:14 |
| rcastillo|rover | marios|ruck: o/ | 15:36 |
| marios|ruck | \o | 15:37 |
| rcastillo|rover | marios|ruck: sync? | 15:39 |
| marios|ruck | rcastillo|rover: sure gimme 5 mins | 15:39 |
| rcastillo|rover | aight | 15:40 |
| marios|ruck | rcastillo|rover: https://meet.google.com/ohe-rbxa-jnn | 15:44 |
| *** dviroel is now known as dviroel|lunch | 15:50 | |
| dasm | jm1: o/ i'm checking your suggestion to use centos8 stream. Which one can install me *all* required components to run ansible with openstack modules? https://paste.opendev.org/show/b9JGVDtzkntIg6ofN7fE/ | 16:11 |
| dasm | cc rcastillo|rover ^ | 16:11 |
| dasm | actually, I might not need openstack components. Just ansible-pull | 16:16 |
| dasm | ansible-core-2.12.2-3.el8.x86_64 : SSH-based configuration management, deployment, and task execution system | 16:16 |
| dasm | Repo : appstream | 16:16 |
| dasm | Matched from: | 16:16 |
| dasm | Filename : /usr/bin/ansible-pull | 16:16 |
| pojadhav | dviroel|lunch, hello | 16:20 |
| pojadhav | I had a discussion with dasm regarding victoria removal from cockpit (upstream-and-rdo-promotions.dashboard). Last time ronelle reverted a patch due to invisibility of promotions data after merging my patch. | 16:22 |
| marios|ruck | bhagyashris: we now have the hash_info at least - it looks right (8 endpoint and hash) but the report is still not using it.. maybe we need to unset the bash vars. anyway will dig more tomorrow just fyi (hash_info there https://sf.hosted.upshift.rdu2.redhat.com/logs/44/434544/4/check/periodic-tripleo-ci-rhel-9-8-multinode-mixed-os-rhos-17.1/2f67635/logs/undercloud/home/zuul/workspace/hash_info.sh ) | 16:24 |
| pojadhav | but patch reverted within short time and also grafana version got changed and influxdb takes time to load data. so dasm suggested to again merge the victoria removal patch and lets wait more time to see results. so I need approval on the patch : https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/46045 | 16:25 |
| marios|ruck | o/ have a good one oooci | 16:29 |
| *** marios|ruck is now known as marios|out | 16:30 | |
| frenzy_friday | pojadhav, ^ sounds good. Maybe we can merge it in the morning and check what happens throughout the day and revert at eod if the data is missing again. Commented on your patch | 16:32 |
| *** yadnesh is now known as yadnesh|away | 16:42 | |
| dasm | jm1: hmm... i need additional modules. | 16:46 |
| dasm | TASK [include base role] ******************************************************* | 16:46 |
| *** dviroel|lunch is now known as dviroel | 16:46 | |
| dasm | ERROR! couldn't resolve module/action 'authorized_key'. This often indicates a misspelling, missing collection, or incorrect module path. | 16:46 |
| jm1 | dasm: simply install ansible from epel-next-release (not ansible-core), this will get you a couple of collections. its similar to 'pip install ansible' | 17:13 |
| jm1 | dasm: in theory, it should be enough to change epel-release to epel-next-release in 'packages'. or add 'epel-next-release' instead of replacing 'epel-release'? | 17:14 |
| dasm | jm1: instead of epel-release or additionally? | 17:14 |
| jm1 | dasm: ah actually both are required: epel-release epel-next-release, https://docs.fedoraproject.org/en-US/epel/#_el8 | 17:15 |
| dasm | k | 17:15 |
| dviroel | pojadhav: i am going to take a look | 17:30 |
| dviroel | pojadhav: sorry for my late ack | 17:31 |
| pojadhav | dviroel, ack | 17:52 |
| dasm | jm1: i finally figured that out. installation order, some workarounds, etc. | 18:29 |
| jm1 | dasm: niceeeeee! | 18:49 |
| jm1 | dasm: how did you manage to setup the staging env without merging that patch first? | 18:50 |
| dasm | jm1: ansible-playbook inventories/inventory.ini provision-all.yml | 18:51 |
| dasm | jm1: the changes are to openstack deployment, hence it's not being pulled to VMs | 18:51 |
| dasm | it's before even running ansible-pull | 18:51 |
| dasm | jm1: fyi, "staging" is only for rhos-dev-stage. More work is coming after that. | 18:53 |
| dasm | Especially to roles and tripleo-infra cloud | 18:53 |
| jm1 | dasm: provision-all.yml => role server_provision => os_server => cloud-init on vms => ansible-pull => master branch of ci-config? | 18:54 |
| jm1 | dasm: so when you run "ansible-playbook inventories/inventory.ini provision-all.yml" then this will create vms on rhos-dev-stage and run ansible-pull on those vms? | 18:55 |
| dasm | ansible-pull does nothing to openstack config | 18:55 |
| dasm | tenant_vars is used during deployment. it's local | 18:55 |
| dasm | VMs are provisioned from patch. | 18:55 |
| jm1 | "VMs are provisioned from patch" < what do you mean? | 18:56 |
| dasm | when you run ansible-playbook it uses tenant_vars and its tenant_vars/rhos-dev-stage/servers.yaml to spin up VMs. | 18:57 |
| dasm | just after that it kicks off 'ansible-pull' and it pulls from the repo | 18:57 |
| dasm | my changes are not related to 'ansible-pull' because VMs are already running. | 18:57 |
| jm1 | dasm: yeah so when you run ansible-playbook on your staging env, then your VMs on that staging env currently pull the master branch of ci-config | 18:59 |
| jm1 | dasm: .. not the code from your patch | 18:59 |
| dasm | no | 18:59 |
| dasm | VMs are configured based on the patch | 18:59 |
| jm1 | dasm: why? how? | 18:59 |
| dasm | because ansible-playbook executes playbooks locally | 19:00 |
| dasm | not on VMs | 19:00 |
| jm1 | dasm: yes but your VMs will run ansible-pull | 19:00 |
| dasm | it's irrelevant to this patch | 19:01 |
| jm1 | dasm: why? isn't the idea of a staging env to test changes to the repo before merging the patches? | 19:01 |
| dasm | ansible-pull runs within VMs | 19:01 |
| dasm | jm1: baby steps. | 19:02 |
| dasm | i need to prepare staging env before introducing further changes | 19:02 |
| jm1 | dasm: ok | 19:03 |
| jm1 | dasm: what you could do to try to test your patch in your staging env is: | 19:05 |
| jm1 | dasm: ansible-playbook inventories/inventory.ini provision-all.yml -e "infra_setup_repo='https://review.rdoproject.org/r/rdo-infra/ci-config -C refs/changes/01/46001/4' | 19:05 |
| jm1 | dasm: missing " at the end | 19:06 |
| jm1 | ansible-playbook inventories/inventory.ini provision-all.yml -e "infra_setup_repo='https://review.rdoproject.org/r/rdo-infra/ci-config -C refs/changes/01/46001/4'" | 19:06 |
| dasm | "ansible-playbook -i [...]" | 19:11 |
| dasm | jm1: i'm giving it a try... however iirc, it won't work | 19:13 |
| jm1 | dasm: why? | 19:13 |
| dasm | because "infra_setup_repo" is loaded in playbook, so it overwrites "-e" | 19:13 |
| dasm | i had this issue before | 19:13 |
| dasm | hence my work to change that behavior | 19:13 |
| jm1 | dasm: -e variables have highest prio in ansible, you cannot overwrite them in playbooks. https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable | 19:14 |
| dasm | no | 19:14 |
| dasm | i mean, you're right... but. | 19:14 |
| dasm | this: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/provisioner_prepare.yml#L28 | 19:14 |
| dasm | overwrites values | 19:14 |
| jm1 | dasm: it will not overwrite infra_setup_repo | 19:15 |
| jm1 | ..when specified at the command line | 19:15 |
| jm1 | dasm: but there will be all kinds of problems, no doubt :D which is why your staging env is so important :) | 19:16 |
| dasm | jm1: you're right. i see it here | 19:17 |
| dasm | "ansible-pull -d /tmp/ci-config --clean -U https://review.rdoproject.org/r/rdo-infra/ci-config -C refs/changes/01/46001/4 ci-scripts/infra-setup/servers_setup.yml 2>&1" | 19:17 |
| dasm | when i did "-e @staging.yaml" my staging.yaml was completely ignored... that what i thought so | 19:17 |
| dasm | maybe it was not ignored, but just missed some vars? idk | 19:18 |
| dasm | jm1: either way, i'm running deployment right now | 19:18 |
| jm1 | dasm: great!!! | 19:18 |
| dasm | jm1: "there must be a better way!" | 19:19 |
| dasm | but... baby steps. one step at a time | 19:19 |
| jm1 | dasm: your patch allows us to build a ci job that verifies our ci-config patches :D | 19:19 |
| jm1 | dasm: true, baby steps | 19:20 |
| dasm | jm1: it just takes so much time :) | 19:20 |
| jm1 | dasm: hehe but its worth it | 19:20 |
| dasm | and i'm gonna rework chain of changes | 19:20 |
| dasm | jm1: fyi | 19:24 |
| dasm | | 30fe353f-fb4c-40be-b26b-d908dcd47eaa | toolbox | ACTIVE | private=10.0.11.161, 192.168.100.241 | CentOS-Stream-8-x86_64-GenericCloud | ci.m1.medium | | 19:24 |
| dasm | | 29acf82b-7578-41c6-b0e7-e453b8a47d0f | rrcockpit | ACTIVE | private=10.0.11.198, 192.168.100.240 | CentOS-7-x86_64-GenericCloud-1907-raw | ci.memory.xl | | 19:24 |
| dasm | | 93e1813b-0fdb-4d13-8752-7e7f226baecf | promoter | ACTIVE | private=10.0.11.178, 192.168.100.42 | CentOS-7-x86_64-GenericCloud-1907-raw | ci.m1.xlarge | | 19:24 |
| jm1 | dasm: gimme a sec, i will try to log in | 19:26 |
| jm1 | dasm: looks like they are still installing. not able to login, permission denied | 19:27 |
| jm1 | dasm: oh its in the upgrade step. nice! | 19:29 |
| jm1 | dasm: installing ansible right now | 19:30 |
| jm1 | dasm: in theory it installed my ssh key, but i am still not able to login.. | 19:32 |
| jm1 | dasm: oh with centos user its possible | 19:32 |
| jm1 | dasm: probably have to wait 5 minutes before ansible-pull runs again | 19:33 |
| jm1 | dasm: your ssh comment says "dasm@dhcp-192-168-104-111" << maybe you could change that to your rh mail so people in future know who's key that is :D | 19:37 |
| *** dviroel is now known as dviroel|biba | 19:51 | |
| jm1 | dasm: ansible-pull failed with: "error: pathspec 'refs/changes/01/46001/4' did not match any file(s) known to git" | 19:51 |
| *** dviroel|biba is now known as dviroel|biab | 19:51 | |
| dasm | jm1: heh | 19:51 |
| dasm | jm1: which comment? | 19:51 |
| jm1 | dasm: comment at the end here: https://github.com/rdo-infra/ci-config/blob/71bbf0acb0a8774ad6746e88b6125b2978c2d71a/ci-scripts/infra-setup/tenant_vars/common.yml#L91 | 19:52 |
| dasm | it doesn't matter what's there | 19:53 |
| dasm | we can change that tho | 19:53 |
| jm1 | dasm: it helps identifying who is who ;) | 19:53 |
| dasm | i am who i am :) | 19:53 |
| jm1 | dasm: sure! but who is raukadah? | 19:54 |
| jm1 | dasm: its chandan ;) | 19:54 |
| dviroel|biab | LOL | 19:54 |
| jm1 | dasm: for the staging dev we need a way to test patches. since ansible-pull does not allow us to specify gerrit patches, we might want to replace ansible-pull with git+ansible-playbook? | 19:56 |
| dasm | hm | 19:57 |
| dasm | jm1: it's chicken and egg problem here | 19:58 |
| dasm | or maybe not? | 19:58 |
| dasm | hmm | 19:58 |
| jm1 | dasm: it is. so i would implement it in your patch | 19:58 |
| jm1 | dasm: or better you do ;) | 19:58 |
| dasm | jm1: tbh this patch is only about prepping staging env, not prepping way to test that ;) | 19:58 |
| jm1 | dasm: ok np. we can do it in a follow up | 20:02 |
| jm1 | dasm: its just that we cannot run a full deployment atm. we will have to log into each vm and run git+ansible manually to test patches | 20:04 |
| jm1 | dasm: but ok for now :D | 20:04 |
| dasm | jm1: which deployment? staging or production? | 20:04 |
| jm1 | dasm: its cool that we actually have a staging env now | 20:04 |
| jm1 | dasm: staging | 20:04 |
| dasm | we can easily teardown staging without affecting others. just: 'openstack server delete toolbox rrcockpit promoter" and then ansible-playbook | 20:05 |
| dasm | it's already better than what we had few weeks ago | 20:05 |
| jm1 | dasm: sure, its great! GREAT :D | 20:06 |
| jm1 | dasm: no, really. before we could not test anything, now we can | 20:07 |
| dasm | jm1: more work is needed though | 20:07 |
| jm1 | dasm: do you want to keep your patch as it is for now or are you still working on it? | 21:09 |
| jm1 | dasm: damn, found another thing that blocks us. added two new comments to that patch. the current playbook/inventory/roles structure is really really REALLY inflexible :( | 21:37 |
| jm1 | dasm: simply adding another tenant turns out to be not so simple :( | 21:37 |
| * jm1 out for today, have a nice evening folks | 21:41 | |
| dasm | jm1[m]: the changes are here temporarily. I need to start somewhere, before I can introduce more updates. | 22:38 |
| *** dasm is now known as dasm|offp | 23:03 | |
| *** dasm|offp is now known as dasm|off | 23:03 | |
| dasm|off | o/ | 23:03 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!