| *** ChanServ changes topic to "Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: http://bit.ly/osa-review-board-v4_1" | 00:44 | |
| moha7 | jrosser: I still get the error on Ubuntu: http://ix.io/4jb7 by running `/opt/openstack-ansible/scripts/bootstrap-ansible.sh` | 07:08 |
|---|---|---|
| moha7 | Also have an error on `openstack-ansible /opt/openstack-ansible/playbooks/setup-openstack.yml --syntax-check`: https://p.teknik.io/D8qoN | 07:09 |
| noonedeadpunk | moha7: hm, I think I know what could be the reason | 08:39 |
| jrosser | morning | 08:39 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible stable/zed: Unset OSA-defined variables for bootstrap https://review.opendev.org/c/openstack/openstack-ansible/+/868270 | 08:47 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible stable/xena: Unset OSA-defined variables for bootstrap https://review.opendev.org/c/openstack/openstack-ansible/+/868271 | 08:48 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible stable/zed: [doc] Fix document on dymanic_inventory tox usage https://review.opendev.org/c/openstack/openstack-ansible/+/868272 | 08:49 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/yoga: Unset OSA-defined variables for bootstrap https://review.opendev.org/c/openstack/openstack-ansible/+/868273 | 09:11 |
| noonedeadpunk | sorry moha7 I was side-pinged and got distracted | 09:12 |
| noonedeadpunk | I think that's the patch that should fix the issue you have https://opendev.org/openstack/openstack-ansible/commit/f933194813de9d18b89040fa8c5b80bcd0dd967c | 09:12 |
| noonedeadpunk | We haven't released it yet. Likely I should prepare bump and release new version for Yoga | 09:13 |
| jrosser | do we need to backport this https://review.opendev.org/c/openstack/openstack-ansible/+/865952 | 09:16 |
| noonedeadpunk | good question | 09:17 |
| jrosser | yes i think we do https://zuul.opendev.org/t/openstack/build/6e8c5d55e15d47e3a25849e7f0b5a6b3 | 09:17 |
| noonedeadpunk | I think yes? At least to Yoga? As I'm not sure we used integrated tests for zfs before | 09:17 |
| jrosser | thats from stable/yoga | 09:17 |
| jrosser | was just looking at the outstanding stable branch patches if we are to make some releases | 09:18 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible stable/yoga: Install ZFS packages for bootstrap-host if needed https://review.opendev.org/c/openstack/openstack-ansible/+/868274 | 09:18 |
| noonedeadpunk | damn, how to make both zuul and me happy :D | 09:28 |
| noonedeadpunk | (with regards to collection names) | 09:28 |
| noonedeadpunk | or maybe reject base on smth else.... | 09:28 |
| noonedeadpunk | but still merge should be done properly.... | 09:29 |
| jrosser | i guess we could have condition: false | 09:30 |
| jrosser | but maybe new/random keys will upset galaxy | 09:30 |
| jrosser | thats maybe not so bad actually, the pattern we use with zuul_item is quite nice to copy only some keys from input to output | 09:32 |
| noonedeadpunk | maybe create mapping for zuul projects / collection names | 09:32 |
| jrosser | we could have orig_item and omit condition: from both of them in the output | 09:32 |
| noonedeadpunk | Well. Then to override some collection location you need to list all current collections with condition and then define ones you want to have | 09:33 |
| jrosser | maybe this? https://paste.opendev.org/show/be2ubQdEzHPG4a9tkN1H/ | 09:35 |
| jrosser | condition is optional | 09:35 |
| noonedeadpunk | but in zuul we have quite finite number of collections, so maybe such mapping also not that bad... | 09:35 |
| noonedeadpunk | So let me explain problem better maybe? So current situation is with input like that - https://paste.opendev.org/show/b6PdgQN1vSoR6I3k81q5/ our play will merge these 2 lists, and since we merge by name, and name is different, then it will try to install from both sources | 09:41 |
| jrosser | right, yes, so the first patch was to regularise 'name' and allow source to be different | 09:43 |
| noonedeadpunk | So with condition you will need to mention every collection from original file with `condition: false` and then add your sources, right? | 09:43 |
| jrosser | oh well i was thinking that the override of local collections would happen first | 09:43 |
| jrosser | so first replace the ones from the original list where "name" matches | 09:43 |
| jrosser | including local ones, and ones you want to condition: false | 09:44 |
| noonedeadpunk | ah | 09:44 |
| noonedeadpunk | ok | 09:44 |
| noonedeadpunk | gotcha | 09:44 |
| noonedeadpunk | (I think) | 09:44 |
| jrosser | :) | 09:44 |
| jrosser | so the first patch is good - to make 'name' be not the url | 09:45 |
| noonedeadpunk | well, in https://paste.opendev.org/show/be2ubQdEzHPG4a9tkN1H/ there's a problem as you iterate only over zuul.results (basically find on localhost) | 09:46 |
| noonedeadpunk | so you can't set item.item.condition | 09:46 |
| noonedeadpunk | But you still can rejectattr based on that | 09:46 |
| noonedeadpunk | (I think) | 09:46 |
| noonedeadpunk | ok, yeah, need to check how galaxy is happy/unhappy if we'll add another key | 09:47 |
| jrosser | ok | 09:48 |
| jrosser | also i wonder really if we should add another key, making it not be a standard galaxy requirements file | 09:49 |
| noonedeadpunk | yeah, that was kind of why I was thinking about using what we have.... | 09:49 |
| noonedeadpunk | Maybe setting type: absent ?:) | 09:49 |
| noonedeadpunk | or better type: skip | 09:49 |
| jrosser | perhaps what this means is that we should have a completely standard requirements file for collections and roles | 09:50 |
| jrosser | but somewhere else give a list of things to skip | 09:50 |
| noonedeadpunk | I'm not sure that for roles we do have as of today | 09:51 |
| jrosser | so then its maybe more a case of when: {{ item.name }} not in collections_to_skip | 09:51 |
| noonedeadpunk | also, we skip roles from ansible-collection-requirements | 09:51 |
| noonedeadpunk | because we do that https://opendev.org/openstack/openstack-ansible/src/branch/master/scripts/get-ansible-collection-requirements.yml#L76 | 09:52 |
| noonedeadpunk | oh, I don't really want to have yet another file where to define what to skip tbh | 09:53 |
| jrosser | can this all be done here? | 09:54 |
| jrosser | https://opendev.org/openstack/openstack-ansible/src/branch/master/scripts/get-ansible-collection-requirements.yml#L36-L48 | 09:54 |
| jrosser | step one takes the original list and removes anything overridden | 09:54 |
| jrosser | step 2 adds in the overrides | 09:55 |
| jrosser | so if step 2 was conditional also on some key that would give the function you want? | 09:55 |
| noonedeadpunk | So the problem that I want to skip roles that are added in step 1 by data that we iterate over in step 2 | 09:57 |
| noonedeadpunk | So easiest thing would be to merge the lists and then remove data based on some condition imo | 09:58 |
| jrosser | https://paste.opendev.org/show/b7FvkXhHqbTSaniQSJfm/ | 09:59 |
| jrosser | tbh this is all a bit ugly with all the with_items as it is | 10:00 |
| noonedeadpunk | I don't want to skip what's in user-collection-requirements, I want to avoid isntalling what's in ansible-collection-requirements | 10:00 |
| noonedeadpunk | For example - I don't want to have gluster collection | 10:00 |
| jrosser | but you'd do that by making an entry in user-collection-requirements with condition: false | 10:00 |
| noonedeadpunk | But that will only prevent the record from user-collection-requirements to be added to galaxy_collections_list | 10:01 |
| noonedeadpunk | it will not drop item from what's already in there | 10:01 |
| noonedeadpunk | as gluster is in required_collections and it's in galaxy_collections_list. And on step2 we only add user_collections (or not adding it) | 10:02 |
| noonedeadpunk | So. rejectattr :) | 10:02 |
| noonedeadpunk | ok, let me explain with some paste :) | 10:03 |
| jrosser | (i am assuming you use your first patch which makes name: be actually the name not the url btw) | 10:03 |
| noonedeadpunk | So that's simplified input https://paste.opendev.org/show/bsL7o6fBae9lWNNbDKeB/ | 10:06 |
| jrosser | yes | 10:06 |
| noonedeadpunk | As a result I want to have only openstack.cloud with source from own git repo | 10:06 |
| noonedeadpunk | Well. Actually. Now thinking about it I wonder if that's gonna work at all.... | 10:06 |
| jrosser | this task takes out of the original list anything you override? https://opendev.org/openstack/openstack-ansible/src/branch/master/scripts/get-ansible-collection-requirements.yml#L36-L41 | 10:07 |
| noonedeadpunk | oh | 10:08 |
| noonedeadpunk | now I see | 10:08 |
| jrosser | i do wonder if we could do this more nicely | 10:08 |
| jrosser | if the 'name' field was not the url but actually the name then some of the ansible filters might be actually usable here | 10:09 |
| noonedeadpunk | So if name in user requirements it won't be added, and then we also omit adding from user. | 10:09 |
| noonedeadpunk | yeah, true, you're right | 10:09 |
| jrosser | yeah | 10:09 |
| jrosser | and i think we can possibly improve the code to remove the loops as a second step | 10:09 |
| jrosser | but i have not really thought too much about that | 10:09 |
| noonedeadpunk | But well | 10:10 |
| noonedeadpunk | I t still doesn't resolve your comment about zuul :D | 10:10 |
| jrosser | ! | 10:11 |
| noonedeadpunk | As now for https://review.opendev.org/c/openstack/openstack-ansible/+/868205/1/ansible-collection-requirements.yml zuul won't apply override properly | 10:11 |
| jrosser | right - i was wondering how it worked at all | 10:11 |
| noonedeadpunk | hm | 10:11 |
| noonedeadpunk | true.... | 10:12 |
| jrosser | or maybe now it doesnt use the local zuul repos or something, but i was a bit surprised it didnt just fail completely | 10:12 |
| noonedeadpunk | To be fair, I wasn't expecting to be stuck with isolated env that early for that long... | 10:14 |
| noonedeadpunk | It's quite good practice though | 10:14 |
| jrosser | once you get past this it should be much easier | 10:17 |
| noonedeadpunk | But actually for skipping repos I'm not sure if the playbook won't fail because of missing collection here https://opendev.org/openstack/openstack-ansible/src/branch/master/playbooks/repo-install.yml#L48-L55 | 10:17 |
| noonedeadpunk | With inlcude I guess it should not.... | 10:17 |
| noonedeadpunk | But I haven't tried yet... | 10:17 |
| noonedeadpunk | So maybe this idea is just useless | 10:17 |
| jrosser | thats the role from plugins though isnt it | 10:18 |
| jrosser | i wonder how far down it attempts to resolve all this | 10:18 |
| noonedeadpunk | well, when checking setup-everything with verbose you see it tries to evaluate all include_vars at least for all playbooks in there | 10:19 |
| noonedeadpunk | so eventually it also checks for roles, but does it evaluate conditions.... | 10:20 |
| noonedeadpunk | no idea | 10:20 |
| jrosser | i'm just testing what happens | 10:20 |
| jrosser | i think you will need the glusterfs collection | 10:25 |
| noonedeadpunk | yeah.... | 10:29 |
| noonedeadpunk | Then likely second part can be jsut abandoned | 10:29 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Define name for all collections in a-r-r https://review.opendev.org/c/openstack/openstack-ansible/+/868205 | 10:35 |
| noonedeadpunk | ok, but ^ is easy/good enough I guess | 10:36 |
| noonedeadpunk | It's sad though that tons of crap must be cloned/mainatained for no good reason | 10:37 |
| jrosser | well, you already need a git mirror with * openstack service repos on for a isolated deployment anyway | 10:38 |
| jrosser | if thats automated then adding more repos to it's list should be ok | 10:38 |
| noonedeadpunk | yeah, true | 10:40 |
| noonedeadpunk | I just for some reason thought that there's space for improvement, but apparently it's not | 10:42 |
| jrosser | really ansible misses some kind of preprocessor thing | 10:44 |
| jrosser | #ifdef GLUSTERFS | 10:45 |
| noonedeadpunk | yeah... | 10:51 |
| noonedeadpunk | also I've spotted infinite loop (or smth like that) in dynamic inventory, if wrongly define used_ips (ie make used out of range for the network) | 10:55 |
| noonedeadpunk | haven't looked depeer though | 10:55 |
| *** dviroel|out is now known as dviroel | 11:01 | |
| noonedeadpunk | And TOX_CONSTRAINTS_FILE that is used for bootstrap is a bit hard to maintain.. | 11:05 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Prevent bootstrap failure when all roles/collections are overriden https://review.opendev.org/c/openstack/openstack-ansible/+/868312 | 12:41 |
| noonedeadpunk | one more nasty thing ^ | 12:41 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Prevent bootstrap failure when all roles/collections are overriden https://review.opendev.org/c/openstack/openstack-ansible/+/868312 | 12:52 |
| jrosser | argh centos-9-stream has conflicting python packages | 12:59 |
| noonedeadpunk | sweet..... | 12:59 |
| jrosser | all hitting RETRY_LIMIT | 13:00 |
| noonedeadpunk | looks like mirrors are out of sync or smth... | 13:01 |
| noonedeadpunk | I can recall switching mirrors for centos on infra side... I wonder if that has shoot again | 13:01 |
| spatel | jamesdenton morning | 14:08 |
| spatel | ping me if you around, i have ovn + neutron question | 14:08 |
| mgariepy | what's the questions ? | 14:09 |
| mgariepy | might not have the answer but.. i'm interested anyway :D | 14:09 |
| spatel | In ovn i have assigned nic to ovn like physnet1:br-ex as a flat network (example br-ex map to eth1). | 14:18 |
| spatel | Now i want to create vlan base provider and map with eth2 | 14:19 |
| spatel | This is what i have currently in ovs - external_ids : {ovn-bridge-mappings="physnet1:br-ex" | 14:20 |
| spatel | if i want to add second nic eth2 then how does that entry will looks ? | 14:20 |
| spatel | https://paste.opendev.org/show/beHSmvKZVlVzoBh7IoEr/ | 14:22 |
| spatel | how does ovn know about physnet2 and do does it add entry in ovn-bridge-mappings? | 14:22 |
| mgariepy | spatel, maybe it's covered in this patch i haven't had the chance to review it yet. https://review.opendev.org/c/openstack/openstack-ansible/+/867577 | 14:29 |
| spatel | hmm | 14:32 |
| *** dviroel is now known as dviroel|lunch | 15:03 | |
| noonedeadpunk | I wonder how relevant this warning is: https://opendev.org/openstack/openstack-ansible/src/branch/master/etc/openstack_deploy/user_secrets.yml#L16-L20 | 15:19 |
| noonedeadpunk | As I bet we do change passowrds in quite a while. Though roles might indeed be no adopted enough to minimize downtime when doing so | 15:19 |
| mgariepy | the comments are also lost on pw-gen irrc. | 15:23 |
| noonedeadpunk | yup | 15:24 |
| mgariepy | so it's probably never have been read. | 15:24 |
| jrosser | i don't really know the internals of the keystone client enough to know if it uses the username/password every time or if theres a token with some $lifetime created | 15:24 |
| mgariepy | beside by you ;p haha | 15:24 |
| noonedeadpunk | I wonder if we should set more pw length by default then 16 | 15:24 |
| noonedeadpunk | from what I know it does ask for token with each command | 15:26 |
| jrosser | thats a bit unfortunate for rotating the password | 15:26 |
| noonedeadpunk | but then we're talkign specifically about `keystone_auth_admin_password` that can't be rotated? | 15:26 |
| noonedeadpunk | or well, and galera_root_password | 15:27 |
| noonedeadpunk | all other service/db/rabbit might be rotated safely? | 15:27 |
| jrosser | why is `keystone_auth_admin_password` so hard? | 15:27 |
| noonedeadpunk | well, it's used only during keystone bootstrap and then in openrc role | 15:28 |
| noonedeadpunk | So you need to kind of manually rotate it, write to secrets.yml and run utility to place updated to the openrc/clouds.yaml | 15:29 |
| jrosser | yes indeed | 15:30 |
| noonedeadpunk | jrosser: if you asked about keystone_authtoken (or well, keystone client inside services), then they should cache tokens in memcached | 15:30 |
| noonedeadpunk | given they're properly configured and it's working as intended | 15:30 |
| jrosser | right so i was thinking if we rotate password for nova or something then do you have whatever the timeout is for those tokens to get it written to all the config files | 15:31 |
| jrosser | well, between zero and timeout really | 15:32 |
| noonedeadpunk | but still we kind of limit effect of that. So potential downtime for service will be between update for <service>_all[0] and until play will end <service>_all[0] and disable rest on haproxy. | 15:32 |
| noonedeadpunk | ah, nova/neutron are good points | 15:32 |
| noonedeadpunk | But after all I think we should have tag to run only service-setup, aren't we? | 15:32 |
| jrosser | if everything is using a cached token then you give it the new password, restart and it gets a new token? | 15:33 |
| noonedeadpunk | `common-service` | 15:33 |
| noonedeadpunk | I'd say it should, not sure how it identifies what's relevant and what's not though | 15:34 |
| jrosser | would certainly be interesting to investigate in an AIO | 15:37 |
| jrosser | changing the password for some services and seeing if/how they break | 15:37 |
| *** dviroel|lunch is now known as dviroel | 16:29 | |
| spatel | I had lots of issue when i changed admin password, i wish there is a tag which just touch all services and replace password and reload them | 20:16 |
| noonedeadpunk | but admin password is not used anywhere, except openrc files | 20:20 |
| noonedeadpunk | so it's only affecting utility container | 20:20 |
| mgariepy | hey, it's my last day for this year i'll be back in january | 20:21 |
| noonedeadpunk | nice, have great holidays mgariepy! | 20:25 |
| mgariepy | you too guys, try to take some time off as well :D | 20:26 |
| mgariepy | so see you guys in 2023! | 20:31 |
| mgariepy | take care. | 20:31 |
| ElDuderino | Random question, but years ago around kernel 3.13 there was that conntrack change, right? As a result, MH commented and worked on it with a reporter: https://bugs.launchpad.net/openstack-ansible/+bug/1579963/comments/6. My question is, 6 years later, should I still see 'net.netfilter.nf_conntrack_max' entries in sysctl if it's all 'br_netfilter' these days? | 21:05 |
| *** dviroel is now known as dviroel|out | 21:26 | |
| noonedeadpunk | ElDuderino: I do remember that we handled that code somewhere in openstack_hosts role indeed, but dropped as no current distributions has kernel that old | 21:35 |
| noonedeadpunk | Like that https://opendev.org/openstack/openstack-ansible-openstack_hosts/src/tag/queens-eol/vars/redhat-7.yml#L27 | 21:37 |
| noonedeadpunk | And there were some other bits somewhere | 21:37 |
| noonedeadpunk | But I'm quite sure it's not relevant anymore | 21:37 |
| ElDuderino | gotcha, thanks @noonedeadpunk | 22:15 |
| ElDuderino | (thought so) | 22:15 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!