| *** cshen has joined #openstack-ansible | 00:14 | |
| *** cshen has quit IRC | 00:19 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_murano stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736356 | 00:34 |
|---|---|---|
| openstackgerrit | Merged openstack/openstack-ansible-os_murano master: Updated from OpenStack Ansible Tests https://review.opendev.org/734202 | 00:34 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ironic master: inspector: fix service catalog creation https://review.opendev.org/735177 | 00:38 |
| openstackgerrit | Merged openstack/openstack-ansible-os_horizon stable/ussuri: Add octavia version to constraints https://review.opendev.org/735943 | 00:53 |
| *** tinwood has quit IRC | 02:08 | |
| *** tinwood has joined #openstack-ansible | 02:10 | |
| *** cshen has joined #openstack-ansible | 02:15 | |
| *** cshen has quit IRC | 02:19 | |
| *** spatel has joined #openstack-ansible | 02:20 | |
| *** spatel has quit IRC | 02:55 | |
| *** spatel has joined #openstack-ansible | 03:16 | |
| *** markvoelker has quit IRC | 03:49 | |
| openstackgerrit | Satish Patel proposed openstack/openstack-ansible-os_horizon master: Add Centos-8 support https://review.opendev.org/736095 | 03:57 |
| *** cshen has joined #openstack-ansible | 04:15 | |
| *** cshen has quit IRC | 04:20 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #openstack-ansible | 04:33 | |
| *** cshen has joined #openstack-ansible | 04:45 | |
| *** cshen has quit IRC | 04:49 | |
| *** spatel has quit IRC | 04:53 | |
| *** dpaclt has joined #openstack-ansible | 05:20 | |
| *** shyamb has joined #openstack-ansible | 05:33 | |
| *** udesale has joined #openstack-ansible | 05:42 | |
| *** markvoelker has joined #openstack-ansible | 05:51 | |
| *** markvoelker has quit IRC | 05:55 | |
| *** markvoelker has joined #openstack-ansible | 06:19 | |
| *** markvoelker has quit IRC | 06:24 | |
| *** yolanda has quit IRC | 06:31 | |
| *** shyamb has quit IRC | 06:31 | |
| *** miloa has joined #openstack-ansible | 06:34 | |
| *** yolanda has joined #openstack-ansible | 06:36 | |
| *** noonedeadpunk has joined #openstack-ansible | 06:41 | |
| *** rpittau|afk is now known as rpittau | 06:43 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-tests stable/train: Add pre-gate cleanup tasks https://review.opendev.org/736464 | 06:45 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible stable/train: Bump ansible version to 2.8.12 https://review.opendev.org/736465 | 06:48 |
| openstackgerrit | jiasirui proposed openstack/ansible-config_template master: Missing document start "---" https://review.opendev.org/736466 | 06:49 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-tests stable/train: Bump ansible version to 2.8.12 https://review.opendev.org/736467 | 06:49 |
| *** cshen has joined #openstack-ansible | 07:05 | |
| noonedeadpunk | gixx: so repo for adjutant has been created https://opendev.org/openstack/openstack-ansible-os_adjutant | 07:24 |
| *** tosky has joined #openstack-ansible | 07:25 | |
| gixx | Nice, now for the cleanup work and make it buildable in latest :) | 07:31 |
| gixx | I won't have much time to help this week (holiday in Sweden tomorrow, busy today). I'll try and put more focus next week | 07:36 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ironic stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736351 | 07:47 |
| openstackgerrit | Merged openstack/openstack-ansible-memcached_server stable/ussuri: remove ansible_python_interpreter https://review.opendev.org/736208 | 07:57 |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736348 | 07:58 |
| openstackgerrit | Merged openstack/openstack-ansible-os_mistral stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736355 | 07:59 |
| openstackgerrit | Merged openstack/openstack-ansible-os_rally stable/ussuri: remove ansible_python_interpreter https://review.opendev.org/736211 | 08:02 |
| openstackgerrit | Merged openstack/openstack-ansible-os_heat stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736350 | 08:02 |
| openstackgerrit | Merged openstack/openstack-ansible-os_designate stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736347 | 08:05 |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736357 | 08:07 |
| openstackgerrit | Merged openstack/openstack-ansible-os_barbican stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736341 | 08:11 |
| *** shyamb has joined #openstack-ansible | 08:11 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_tacker stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736364 | 08:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_blazar stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736342 | 08:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736344 | 08:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_placement stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736361 | 08:13 |
| *** jbadiapa has joined #openstack-ansible | 08:13 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_octavia stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736359 | 08:14 |
| openstackgerrit | Merged openstack/ansible-role-systemd_mount stable/ussuri: remove ansible_python_interpreter https://review.opendev.org/736202 | 08:15 |
| jrosser | good morning | 08:15 |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736352 | 08:16 |
| jrosser | noonedeadpunk: you remember we talked about config_template and needing to pin it | 08:16 |
| noonedeadpunk | yep | 08:17 |
| jrosser | do you remember anything about it ? :) | 08:17 |
| noonedeadpunk | but we ended up that we need to describe how ci works | 08:17 |
| noonedeadpunk | as it's the root cause | 08:17 |
| *** arkan has joined #openstack-ansible | 08:17 | |
| jrosser | yeah it did sort of spiral into that | 08:17 |
| jrosser | but in the immediate term stable/train is totally not working | 08:18 |
| jrosser | and it would be nice just to check with you whats causing that | 08:18 |
| noonedeadpunk | eventually, just pinning not working at the moment, and there were several options to fix that... | 08:18 |
| *** udesale_ has joined #openstack-ansible | 08:19 | |
| noonedeadpunk | We just use what's zuul is providing. and it provides us either with the same branch or with master | 08:20 |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736362 | 08:20 |
| openstackgerrit | Merged openstack/openstack-ansible-os_masakari stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736354 | 08:20 |
| jrosser | do you think it is possible to make config_template aware of the ansible version? | 08:21 |
| jrosser | and keep both code paths until we sort out the wider CI issue | 08:21 |
| noonedeadpunk | we can remove it from required-projects I guess | 08:21 |
| *** udesale has quit IRC | 08:21 | |
| noonedeadpunk | but then depend-os won't work | 08:21 |
| noonedeadpunk | os set url to github | 08:22 |
| noonedeadpunk | but eventually... | 08:22 |
| jrosser | well | 08:22 |
| noonedeadpunk | I think we should just fix ci asap... | 08:22 |
| openstackgerrit | Merged openstack/ansible-role-systemd_networkd stable/ussuri: remove ansible_python_interpreter https://review.opendev.org/736201 | 08:22 |
| openstackgerrit | Merged openstack/ansible-role-python_venv_build stable/ussuri: remove ansible_python_interpreter https://review.opendev.org/736200 | 08:22 |
| noonedeadpunk | as any of these options touches CI | 08:22 |
| jrosser | ok thats cool - but you are right then we must do it asap | 08:23 |
| openstackgerrit | Merged openstack/openstack-ansible-os_gnocchi stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736349 | 08:23 |
| noonedeadpunk | ok, then I'll take a look today on it | 08:26 |
| *** maharg101 has quit IRC | 08:27 | |
| noonedeadpunk | I was just kinda trying to understand if we _really_ want to get SHA's installed isntead of stable/train | 08:27 |
| noonedeadpunk | as from some point of view - it's handy that we don't need to bump shas every time | 08:28 |
| openstackgerrit | Merged openstack/openstack-ansible-os_swift stable/ussuri: remove ansible_python_interpreter https://review.opendev.org/736207 | 08:28 |
| noonedeadpunk | or we want to do that only for branchless projects | 08:28 |
| noonedeadpunk | but from other point, even on master, we're on bleeding edge, without any bumps and being able to controll services versions... | 08:31 |
| jrosser | spec sounds good | 08:33 |
| openstackgerrit | Merged openstack/openstack-ansible-os_aodh stable/ussuri: remove ansible_python_interpreter https://review.opendev.org/736206 | 08:34 |
| noonedeadpunk | ok, cool | 08:35 |
| noonedeadpunk | so patching train so it just works now and writing spec? | 08:36 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ceilometer stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736343 | 08:36 |
| jrosser | noonedeadpunk: i guess that what i was interested in | 08:38 |
| jrosser | if we should just apply something to train right now, or revert / fix config_template, $other-thing | 08:39 |
| sri_ | noonedeadpunk: quick question, does anyone tried o-a in Centos8 ? | 08:39 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736358 | 08:40 |
| noonedeadpunk | it's not ready yet, work in progress.. but spatel plays with it at least | 08:42 |
| sri_ | noonedeadpunk: ack, thanks for the info | 08:44 |
| *** maharg101 has joined #openstack-ansible | 08:49 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/openstack-ansible master: [WIP] improve ironic tempest testing https://review.opendev.org/736507 | 08:54 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/openstack-ansible-os_ironic master: [DNM] test patch https://review.opendev.org/727067 | 08:54 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Disable ceph-mgr-dashboard https://review.opendev.org/736328 | 09:02 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Add Ubuntu Focal support https://review.opendev.org/723597 | 09:02 |
| *** jawad_axd has joined #openstack-ansible | 09:09 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/ansible-config_template master: Make code compatible with older ansible versions https://review.opendev.org/736512 | 09:14 |
| noonedeadpunk | jrosser: ^ I hope that should do a trick | 09:15 |
| jrosser | i guess we can depends-on that for a stable/train patch | 09:16 |
| openstackgerrit | Merged openstack/openstack-ansible-os_trove stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736365 | 09:16 |
| jrosser | i'll take a look at that | 09:20 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-tests stable/train: Add pre-gate cleanup tasks https://review.opendev.org/736464 | 09:23 |
| *** rgogunskiy has joined #openstack-ansible | 09:25 | |
| *** rgogunskiy has quit IRC | 09:27 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible stable/train: Remove pre-gate cleanup tasks for the new plain CI images https://review.opendev.org/735785 | 09:35 |
| *** rpittau is now known as rpittau|bbl | 10:06 | |
| *** cshen has quit IRC | 10:10 | |
| chandankumar | jrosser, Hello | 10:11 |
| chandankumar | jrosser, I have started working on improving tempest test coverage project by project https://review.opendev.org/#/c/736507/ | 10:12 |
| chandankumar | and testing here https://review.opendev.org/#/c/727067/ | 10:12 |
| *** dpaclt has quit IRC | 10:15 | |
| *** isakgicu has joined #openstack-ansible | 10:23 | |
| isakgicu | Hi | 10:23 |
| isakgicu | how can I add a storage host to my current infra ? | 10:24 |
| isakgicu | using cinder | 10:24 |
| isakgicu | openstack stein release | 10:24 |
| isakgicu | sorry, not stein, I am using Train on this infra | 10:24 |
| arkan | isakgicu: I have installed ceph with ansible (I'm not a dev or geek in Openstack) | 10:43 |
| arkan | isakgicu: I associated with cinder | 10:43 |
| arkan | isakgicu: maybe I can help you setup your ceph with cinder if you need to | 10:44 |
| jrosser | chandankumar: sorry just in a meeting, back in a while | 10:49 |
| openstackgerrit | Merged openstack/openstack-ansible-os_swift stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736363 | 10:54 |
| isakgicu | I am not using ceph at the moment, but I am about to use this in close future, it would be gr8 to get some tips from some1 who have done this already | 10:57 |
| arkan | isakgiuc: if you need info about how to install ceph and connect it to OS, we can discuss privately in order not to clutter the chat here | 10:59 |
| *** shyamb has quit IRC | 11:00 | |
| isakgicu | agree | 11:05 |
| nsmeds | Hey everyone. Question for y'all: when we first ran openstack-ansible we (accidentally) lacked the br-tenant interfaces on a number of servers. Even after re-adding them, they do not exist in `openstack_hostnames_ips.yml` or `openstack_inventory.json` for those servers. There's a warning to "Never edit or delete the files" - but I feel like we'll have to if we want to resolve this. | 11:08 |
| nsmeds | Is that true? | 11:09 |
| nsmeds | e.g. right now every time we run setup-openstack, the `/etc/neutron/plugins/ml2/linuxbridge_agent.ini` has a parameter `local_ip` which resets to the wrong interface's IP | 11:10 |
| *** udesale_ has quit IRC | 11:12 | |
| jrosser | nsmeds: that would normally be br-vxlan | 11:15 |
| nsmeds | Ja, ours has different name (from legacy config... just never updated it before respinning recently) | 11:16 |
| nsmeds | Works the same | 11:16 |
| jrosser | how do you override the name? | 11:16 |
| openstackgerrit | Merged openstack/openstack-ansible-os_panko stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736360 | 11:17 |
| *** arkan has quit IRC | 11:17 | |
| *** arkan has joined #openstack-ansible | 11:17 | |
| nsmeds | https://pastebin.com/3ma8bAkt | 11:18 |
| nsmeds | that's the only reference to it tbh | 11:18 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_server stable/ussuri: remove ansible_python_interpreter https://review.opendev.org/736209 | 11:20 |
| nsmeds | I've inherited an environment that previously deployed openstack-ansible queens (project was on hold for 1+ year, and original guy left). We've used _mostly_ the same config to redeploy with Train | 11:20 |
| jrosser | ok, so i would not expect those addresses to be in the inventory | 11:21 |
| nsmeds | the br-tenant addresses? | 11:22 |
| jrosser | there is a common playbook thats used as part of the neutron install https://github.com/openstack/openstack-ansible/blob/master/playbooks/common-playbooks/neutron.yml | 11:22 |
| jrosser | key part being here https://github.com/openstack/openstack-ansible/blob/master/playbooks/common-playbooks/neutron.yml#L35-L40 | 11:22 |
| jrosser | which then uses this https://github.com/openstack/openstack-ansible/blob/master/playbooks/common-tasks/dynamic-address-fact.yml | 11:23 |
| *** markvoelker has joined #openstack-ansible | 11:24 | |
| jrosser | and it's looking the bridge up with this find_bridge: "container_networks.{{ network_address }}.bridge" | 11:24 |
| nsmeds | Sorry @jrosser - to clarify, you're worried that the br-tenant would never make it into `openstack_inventory.json`, or that we cannot add if we missed on initial deployment? | 11:25 |
| jrosser | i'm showing you how it works :) | 11:26 |
| nsmeds | Ok, yep | 11:26 |
| jrosser | it's trying to look up the bridge in container_networks, and i think if that goes wrong its going to fall back to ansible_host | 11:26 |
| jrosser | which is what you see i think | 11:26 |
| *** markvoelker has quit IRC | 11:29 | |
| nsmeds | Okay. I realize I misspoke slightly when I first mentioned the issue. The `openstack_inventory.json` looks correct - e.g. https://pastebin.com/Zcu7Ppfx | 11:34 |
| nsmeds | It's just missing from the `ansible_facts` for those hosts (which looks related to what you shared above) | 11:34 |
| jrosser | ah ok, you can delete /etc/openstack_deploy/ansible_facts to get those refreshed | 11:36 |
| jrosser | on the deploy host | 11:36 |
| nsmeds | Ok, makes sense. Thanks for clarifying - I realize the warning doesn't refer to the facts now | 11:37 |
| nsmeds | Just got a bit nervous about deleting anything in there :) | 11:37 |
| jrosser | you can just rename the dir :) | 11:37 |
| jrosser | there is a bit of magical stuff there which determines the local_ip | 11:38 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/openstack-ansible master: [WIP] improve ironic tempest testing https://review.opendev.org/736507 | 11:39 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/openstack-ansible-os_ironic master: [DNM] test patch https://review.opendev.org/727067 | 11:39 |
| *** mgariepy has quit IRC | 11:43 | |
| *** cshen has joined #openstack-ansible | 11:55 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Cleanup ansible_python_interpreter https://review.opendev.org/736263 | 12:08 |
| openstackgerrit | Merged openstack/openstack-ansible-os_gnocchi master: Cleanup ansible_python_interpreter https://review.opendev.org/736265 | 12:09 |
| *** arkan has quit IRC | 12:17 | |
| *** mgariepy has joined #openstack-ansible | 12:19 | |
| *** rpittau|bbl is now known as rpittau | 12:20 | |
| *** rh-jelabarre has joined #openstack-ansible | 12:28 | |
| *** arkan_ has joined #openstack-ansible | 12:34 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/ussuri: auto-detect interpreter. https://review.opendev.org/736199 | 12:39 |
| *** spatel has joined #openstack-ansible | 12:40 | |
| *** isakgicu has quit IRC | 12:43 | |
| *** arkan_ has quit IRC | 12:46 | |
| *** jawad_axd has quit IRC | 12:47 | |
| *** vakuznet has quit IRC | 12:52 | |
| jrosser | noonedeadpunk: mgariepy i made some patches for ceph-ansible https://github.com/jrosser/ceph-ansible/commits/jrosser-fixes | 12:55 |
| jrosser | noonedeadpunk: one of those is for nfs which i guess will affect manila | 12:55 |
| jrosser | mgariepy: do you still have the focal/ceph test VM around from yesterday? | 12:56 |
| mgariepy | yes i beleive | 12:56 |
| mgariepy | yep i have it. | 12:57 |
| jrosser | i was hoping that https://github.com/jrosser/ceph-ansible/commit/e92591045a2b01d77a045b67d2b2c78d1685ec40 would address the mgr installation | 12:57 |
| *** noonedeadpunk is now known as noonedeadpunk_ | 12:57 | |
| noonedeadpunk_ | jrosser: did you placed PR for these? | 12:57 |
| jrosser | not yet - i thought if we had the VM from yesterday it would be good to test drive before making a PR | 12:58 |
| jrosser | but i had planned to do it ASAP | 12:58 |
| mgariepy | https://github.com/openstack/openstack-ansible/blob/master/ansible-role-requirements.yml#L273 ? overwrite this in a patch for zuul ? | 12:59 |
| jrosser | can you cherry pick my github patch and re-run the ceph install playbook with mgr enabled? | 12:59 |
| jrosser | noonedeadpunk_: seems we have nfs failure on focal even without manila :( https://zuul.opendev.org/t/openstack/build/2d94c87320c945a599cb03a4880c3918/log/job-output.txt#16322 | 13:00 |
| noonedeadpunk_ | ah.... | 13:01 |
| noonedeadpunk_ | these filters... | 13:02 |
| noonedeadpunk_ | good that you have patch for this | 13:02 |
| noonedeadpunk_ | but um... | 13:03 |
| noonedeadpunk_ | why does it pass for bionic? | 13:03 |
| noonedeadpunk_ | as ansible version shoud be the same? | 13:03 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: DNM test fixes to ceph-ansible https://review.opendev.org/736707 | 13:04 |
| jrosser | noonedeadpunk_: i have no idea how it works for bionic | 13:04 |
| jrosser | that is very surprising, unless it's a different version of jinja | 13:04 |
| jrosser | becasue its actually throwing a templating error rather than and error from inside ansible | 13:04 |
| mgariepy | deleting the containers to see if your patch fixes it. | 13:05 |
| noonedeadpunk_ | we be have jinja inside venv as well | 13:05 |
| noonedeadpunk_ | at least supposed to | 13:05 |
| mgariepy | but if it works the fastest path to having it is probably a fork until the pr are merged. | 13:06 |
| jrosser | noonedeadpunk_: maybe the template is executed on the target? | 13:06 |
| jrosser | i'm really not sure where its actually running that | 13:06 |
| jrosser | very very odd how it works for bionic | 13:07 |
| noonedeadpunk_ | template is an action plugin, so it's supposed to run on controller... | 13:08 |
| noonedeadpunk_ | (just talked today about it in #ansible-devel) | 13:08 |
| mgariepy | jrosser, maybe we need to remove https://review.opendev.org/#/c/736328/3 from the stack to tests your patch :/ | 13:09 |
| jrosser | i was thinking this was higher priority that the defaults https://review.opendev.org/#/c/736707/1/inventory/group_vars/ceph_all.yml | 13:10 |
| mgariepy | maybe. | 13:10 |
| mgariepy | probably | 13:10 |
| mgariepy | let's see anyway. | 13:10 |
| jrosser | either way it looks like we need to restack the patches at some point | 13:11 |
| mgariepy | yep indeed. | 13:12 |
| mgariepy | running ceph-install now. | 13:19 |
| mgariepy | let's see how it goes. | 13:20 |
| jrosser | super, thankyou | 13:20 |
| *** sshnaidm is now known as sshnaidm|mtg | 13:23 | |
| *** jcath has joined #openstack-ansible | 13:33 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Remove python3 specific test job https://review.opendev.org/735875 | 13:35 |
| mgariepy | https://github.com/jrosser/ceph-ansible/commit/e92591045a2b01d77a045b67d2b2c78d1685ec40#diff-7d5a6ea1f75bd585af8e26ba6e9797baR3 | 13:37 |
| mgariepy | jrosser, ceph-mgr not ceph_mgr ;) | 13:37 |
| jrosser | ops | 13:37 |
| mgariepy | probably the same for all the other pkg names :D | 13:38 |
| openstackgerrit | Merged openstack/openstack-ansible stable/ussuri: Use ansible openstack collection https://review.opendev.org/734419 | 13:40 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_adjutant master: Updated from OpenStack Ansible Tests https://review.opendev.org/736723 | 13:44 |
| jrosser | mgariepy: thanks for testing it, i've fixed that and will make a PR | 13:52 |
| noonedeadpunk_ | why in the world this tasks creates directory with 700 mode https://opendev.org/openstack/ansible-role-python_venv_build/src/branch/master/tasks/python_venv_install.yml#L42-L45 http://paste.openstack.org/show/794944/ | 13:53 |
| mgariepy | noonedeadpunk_, if you create a dir with mkdir on that system does it do the same ? | 14:07 |
| noonedeadpunk_ | moreover, when I do http://paste.openstack.org/show/794945/ is goes well | 14:10 |
| noonedeadpunk_ | but, all directories created with this task, have 700 | 14:11 |
| noonedeadpunk_ | or this role... | 14:11 |
| noonedeadpunk_ | it's hold which gave me infra... | 14:11 |
| noonedeadpunk_ | can put your keys there | 14:11 |
| mgariepy | sure. | 14:11 |
| noonedeadpunk_ | so there was mount to /openstack/ but I even unmounted and things are still the same... | 14:12 |
| noonedeadpunk_ | root@104.239.168.126 | 14:14 |
| noonedeadpunk_ | easily reproducable with openstack-ansible /home/zuul/src/opendev.org/openstack/openstack-ansible/playbooks/os-keystone-install.yml | 14:14 |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736353 | 14:18 |
| mgariepy | hmm. | 14:18 |
| mgariepy | it did fix the perm. | 14:18 |
| mgariepy | http://paste.openstack.org/show/794950/ | 14:20 |
| noonedeadpunk_ | I put there explicitly | 14:20 |
| noonedeadpunk_ | but, virtualenv command, launched with command module, also does create with 700 perms | 14:21 |
| noonedeadpunk_ | So it feels smth wrong with env at this role scope... | 14:22 |
| noonedeadpunk_ | have no idea tbh what's going on there | 14:22 |
| noonedeadpunk_ | mgariepy: I reverted to the upstream version | 14:23 |
| mgariepy | lol | 14:23 |
| mgariepy | ok | 14:23 |
| noonedeadpunk_ | yeah, sorry was just testing if just changing that speciic thing will unbreak | 14:24 |
| noonedeadpunk_ | but no | 14:24 |
| *** pcaruana has quit IRC | 14:25 | |
| * jrosser wonders about umask | 14:25 | |
| noonedeadpunk_ | put your key as well | 14:29 |
| *** noonedeadpunk_ is now known as noonedeadpunk | 14:29 | |
| noonedeadpunk | maybe I don't see smth pretty obvious | 14:30 |
| noonedeadpunk | but getfacl /openstack has 755 | 14:30 |
| noonedeadpunk | so I really don't understand where does this come from... | 14:31 |
| noonedeadpunk | umask is 0022 | 14:32 |
| noonedeadpunk | dunno maybe after some connection plugin it differs... | 14:33 |
| *** pcaruana has joined #openstack-ansible | 14:38 | |
| *** jcath has quit IRC | 14:41 | |
| mgariepy | where does it take the mode from | 14:45 |
| mgariepy | lol | 14:45 |
| mgariepy | and i put the extracted zip in /root | 14:49 |
| mgariepy | - /root/.ansible/tmp/ansible-tmp-1592490534.2674892-225138-25305769262155 is the script for the file task. | 14:49 |
| mgariepy | - /root/ansible_file_payload_wka07ehy | 14:49 |
| spatel | folks, how do i find where it failed? Zuul logs are massive to splunk - https://review.opendev.org/#/c/736095/ | 14:51 |
| jrosser | spatel: i usually search through for "RUN END" and stop at the one with failed: 1 just above it | 14:52 |
| jrosser | then scroll back a little | 14:52 |
| jrosser | and there is also the ARA report in there too | 14:52 |
| openstackgerrit | Georgina Shippey proposed openstack/openstack-ansible-os_horizon master: Display name in IDP dropdown https://review.opendev.org/736739 | 14:52 |
| spatel | I am looking here .txt format - https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_042/736095/3/check/openstack-ansible-deploy-aio_lxc-centos-7/042b583/ | 14:53 |
| spatel | let me know if any other format you guys using which help to find quickly. | 14:54 |
| chandankumar | jrosser, please have a look at https://review.opendev.org/736507 and https://review.opendev.org/#/c/727067/ | 14:54 |
| chandankumar | ironic tests are not triggering please have a look | 14:54 |
| jrosser | spatel: so ctrl-f or whatever | 14:54 |
| jrosser | enter "RUN END" | 14:54 |
| jrosser | and hit enter a few times till you find the one just after the failed task | 14:54 |
| spatel | i did and found something | 14:54 |
| jrosser | chandankumar: looking | 14:55 |
| spatel | found this one - http://paste.openstack.org/show/794952/ | 14:55 |
| jrosser | spatel: thats just log noise from the log collection script | 14:56 |
| spatel | so this job isn't truly testing my patch right? | 14:56 |
| jrosser | you need to go before that and find the ansible task with failed: on it | 14:56 |
| jrosser | spatel: try the ARA report | 14:58 |
| jrosser | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_042/736095/3/check/openstack-ansible-deploy-aio_lxc-centos-7/042b583/logs/ara-report/result/80a43f9e-9c40-4da2-a2a0-9866e08571dc/ | 14:58 |
| spatel | I am also looking "failed: 1" keyword | 14:58 |
| spatel | oh!! how did you find that.. | 14:58 |
| jrosser | instead of the job output txt file go in the logs directory | 14:58 |
| jrosser | then ara-report dir, and hit the index file | 14:59 |
| jrosser | then you see it failed on the top row of the report | 14:59 |
| jrosser | click "1970 tasks" | 14:59 |
| jrosser | then press the "elapsed" header at the top of the column to reverse the time order | 14:59 |
| jrosser | and bingo there you are | 15:00 |
| jrosser | if you press the "failed" red square at the right of the row with the task it it you get a popup with the details | 15:00 |
| spatel | Great!!!! this is what i was looking for... | 15:01 |
| openstackgerrit | Georgina Shippey proposed openstack/openstack-ansible-os_keystone master: Add IDP display_name to defaults https://review.opendev.org/736741 | 15:01 |
| jrosser | and the link i gave you is accessible as a permalink from the top-right of the popup | 15:01 |
| spatel | Got it | 15:02 |
| spatel | let me see why it failed.. and try to fix it | 15:02 |
| spatel | rules is all job should pass right? | 15:02 |
| jrosser | when all the jobs pass then you get code review | 15:03 |
| mgariepy | need to go back home i'll be back in a few hours. | 15:03 |
| spatel | jrosser: Thanks for the help.. let me try to fix and resubmit patch | 15:04 |
| jrosser | after you get feedback and any updates also passing it needs two core reviewers to OK it then it has to all run again and pass in order to merge | 15:04 |
| spatel | jrosser: question, if same patch passing in my lab box but zuul failed it in that case? | 15:06 |
| spatel | how do i make sure my lab box is similar to zuul version? | 15:06 |
| jrosser | very good question indeed | 15:06 |
| spatel | there is no way we can ssh into Zuul worker nodes right? | 15:07 |
| jrosser | the CI images are custom built, so they are close, but not exactly like your lab | 15:07 |
| jrosser | i tend to do local development using a ubnutu or centos cloud image VM | 15:08 |
| jrosser | thats generally "close enough" | 15:08 |
| spatel | jrosser: sounds good | 15:08 |
| *** mgariepy has quit IRC | 15:08 | |
| jrosser | and if you really do get hugely stuck then you can ask for failed job to be held rather than disposed of at failure, and your ssh keys inserted | 15:09 |
| jrosser | but thats kinda last resort stuff as it requires someone to configure that manually in zuul and also to sort putting your key in by hand | 15:09 |
| spatel | jrosser: good to know, i think you guys will rescue me before that :) | 15:10 |
| jrosser | chandankumar: you're not just missing a .tests like this are you https://github.com/openstack/openstack-ansible/blob/master/tests/roles/bootstrap-host/templates/user_variables_magnum.yml.j2#L54-L55 | 15:14 |
| jrosser | and octiava is also more specific that just the plugin name https://github.com/openstack/openstack-ansible/blob/master/tests/roles/bootstrap-host/templates/user_variables_octavia.yml.j2#L13-L14 | 15:14 |
| chandankumar | jrosser, sorry, https://review.opendev.org/#/c/736507/2/tests/roles/bootstrap-host/templates/user_variables_ironic_yml.j2@13 I think i have added that | 15:20 |
| jrosser | chandankumar: yes thats the name of the module | 15:20 |
| chandankumar | sorry got it | 15:20 |
| jrosser | ah ok :) | 15:20 |
| jrosser | not sure if you want something specific out of that plugin, as it might cover an awful lot of stuff i guess | 15:21 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/openstack-ansible master: [WIP] improve ironic tempest testing https://review.opendev.org/736507 | 15:21 |
| chandankumar | jrosser, I will first see how much it passes without config | 15:21 |
| chandankumar | then move forward | 15:21 |
| jrosser | great, jamesdenton have you seen this ironic tempest stuff? | 15:21 |
| jamesdenton | i have not. | 15:23 |
| jrosser | jamesdenton: chandankumar is working on making tempest actually do something useful for os_ironic | 15:24 |
| jamesdenton | i have some patches as well that might/might not be useful: https://review.opendev.org/#/q/status:open+branch:master+topic:tempest-resource-breakout | 15:24 |
| jamesdenton | there is also this old thing that might be revisited? https://review.opendev.org/#/c/381049/ | 15:25 |
| jamesdenton | i think i have some similar patches on another machine, lemme fire it up and see if they are still there | 15:26 |
| jrosser | now thats interesting | 15:27 |
| jrosser | becasue when that ealier patch was written the tests were done in the role repo | 15:27 |
| jrosser | so there was an obvious place to put the creation of all the ancilliary stuff | 15:27 |
| jrosser | so really related to the discussion we've had about where resource creation should go when the tests are coming from the integrated repo | 15:28 |
| *** mgariepy has joined #openstack-ansible | 15:28 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/openstack-ansible master: [WIP] improve ironic tempest testing https://review.opendev.org/736507 | 15:28 |
| openstackgerrit | Merged openstack/openstack-ansible master: Update default requirements_git_url for utility https://review.opendev.org/732528 | 15:29 |
| jrosser | could i get reviews on this https://review.opendev.org/#/c/735785 | 15:30 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible stable/train: Bump SHAs for stable/train https://review.opendev.org/735469 | 15:30 |
| *** sshnaidm|mtg is now known as sshnaidm|ruck | 15:41 | |
| *** cshen has quit IRC | 15:43 | |
| mgariepy | jrosser, quick nit. update the commit message to remove the depends-on master ;) | 15:46 |
| jrosser | actually thats a good point, just maybe not as you intended :D | 15:47 |
| jrosser | config_template is branchless so only has master | 15:47 |
| jrosser | but of course that does need to merge first, and is the thing that has unwedged stable/train for openstack-ansible | 15:48 |
| jrosser | so we need reviews on here as well https://review.opendev.org/#/c/736512/ | 15:49 |
| *** gyee has joined #openstack-ansible | 15:50 | |
| *** spatel has quit IRC | 15:55 | |
| *** spatel has joined #openstack-ansible | 15:55 | |
| *** also_stingrayza has joined #openstack-ansible | 15:58 | |
| *** cshen has joined #openstack-ansible | 15:58 | |
| *** stingrayza has quit IRC | 16:01 | |
| *** cshen has quit IRC | 16:03 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible stable/train: Suppress errors when ara is not installed https://review.opendev.org/731289 | 16:09 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible stable/train: Bump ansible version to 2.8.12 https://review.opendev.org/736465 | 16:09 |
| mgariepy | jrosser, ok. lol good to know. | 16:18 |
| mgariepy | i'll probably forget in the near future anyway tho. | 16:18 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-tests stable/train: Add pre-gate cleanup tasks https://review.opendev.org/736464 | 16:24 |
| *** cshen has joined #openstack-ansible | 16:37 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_aodh stable/ussuri: Updated from OpenStack Ansible Tests https://review.opendev.org/736340 | 16:41 |
| *** cshen has quit IRC | 16:42 | |
| *** sshnaidm|ruck is now known as sshnaidm|off | 16:52 | |
| *** rpittau is now known as rpittau|afk | 17:00 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-tests stable/train: Add pre-gate cleanup tasks https://review.opendev.org/736464 | 17:06 |
| openstackgerrit | Satish Patel proposed openstack/openstack-ansible-os_horizon master: Add Centos-8 support https://review.opendev.org/736095 | 17:12 |
| jrosser | noonedeadpunk: is this correct for centos? https://opendev.org/openstack/openstack-ansible/blame/branch/master/playbooks/common-tasks/ceph-server.yml#L52-L60 | 17:14 |
| jrosser | i'm just looking at this https://zuul.opendev.org/t/openstack/build/d42294ce067145aab62c1f24474d3b49/log/job-output.txt#10110-10114 | 17:15 |
| noonedeadpunk | yep python3-pyyaml is centos8 package | 17:15 |
| jrosser | ok, so the first task fails and is rescued | 17:16 |
| jrosser | but the rescue also fails | 17:16 |
| *** arkan has joined #openstack-ansible | 17:32 | |
| noonedeadpunk | whaaaaat | 17:36 |
| noonedeadpunk | http://paste.openstack.org/show/794968/ | 17:36 |
| noonedeadpunk | jrosser: rescue fails on centos8??? | 17:37 |
| noonedeadpunk | on 7 I guess | 17:37 |
| noonedeadpunk | I think these days I tested it with manil role https://review.opendev.org/#/c/728578/ and it was passing... | 17:39 |
| noonedeadpunk | ok, so why and how umask in ansible role differs from user under which role is launched... | 17:40 |
| jrosser | I saw the Manila fail on centos 7 | 17:55 |
| jrosser | does the unmask thing depend on if it’s a login shell or not? | 17:56 |
| noonedeadpunk | I don't know tbh... | 18:09 |
| noonedeadpunk | but the thing is, that when I run just sample playbook, umask is 0022 | 18:12 |
| spatel | possible umask is different for ansible or that specific bash process. may be worth dumping bash environment variable and see if its hardcoded | 18:14 |
| noonedeadpunk | spatel: so I created http://paste.openstack.org/show/794969/ | 18:16 |
| noonedeadpunk | (but as root) | 18:16 |
| noonedeadpunk | and it gives me 0022 | 18:16 |
| noonedeadpunk | but when I place that code into playbooks/os-keystone-install.yml as pre-task it's 0077 | 18:18 |
| noonedeadpunk | but.... | 18:18 |
| noonedeadpunk | it's run against keystone_all | 18:18 |
| noonedeadpunk | yes. that's it | 18:19 |
| noonedeadpunk | when run against localhost - 0022, when run against aio1 - 0077 | 18:19 |
| spatel | may be inside container umask is different | 18:19 |
| noonedeadpunk | (while it's the same host eventually) | 18:19 |
| noonedeadpunk | it's bare metal... | 18:19 |
| spatel | may be se shell: umask 022 before that task | 18:20 |
| spatel | s/se/set | 18:20 |
| spatel | make sure nothing inside ~/.ssh/environment | 18:21 |
| noonedeadpunk | that's good call | 18:21 |
| spatel | may be over SSH mask getting alter | 18:21 |
| noonedeadpunk | yeah, issue is when we ssh | 18:22 |
| noonedeadpunk | this is when I run against localhost and aio1 http://paste.openstack.org/show/794970/ | 18:23 |
| noonedeadpunk | but when I manually ssh it's still 0022 | 18:25 |
| noonedeadpunk | and no environment files or smth present | 18:25 |
| spatel | anything here /etc/pam.d/common-session ? | 18:26 |
| noonedeadpunk | oh, hm | 18:28 |
| spatel | something is coming from session or ssh config | 18:28 |
| noonedeadpunk | I get 0077 for this ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=publickey -o PasswordAuthentication=no -o ConnectTimeout=5 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o ServerAliveInterval=64 -o ServerAliveCountMax=1024 -o Compression=no -o TCPKeepAlive=yes -o VerifyHostKeyDNS=no -o ForwardX11=no -o ForwardAgent=yes 172.29.236. | 18:29 |
| noonedeadpunk | 100 '/bin/sh -c '"'"'umask'"'"'' | 18:29 |
| noonedeadpunk | * http://paste.openstack.org/show/794971/ | 18:29 |
| noonedeadpunk | eventually even `ssh 172.29.236.100 '/bin/sh -c '"'"'umask'"'"''` | 18:30 |
| noonedeadpunk | spatel: if you give me your pub key I can place it | 18:30 |
| spatel | may be ansible security stuff 0077 is more secure mask so ansible default using it until you specify | 18:31 |
| spatel | let me give you my key | 18:31 |
| noonedeadpunk | eventually I run this command manually and get 0077 | 18:32 |
| noonedeadpunk | but ssh and running umask gives different result from ssh -c umask | 18:32 |
| spatel | must be security reason ssh using 0077 | 18:33 |
| spatel | on my PC i did ssh 10.30.1.141 umask (i got 0022) | 18:35 |
| spatel | over SSH | 18:35 |
| spatel | I am using centOS | 18:35 |
| noonedeadpunk | my centos 8 sandbox is ok | 18:36 |
| noonedeadpunk | but our CI image.... has this behaviour... | 18:36 |
| spatel | check file /etc/login.defs | 18:36 |
| spatel | is this ubuntu? | 18:36 |
| noonedeadpunk | centos 8 | 18:37 |
| spatel | hope SELinux is off | 18:37 |
| noonedeadpunk | it's permissive | 18:37 |
| noonedeadpunk | selinux packages on my sanbox and CI one are the same... | 18:37 |
| *** cshen has joined #openstack-ansible | 18:38 | |
| noonedeadpunk | in /etc/login.defs I have `UMASK 077` | 18:38 |
| noonedeadpunk | but... it's the same for my sandbox which is ok... | 18:38 |
| spatel | hmm | 18:39 |
| noonedeadpunk | will go and try my luck with infra folks | 18:42 |
| spatel | its very odd.. let me poke around and see | 18:44 |
| *** cshen has quit IRC | 19:18 | |
| spatel | it feels like security feature, i am looking for doc in redhat KB to see if i can find something. | 19:21 |
| spatel | I can see Ansible hardending in /etc/ssh/sshd_config file so may be that feature came from there somewhere | 19:22 |
| mgariepy | umask diffrent for ansible ? | 19:28 |
| spatel | noonedeadpunk: read this - https://bugzilla.redhat.com/show_bug.cgi?id=1777718 | 19:29 |
| openstack | bugzilla.redhat.com bug 1777718 in shadow-utils "default umask for sshd process is 0077" [Unspecified,On_qa] - Assigned to ipedrosa | 19:29 |
| spatel | that is our issue.. | 19:29 |
| spatel | The final solution should be to set the UMASK to 022 in login.defs and add support for HOME_MODE and set it to 077. I am reassigning the bug to shadow-utils as that is the component that contains login.defs. | 19:29 |
| spatel | Its like a bug (may be effected version is CentOS 8.2.x ) | 19:30 |
| jrosser | the infra images are built very often so could be 8.2? | 19:37 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: [WIP] Centos-8 support https://review.opendev.org/689629 | 19:39 |
| noonedeadpunk | they were 8.1 unttil updated by us | 19:40 |
| noonedeadpunk | but I upgraded 8.1 to 8.2 and it was ok | 19:41 |
| noonedeadpunk | (not 100% sure that in CI it realy wasn't 8.2 from the beggining | 19:41 |
| noonedeadpunk | but | 19:41 |
| noonedeadpunk | https://review.opendev.org/#/c/689629/59/tests/roles/bootstrap-host/templates/user_variables.aio.yml.j2 is going to fix that | 19:41 |
| noonedeadpunk | it's pretty nasty, but.... | 19:41 |
| noonedeadpunk | infra folks also helped to come up to this workaround | 19:42 |
| noonedeadpunk | it might open security breach though... | 19:43 |
| noonedeadpunk | but in CI - whatever | 19:43 |
| noonedeadpunk | I'm about to ask to drop this hold node | 19:43 |
| mgariepy | ok with me. | 19:44 |
| noonedeadpunk | and thank you for the help | 19:44 |
| noonedeadpunk | it was quite... weird thing | 19:44 |
| noonedeadpunk | spatel: if it pass keystone, I will include your patches as depends-on | 19:45 |
| noonedeadpunk | I'm also about to drop some changes from rabbitmq patch | 19:46 |
| noonedeadpunk | as I'm sure it's related | 19:46 |
| *** arkan has quit IRC | 19:48 | |
| spatel | noonedeadpunk: thanks | 20:03 |
| spatel | noonedeadpunk: RedHat community saying umask bug will get fix in which will be fixed in RHEL8.3 | 20:14 |
| noonedeadpunk | so it's a bug and they aware | 20:23 |
| noonedeadpunk | lol | 20:23 |
| noonedeadpunk | better late then never | 20:24 |
| noonedeadpunk | spatel: any link to the bug itself? | 20:25 |
| spatel | https://bugzilla.redhat.com/show_bug.cgi?id=1833664 | 20:25 |
| openstack | bugzilla.redhat.com bug 1833664 in shadow-utils "Regression in default file permission in RHEL 8.x as compared to RHEL 7.x when file is generated via a a program ran via su" [Urgent,Closed: duplicate] - Assigned to ipedrosa | 20:25 |
| spatel | Iker Pedrosa put that comments last month so look like they are aware of it and must be releasing fix in next version | 20:27 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-rabbitmq_server master: Add Centos-8 support https://review.opendev.org/735552 | 20:34 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: [WIP] Centos-8 support https://review.opendev.org/689629 | 20:37 |
| *** miloa has quit IRC | 21:01 | |
| *** cshen has joined #openstack-ansible | 21:14 | |
| *** mloza has quit IRC | 21:15 | |
| *** cshen has quit IRC | 21:18 | |
| openstackgerrit | Satish Patel proposed openstack/openstack-ansible-lxc_container_create master: Add Centos-8 support https://review.opendev.org/736850 | 21:49 |
| *** spatel has quit IRC | 21:52 | |
| *** markvoelker has joined #openstack-ansible | 21:58 | |
| *** markvoelker has quit IRC | 22:02 | |
| *** tosky has quit IRC | 22:25 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_tempest master: Fix tempestconf URL https://review.opendev.org/735429 | 23:05 |
| *** cshen has joined #openstack-ansible | 23:14 | |
| *** cshen has quit IRC | 23:19 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!