| *** gyee has quit IRC | 00:32 | |
| *** pto has joined #openstack-ansible | 01:03 | |
| *** pto has quit IRC | 01:07 | |
| *** d34dh0r53 has quit IRC | 01:24 | |
| *** macz_ has joined #openstack-ansible | 01:35 | |
| *** macz_ has quit IRC | 01:40 | |
| *** macz_ has joined #openstack-ansible | 01:56 | |
| *** macz_ has quit IRC | 02:00 | |
| *** dasp_ has quit IRC | 02:31 | |
| *** evrardjp has quit IRC | 02:33 | |
| *** evrardjp has joined #openstack-ansible | 02:33 | |
| *** dasp has joined #openstack-ansible | 02:35 | |
| *** miloa has joined #openstack-ansible | 03:34 | |
| *** miloa has quit IRC | 03:37 | |
| *** macz_ has joined #openstack-ansible | 03:55 | |
| *** macz_ has quit IRC | 04:00 | |
| *** openstackgerrit has quit IRC | 05:01 | |
| *** pto has joined #openstack-ansible | 05:02 | |
| *** openstackgerrit has joined #openstack-ansible | 05:43 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Change task ordering to perform smooth upgrades https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/788501 | 05:43 |
|---|---|---|
| *** kleini has joined #openstack-ansible | 05:43 | |
| *** snapdeal has joined #openstack-ansible | 05:43 | |
| noonedeadpunk | mornings | 05:53 |
| *** macz_ has joined #openstack-ansible | 05:56 | |
| *** raukadah is now known as chandankumar | 05:56 | |
| *** openstackgerrit has quit IRC | 05:59 | |
| *** macz_ has quit IRC | 06:00 | |
| jrosser | morning | 06:14 |
| *** shyamb has joined #openstack-ansible | 06:28 | |
| *** luksky has joined #openstack-ansible | 06:39 | |
| *** macz_ has joined #openstack-ansible | 06:46 | |
| *** macz_ has quit IRC | 06:52 | |
| *** openstackgerrit has joined #openstack-ansible | 06:54 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Change task ordering to perform smooth upgrades https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/788501 | 06:54 |
| recyclehero | morning | 07:01 |
| recyclehero | jrosser: I think something like nfs cache is causing my instances to boot into grub rescue. is there a way I can change the introduced nfs to openstack? | 07:03 |
| recyclehero | since I have deployed with ansible I should replay the the books and not manually change stuff, right/ | 07:05 |
| *** shyamb has quit IRC | 07:16 | |
| jrosser | recyclehero: i'm sorry it's really hard to know what is going on, the fact that they boot grub at all means something is working from the nfs? | 07:16 |
| recyclehero | yes even someof the instances boot to the os and some dont. I think the process of copying the volumes from the nfs to external hdd and back to the newly created raid array mounted on the previous nfs share point changed something that is causing inconsistencies. | 07:18 |
| jrosser | i'm not sure i see how re-running any of the openstack-ansible is helpful, as that really just takes the relevant parts for NFS config and puts that into the service .conf files | 07:18 |
| recyclehero | so I should change /etc/cinder/nfs_shares in cincer volume container and restart compute node and controller and see what happens | 07:19 |
| *** andrewbonney has joined #openstack-ansible | 07:19 | |
| recyclehero | anything beside this change come to your mind? | 07:19 |
| jrosser | i don't understand why adjusting the nfs_shares stuff helps | 07:20 |
| jrosser | i think if it were me i'd be doing more basic debugging on the volumes, given that some work and some dont | 07:21 |
| jrosser | check permissions on what in the nfs server, try to manually mount the volumes yourself, see if you can fsck them and the filesystems look ok | 07:21 |
| jrosser | that sort of thing | 07:21 |
| recyclehero | maybe there is some caching going on with the previous nfs share. | 07:21 |
| recyclehero | you mean qemu-nbd mount them? | 07:22 |
| jrosser | i don't know, i do ceph here rather than NFS so just thinking out loud | 07:22 |
| recyclehero | thats good and I appreciate that | 07:23 |
| jrosser | maybe what i'm trying to say is work down through the levels of stuff to find why some are good and some bad | 07:23 |
| jrosser | in a sense its great that some are working, as it gives you a reference point | 07:23 |
| noonedeadpunk | well, we're still having nfs here, but it's run on solaris with zfs so it's pretty unique I guess... | 07:24 |
| jrosser | oh well thats cool :) outside our openstack weve got $lots PB of solaris/zfs | 07:24 |
| noonedeadpunk | eventually if you go to grub, that means your vms see volume | 07:24 |
| noonedeadpunk | I don't think you would love having it inside of the openstack though :) | 07:25 |
| *** macz_ has joined #openstack-ansible | 07:25 | |
| *** rpittau|afk is now known as rpittau | 07:27 | |
| *** macz_ has quit IRC | 07:30 | |
| *** gokhani has joined #openstack-ansible | 07:32 | |
| gokhani | noonedeadpunk, https://zuul.opendev.org/t/openstack/build/6fd8e18a90784ea996ee039a786413fe/log/job-output.txt#15686 this error is not related to my patch. Do I need to recheck or rebase my patch? | 07:35 |
| noonedeadpunk | recheck | 07:38 |
| recyclehero | I replaced the volume with an earlier backup. that didnt help though | 07:38 |
| *** tosky has joined #openstack-ansible | 07:44 | |
| recyclehero | I dint look at this before but could someone please check if their volumes owners and groups in the nfs share changes to 64055:64055 which is libvirt-qemu after they are in-use? | 07:44 |
| *** dirk has quit IRC | 07:45 | |
| *** pto has quit IRC | 07:50 | |
| noonedeadpunk | I think mine are owned by libvirt-qemu | 07:51 |
| *** dirk has joined #openstack-ansible | 07:51 | |
| *** pto has joined #openstack-ansible | 07:51 | |
| recyclehero | noonedeadpunk: thanks and its okay that 64055 is shown on the nfs server as its not known there? | 07:54 |
| noonedeadpunk | yeah, totally. as nfs server is just a storage, it does not maniplate with files, except store them | 08:07 |
| noonedeadpunk | another thing, that 64055 uid and guid should mean the same user on your computes | 08:07 |
| pto | It seems like ceph throttles rbd export down, is it possible to increase this priotity with an runtime option like osd_recovery_op_priority? | 08:24 |
| *** shyamb has joined #openstack-ansible | 08:32 | |
| *** macz_ has joined #openstack-ansible | 08:37 | |
| *** macz_ has quit IRC | 08:41 | |
| recyclehero | you know whats the funny part about my situation? now all the instances boot except two which were my most data intensive ones | 08:42 |
| recyclehero | thanks noonedeadpunk | 08:42 |
| recyclehero | now I will try to convert this tow volumes to images then try launching from them | 08:42 |
| recyclehero | and bigger all were 10|15GiB these two 20Gib | 08:45 |
| admin0 | is "mimic" the final ceph version that is possible on ubuntu xenial/rocky ? | 08:52 |
| *** shyamb has quit IRC | 09:07 | |
| *** shyamb has joined #openstack-ansible | 09:07 | |
| recyclehero | I ro mounted the volume with qemu-nbd. I am able to see data :) | 09:18 |
| recyclehero | why doesnt it boot though | 09:18 |
| recyclehero | oh I see it . in /boot/grub when I ls "ls: reading directory '.': Bad message" | 09:23 |
| *** shyamb has quit IRC | 09:28 | |
| *** shyamb has joined #openstack-ansible | 09:30 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-lxc_container_create master: Do not create extras in /openstack/{{ inventory_hostname }} https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/788222 | 09:43 |
| noonedeadpunk | yeah, eventually if you see grub, then it means that drive is inside VM and libvirt managed to pass it. But most likely you just can not write or read from it for some reason | 09:43 |
| *** tobberydberg has quit IRC | 09:47 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_trove master: Do not override service project description https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/787675 | 09:50 |
| *** shyamb has quit IRC | 09:50 | |
| *** shyamb has joined #openstack-ansible | 09:51 | |
| *** shyamb has quit IRC | 09:52 | |
| *** tobberydberg has joined #openstack-ansible | 09:52 | |
| *** shyamb has joined #openstack-ansible | 09:53 | |
| openstackgerrit | Merged openstack/openstack-ansible-rabbitmq_server master: Use integrated tests for rabbitmq https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/788043 | 09:57 |
| Carcer | pto: --rbd-concurrent-management-ops arg to rbd client might be able to speed things up (it defaults to 10) | 10:05 |
| pto | Carcer: I have tried that, but its still very slow compared to raw IO operations | 10:06 |
| *** shyamb has quit IRC | 10:08 | |
| *** macz_ has joined #openstack-ansible | 10:10 | |
| *** macz_ has quit IRC | 10:15 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Remove cephfs_enable_snapshots https://review.opendev.org/c/openstack/openstack-ansible/+/787341 | 10:30 |
| Carcer | hmm. I admit I haven't really paid much attention to how fast my rbd exports go after getting them to be acceptably fast for overnight backups | 10:38 |
| Carcer | and that was mainly by doing export-diffs | 10:38 |
| *** shyamb has joined #openstack-ansible | 10:45 | |
| *** shyamb has quit IRC | 10:49 | |
| *** shyamb has joined #openstack-ansible | 10:50 | |
| *** ChipOManiac has joined #openstack-ansible | 10:50 | |
| *** akahat is now known as akahat|ruck | 10:50 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/ussuri: Bump SHAs for stable/ussuri https://review.opendev.org/c/openstack/openstack-ansible/+/787897 | 10:53 |
| *** shyamb has quit IRC | 10:55 | |
| *** shyamb has joined #openstack-ansible | 10:55 | |
| *** macz_ has joined #openstack-ansible | 11:02 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/victoria: Bump SHAs for stable/victoria https://review.opendev.org/c/openstack/openstack-ansible/+/787898 | 11:05 |
| *** shyamb has quit IRC | 11:06 | |
| ChipOManiac | Hi all, | 11:06 |
| *** macz_ has quit IRC | 11:06 | |
| ChipOManiac | Has anyone here done a deployment of Masakari with openstack-ansible. | 11:07 |
| *** shyamb has joined #openstack-ansible | 11:07 | |
| ChipOManiac | Have some issues that need to be fixed. | 11:07 |
| jonher | it's on my list to test in a physical env, but haven't done it yet | 11:07 |
| jonher | but i'm pretty sure there are people here running it | 11:08 |
| ChipOManiac | I've gotten the services to work, and set up everything according to the documents on how to create a segment. | 11:08 |
| ChipOManiac | Where the issue is looks like something to do with the hypervisor hostname. | 11:08 |
| openstackgerrit | Jonathan Herlin proposed openstack/openstack-ansible-os_horizon stable/victoria: Install adjutant-ui when the adjutant service is deployed https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/788611 | 11:09 |
| ChipOManiac | And I'm not sure why that happens because I used the vanilla configuration when deploying it. | 11:09 |
| ChipOManiac | @jonher I had a couple of snags but managed to fix them. | 11:10 |
| *** shyamb has quit IRC | 11:12 | |
| *** shyamb has joined #openstack-ansible | 11:12 | |
| openstackgerrit | Jonathan Herlin proposed openstack/openstack-ansible stable/victoria: Add adjutant-ui repo package and zuul repo https://review.opendev.org/c/openstack/openstack-ansible/+/788612 | 11:13 |
| openstackgerrit | Jonathan Herlin proposed openstack/openstack-ansible-os_horizon stable/victoria: Install adjutant-ui when the adjutant service is deployed https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/788611 | 11:14 |
| jonher | maybe you could paste some output on http://paste.openstack.org/ what you are seeing/suspecting of being the issue | 11:16 |
| openstackgerrit | Merged openstack/openstack-ansible stable/train: Bump SHAs for stable/train https://review.opendev.org/c/openstack/openstack-ansible/+/787896 | 11:17 |
| openstackgerrit | Merged openstack/openstack-ansible stable/victoria: Add custom user-agent for git clones from opendev.org https://review.opendev.org/c/openstack/openstack-ansible/+/787607 | 11:17 |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Fix watchdog version https://review.opendev.org/c/openstack/openstack-ansible/+/787747 | 11:17 |
| ChipOManiac | jonher Ill take a look. | 11:18 |
| openstackgerrit | yang yawei proposed openstack/ansible-hardening master: setup.cfg: Replace dashes with underscores https://review.opendev.org/c/openstack/ansible-hardening/+/788715 | 11:18 |
| ChipOManiac | @jonher | 11:19 |
| ChipOManiac | Will you be doing a deployment anytime soon? | 11:20 |
| ChipOManiac | In which case I have some tips for you if you're using Ubuntu Bionic. | 11:20 |
| jonher | might be a little while, working on getting cloudkitty and adjutant on victoria first on top of other work currently | 11:21 |
| jonher | if you have notes that could be of help i'm certainly interested | 11:21 |
| *** shyamb has quit IRC | 11:21 | |
| ChipOManiac | @jonher I'll paste them in and send you the link. | 11:22 |
| jonher | thanks | 11:22 |
| noonedeadpunk | Eventually on Victoria I think it has all sense to deploy on Focal rather then Bionic | 11:25 |
| ChipOManiac | noonedeadpunk What about the requirements for ifupdown? | 11:26 |
| ChipOManiac | Are they still there? | 11:28 |
| jrosser | there isnt a specific requirement for ifupdown | 11:28 |
| jrosser | you can configure the host networking however you like | 11:28 |
| ChipOManiac | jrosser Wasn't aware of that. Thanks. | 11:29 |
| jrosser | but ifupdown is present in focal/groovy/hirsute regardless so it's easy to copy config across from bionic | 11:29 |
| jrosser | alternatively there is suggestions for netplan here https://github.com/openstack/openstack-ansible/tree/master/etc/netplan | 11:30 |
| ChipOManiac | Would anyone know (and this ties to what I suspect is the problem with my Masakari deployment) whether the hypervisor hostname that comes when you do "hypervisor list" in the openstack cli has to be the FQDN or just the hostname? | 11:32 |
| jrosser | my lab setup has FQDN in that list | 11:34 |
| ChipOManiac | jrosser Same here. | 11:34 |
| jrosser | which admittedly isn't really "hostname" as the column heading suggests | 11:34 |
| ChipOManiac | It's been done by openstack-ansible by default. | 11:34 |
| ChipOManiac | Then again, I am still on Train. | 11:36 |
| *** gokhani has quit IRC | 11:38 | |
| *** dave-mccowan has quit IRC | 11:40 | |
| *** dave-mccowan has joined #openstack-ansible | 11:42 | |
| *** macz_ has joined #openstack-ansible | 11:42 | |
| *** ChipOManiac has quit IRC | 11:46 | |
| *** macz_ has quit IRC | 11:47 | |
| *** rh-jelabarre has joined #openstack-ansible | 11:52 | |
| *** macz_ has joined #openstack-ansible | 12:03 | |
| *** macz_ has quit IRC | 12:08 | |
| *** pcaruana has quit IRC | 12:10 | |
| *** dave-mccowan has quit IRC | 12:15 | |
| *** dave-mccowan has joined #openstack-ansible | 12:17 | |
| *** snapdeal has quit IRC | 12:20 | |
| *** pcaruana has joined #openstack-ansible | 12:23 | |
| *** gokhani has joined #openstack-ansible | 12:30 | |
| noonedeadpunk | ChipOManiac: oh, I know what the issue is | 12:37 |
| noonedeadpunk | pity that he left almost hour ago:( | 12:37 |
| *** pcaruana has quit IRC | 12:53 | |
| *** spatel_ has joined #openstack-ansible | 12:58 | |
| *** spatel_ is now known as spatel | 12:58 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Change task ordering to perform smooth upgrades https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/788501 | 13:00 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Change task ordering to perform smooth upgrades https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/788501 | 13:05 |
| *** pcaruana has joined #openstack-ansible | 13:06 | |
| *** macz_ has joined #openstack-ansible | 13:15 | |
| *** macz_ has quit IRC | 13:20 | |
| admin0 | hmm.. when doing a xenial -> bionic upgrade, ceph-mon is installing luminous version, while in the user_variables ( and rest of the cluster) = mimic version .. .. any pointers | 13:25 |
| noonedeadpunk | um, rest of the cluster - you mean osds, mgrs, rgws? | 13:35 |
| noonedeadpunk | or clients? | 13:35 |
| admin0 | meaning osds, etc | 13:39 |
| noonedeadpunk | I suggest that on mons there might be apt pinnings in /etc/apt/preferences.d | 13:40 |
| admin0 | so osa playbook will do a diff pinning for xenial vs bionic ? | 13:42 |
| admin0 | i will check and report | 13:43 |
| admin0 | just deleted the container to start again (fresh) | 13:43 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_container_create master: Do not create extras in /openstack/{{ inventory_hostname }} https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/788222 | 13:50 |
| *** macz_ has joined #openstack-ansible | 13:52 | |
| *** dave-mccowan has quit IRC | 13:52 | |
| *** dave-mccowan has joined #openstack-ansible | 13:56 | |
| *** macz_ has quit IRC | 13:57 | |
| admin0 | noonedeadpunk, https://gist.githubusercontent.com/a1git/f3e9de9254903c612cc24bd344024be9/raw/abd2aaea56a422fb8356faff8c619969cd7194e6/gistfile1.txt | 13:57 |
| admin0 | the apt says mimic .. while lum is insalled :) | 13:58 |
| noonedeadpunk | admin0: iirc lum is what's installed in natie repos | 14:05 |
| noonedeadpunk | the question is if you have enything in your preferences.d which may fix ceph version | 14:05 |
| noonedeadpunk | oh, actually there might be thing like apt repo priority | 14:08 |
| noonedeadpunk | admin0: what's `apt-cache policy ceph-mon`? | 14:08 |
| *** d34dh0r53 has joined #openstack-ansible | 14:09 | |
| mgariepy | admin0, where does the pkg comes from ? | 14:10 |
| mgariepy | apt-cache policy <pkg> | 14:11 |
| admin0 | noonedeadpunk, mgariepy this is what i get: https://gist.githubusercontent.com/a1git/536a8d16bda8a3ddd5bf9af7d95887e7/raw/8e5db1e31d6e966111630c7c72c820b55ba9aefe/gistfile1.txt | 14:12 |
| admin0 | i have not touched any variables or overrides or done any custom settings | 14:12 |
| noonedeadpunk | so yes, looks like apt pinning in place | 14:13 |
| noonedeadpunk | as native repo wont' have 1000 prio | 14:13 |
| mgariepy | oops didn't saw noonedeadpunk's apt-cache policy stuff before i asked :(. i need to focus more :) | 14:15 |
| admin0 | my question is more like .. from the same playbook version(18.1.20) does it pin a different stuff for xenial and diff for bionic ? | 14:16 |
| admin0 | while in user_variables, it set to mimic | 14:16 |
| noonedeadpunk | I think it doesn't pin anything right now but it did some time ago | 14:16 |
| noonedeadpunk | it just doesn't remove that pinning | 14:17 |
| mgariepy | what is in /etc/apt/preferences.d ? | 14:19 |
| noonedeadpunk | already asked that as well haha | 14:19 |
| mgariepy | didn't saw the answer tho. | 14:20 |
| noonedeadpunk | admin0: or, actually that's possible that we pin differently for bionic and xenial | 14:20 |
| openstackgerrit | Merged openstack/openstack-ansible master: Set DynamicUser in systemd-journal-upload unit file https://review.opendev.org/c/openstack/openstack-ansible/+/630671 | 14:21 |
| noonedeadpunk | as we might have variable set to external_repo for bionic and native for xenial | 14:21 |
| noonedeadpunk | or smth like that | 14:21 |
| mgariepy | or unattended-upgrade pkg adding stuff,. | 14:21 |
| *** spatel has quit IRC | 14:27 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Make journal_dump py3.6 compatable https://review.opendev.org/c/openstack/openstack-ansible/+/788465 | 14:28 |
| jrosser | regarding the apt pinning for ceph, i think that is just updated as we move from luminous->mimic->nautilis->octopus between major releases of OSA | 14:30 |
| jrosser | it's important to pin the download.ceph.com repos at a higher priority than either the distro or UCA repos | 14:31 |
| mgariepy | isn't the ceph-ansible repo version supporting also only one release per stable tag? | 14:32 |
| jrosser | it certainly used to be like that | 14:32 |
| jrosser | though i think it was possible to install N & O ceph versions with the same branch recently | 14:32 |
| mgariepy | https://docs.ceph.com/projects/ceph-ansible/en/stable-5.0/#releases | 14:33 |
| jrosser | oh hold on | 14:34 |
| jrosser | it was the ansible version i think that you could move forward | 14:34 |
| mgariepy | yeah they pushed 2.8 > 2.9 for stable 4.0 | 14:34 |
| jrosser | and we use 2.10 with it | 14:34 |
| jrosser | all of this is why i've never done a OSA+ceph | 14:35 |
| mgariepy | i did tried it for an initial install. | 14:36 |
| mgariepy | but moved to separate when i needed to do the first ceph update :D | 14:36 |
| jrosser | yeah :) | 14:36 |
| jrosser | admin0: key thing here is that you can get ceph installed from several places | 14:37 |
| jrosser | either the ubuntu repos, the download.ceph.com repos or UCA | 14:37 |
| jrosser | just becasue there is a apt sources list installed with mimic in the name doesnt mean it's the only place ceph can come from | 14:37 |
| jrosser | if one of the other repos has a later version which apt decides is to be installed, then thats what you get | 14:38 |
| *** dave-mccowan has quit IRC | 14:38 | |
| jrosser | it is why it is important to have proper apt pins as well as the correct repos | 14:38 |
| *** spatel_ has joined #openstack-ansible | 14:42 | |
| *** spatel_ is now known as spatel | 14:42 | |
| openstackgerrit | Jonathan Rosser proposed openstack/ansible-role-pki master: WIP - Create server certificates https://review.opendev.org/c/openstack/ansible-role-pki/+/788021 | 14:43 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: WIP - Test PKI role https://review.opendev.org/c/openstack/openstack-ansible/+/788031 | 14:44 |
| *** dave-mccowan has joined #openstack-ansible | 14:44 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server master: DNM - Test PKI role https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/788032 | 14:45 |
| admin0 | jrosser, " it is why it is important to have proper apt pins as well as the correct repos" -- isn't that something to be expected from the osa playbooks on a greenfield setup ? | 14:46 |
| jrosser | theres no testing done of ceph upgrades | 14:46 |
| jrosser | you'll get a working ceph on a greefield deployment of a particular branch | 14:47 |
| admin0 | i did found this: https://gist.githubusercontent.com/a1git/d7ea1f7a33d5f054bafd90ad75bcb928/raw/b829beecef5750c5c0e0f1f9ba3a34d4ec61a89c/gistfile1.txt | 14:47 |
| admin0 | where ansible is pinning it .. just that it defaults to a lower version | 14:47 |
| admin0 | i think i can move ahead by adding version=13.2.10 and it will fix itself | 14:49 |
| jrosser | https://github.com/openstack/openstack-ansible/blob/master/playbooks/common-tasks/ceph-server.yml#L33-L42 | 14:51 |
| jrosser | from your paste the stuff is set up to prefer the download.ceph.com repos on xenial and archive.ubuntu.com on bionic | 14:53 |
| *** macz_ has joined #openstack-ansible | 14:54 | |
| admin0 | what is the recommended way to fix it ? | 14:54 |
| jrosser | find out why o=Ubuntu is in that file | 14:55 |
| jrosser | thats whats deciding which is the preferred repo | 14:55 |
| admin0 | jrosser, i think its this: https://github.com/openstack/openstack-ansible/blob/stable/rocky/playbooks/ceph-install.yml#L95-104 | 15:00 |
| jrosser | This is rocky you're doing? | 15:00 |
| admin0 | yes .. I am doing an upgrade from xenial -> bionic | 15:01 |
| admin0 | so right now, controller2 is the only controller reformatted with bionic and following the upgrade doc . all services came up OK in bionic in the same 18.1.20 version, except ceph-mon | 15:01 |
| jrosser | well, i guess you have to remember that this code was written at a point in time where the contents of the Ubuntu repos are not necessarily the same as they are today | 15:03 |
| jrosser | specifically https://github.com/openstack/openstack-ansible/commit/66a13d7f8c6b54b230886f48c901d5cd5be78a74 | 15:03 |
| jrosser | though that is a recent patch | 15:04 |
| admin0 | i will try to make it ceph.conf (same as in xenial) and attempt again | 15:06 |
| admin0 | once upgraded, then for the rest, i can just follow stein -> train -> ussuri -> victoria | 15:06 |
| jrosser | well what ceph version do we actually expect for a rocky deployment? | 15:06 |
| admin0 | my main goal now is to do this without downtime, sofor the purpose of upgrade, if its on mimic . and later ( s/t/u/v ) upgrades it to a diff one, that is good enough | 15:07 |
| jrosser | it should be luminous https://github.com/openstack/openstack-ansible/blob/stable/rocky/inventory/group_vars/all/ceph.yml#L28 | 15:07 |
| admin0 | rocky did allowed mimic also, so this one is on mimic: ceph_origin: repository ceph_repository: community ceph_stable_release: mimic | 15:09 |
| admin0 | though i was surprized that having this on variables, the bionic reinstall pocked up luminous and bypassed whats on user_variables | 15:10 |
| admin0 | *picked up | 15:10 |
| jrosser | it's because of the apt pins | 15:10 |
| admin0 | i am goign to add back apt_pinned_packages: [{ package: '*', release: 'ceph.com' }] and hope for the best :) | 15:10 |
| admin0 | which will be just for the 18.1.20 only | 15:11 |
| jrosser | are you putting that in user_variables? | 15:12 |
| admin0 | no .. editing the playbook that sets this up directly | 15:13 |
| jrosser | ok, good - that shouldnt go in user_variables | 15:13 |
| jrosser | i think you do have to take care with keeping the appropriate version of ceph | 15:14 |
| jrosser | as you need to match up ansible version + osa version + ceph_ansible version simulatnously | 15:14 |
| jrosser | and back in the rocky days that was really tricky to get them all to be simultaneously compatible | 15:15 |
| admin0 | assuming upgrade from xenial -> bionic works .( and I am on mimic now ) .. what version do I use when upgrading to stein, train , ussuri and then victoria ? | 15:16 |
| admin0 | or i can keep using mimic till the end, setup osa to latest, and then slowly go up 1 level in ceph by only running ceph-playbooks ? | 15:16 |
| jrosser | you would look at the version defined in group_vars/all/ceph.yml | 15:32 |
| jrosser | thats the one which is tested, you wont be able to keep mimic because the version of ansible incrememts on most OSA releases | 15:33 |
| jrosser | and as you can see here https://docs.ceph.com/projects/ceph-ansible/en/stable-5.0/#releases the different versions of ceph-ansible (which also increments on most OSA releases) require very specific ansible versions | 15:34 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Enable senlin tempest plugin when senlin is deployed https://review.opendev.org/c/openstack/openstack-ansible/+/754105 | 15:37 |
| jrosser | noonedeadpunk: i've done another big set of stuff on the pki role - probably worth taking a look soon at how thats used in rabbitmq here https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/788032/ | 15:46 |
| jrosser | next thing i'll do (next week) is an infra scenario AIO and see if a whole cluster works | 15:47 |
| jrosser | i'm not really sure what we do with the haproxy role as there is lots of complexity there already | 15:47 |
| admin0 | quick question .. for xenial -> bionic, once the 2 (non-primary) controllers are done, do I have to do anything new to do the last one ( c3, primary repo) , or i can just reformat and redo it in bionic ? .. and once all controllers are in bionic, can I just do a release-upgrade for xenial -> bionic on the compute nodes and the OSD's ? | 15:51 |
| *** gyee has joined #openstack-ansible | 15:52 | |
| noonedeadpunk | jrosser: that's really impressive | 15:52 |
| jrosser | i forgot to update the commit message, but it creates it's own CA if you dont set the openstack_pki_<blah> varaibles | 15:53 |
| jrosser | that lets the role be used standalone | 15:54 |
| noonedeadpunk | (and remove test file jsut in case) | 15:54 |
| noonedeadpunk | from rabbit one I meant | 15:54 |
| *** gokhani has quit IRC | 15:55 | |
| jrosser | oh yes! | 15:58 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/ansible-role-pki master: Add boilerplate ansible role components https://review.opendev.org/c/openstack/ansible-role-pki/+/774620 | 16:02 |
| *** spatel has quit IRC | 16:24 | |
| *** spatel_ has joined #openstack-ansible | 16:25 | |
| *** spatel_ is now known as spatel | 16:25 | |
| noonedeadpunk | jrosser: am I missing smth that `---` is not needed in vars files? I think they're still yaml? | 16:26 |
| noonedeadpunk | but actually this looks just super smart | 16:28 |
| noonedeadpunk | and pretty readable | 16:29 |
| *** macz_ has quit IRC | 16:40 | |
| *** oleksandry has joined #openstack-ansible | 16:48 | |
| *** rpittau is now known as rpittau|afk | 16:59 | |
| *** oleksandry has quit IRC | 17:04 | |
| *** andrewbonney has quit IRC | 17:19 | |
| jrosser | no you are right, i think we use them by convention elsewhere | 17:22 |
| jrosser | it's been kind of crazy times trying to concentrate on this between meetings so theres some rough bits to tidy | 17:23 |
| jrosser | perhaps considering if the way all the paths are generating in the pki_dir would be interesting, feels all a bit complex and loads of vars somehow | 17:24 |
| noonedeadpunk | yeah I was looking on pki_dir creation loop thinking how it can be moved into include, but seems not the best idea... | 17:46 |
| noonedeadpunk | any thoughts regarding https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/788501 ? | 17:49 |
| noonedeadpunk | I think actually the main issue out of everything was replacing sudoers before service restart | 17:49 |
| noonedeadpunk | it might be limited to only this part | 17:49 |
| noonedeadpunk | but symlinking during restart might be also good idea overall I think... | 17:50 |
| openstackgerrit | Merged openstack/openstack-ansible-os_senlin master: Use global service_install_method for senlin https://review.opendev.org/c/openstack/openstack-ansible-os_senlin/+/777315 | 18:05 |
| *** dave-mccowan has quit IRC | 18:06 | |
| *** dave-mccowan has joined #openstack-ansible | 18:07 | |
| *** macz_ has joined #openstack-ansible | 18:09 | |
| *** oleksandry has joined #openstack-ansible | 18:10 | |
| *** macz_ has quit IRC | 18:14 | |
| *** oleksandry has quit IRC | 18:19 | |
| *** macz_ has joined #openstack-ansible | 19:25 | |
| *** macz_ has joined #openstack-ansible | 19:26 | |
| *** oleksandry has joined #openstack-ansible | 19:28 | |
| *** dwilde has joined #openstack-ansible | 19:33 | |
| *** dwilde has quit IRC | 19:50 | |
| *** dwilde has joined #openstack-ansible | 20:00 | |
| *** dwilde has quit IRC | 20:05 | |
| *** dwilde has joined #openstack-ansible | 20:06 | |
| *** partlycl- has joined #openstack-ansible | 20:14 | |
| *** partlycloudy has quit IRC | 20:14 | |
| *** macz_ has quit IRC | 20:37 | |
| *** dwilde has quit IRC | 21:05 | |
| openstackgerrit | Oleksandr Yeremko proposed openstack/openstack-ansible-specs master: Protecting plaintext configs https://review.opendev.org/c/openstack/openstack-ansible-specs/+/788829 | 21:18 |
| *** spatel has quit IRC | 21:41 | |
| *** luksky has quit IRC | 22:24 | |
| *** dwilde has joined #openstack-ansible | 23:05 | |
| *** dwilde has quit IRC | 23:12 | |
| *** tosky has quit IRC | 23:40 | |
| *** spatel_ has joined #openstack-ansible | 23:59 | |
| *** spatel_ is now known as spatel | 23:59 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!