| *** ysandeep|out is now known as ysandeep | 04:37 | |
| *** raukadah is now known as chandankumar | 04:48 | |
| noonedeadpunk | jrosser_: yup, it's https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/859337 | 07:15 |
|---|---|---|
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rabbitmq_server stable/xena: Upgrade erlang version https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/860208 | 07:29 |
| noonedeadpunk | NeilHanlon: no, that's valid failure... doh | 07:30 |
| *** ysandeep is now known as ysandeep|lunch | 08:08 | |
| *** ysandeep|lunch is now known as ysandeep | 10:18 | |
| noonedeadpunk | hm, what does provide tgtadm.... | 10:28 |
| jrosser_ | for ubuntu https://packages.ubuntu.com/search?searchon=contents&keywords=tgtadm&mode=exactfilename&suite=jammy&arch=any | 10:47 |
| jrosser_ | but i am going to guess this is centos | 10:47 |
| anskiy | afaik, osa doesn't install tgtadm for centos, as it's using rdstool from cinder venv | 10:55 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_cinder master: Add nfs and ceph jobs to templates https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/859337 | 10:57 |
| noonedeadpunk | yeah, for centos it should be jsut different helper be used | 10:57 |
| noonedeadpunk | jrosser_: oh, well I thought we use lioadm for redhat and tgtadm for ubuntu... | 10:58 |
| noonedeadpunk | https://opendev.org/openstack/openstack-ansible-os_cinder/src/branch/master/defaults/main.yml#L226-L228 | 10:58 |
| noonedeadpunk | Maybe we indeed can just use rdstool if it's smth universal.... | 10:58 |
| anskiy | noonedeadpunk: that should be the case: lio on red hat and tgtadm on deb | 10:59 |
| noonedeadpunk | also I realized that our cinder role does need love | 10:59 |
| noonedeadpunk | As no reason to define bunch of iscsi stuff for rdb or nfs I guess | 11:00 |
| anskiy | I don't think you can use rdstool with tgt, as it is specifically designed to be used with kernel-space targets | 11:00 |
| anskiy | that is, all its code is just messing with stuff in /sys | 11:00 |
| noonedeadpunk | yeah, well I never used rdstool so was not sure how it does work | 11:01 |
| anskiy | and the reasoning behind using lio on rhel is that it's theirs recommended way to usi iSCSI | 11:02 |
| anskiy | well, that thing can support discard/unmap, whilst tgtadm can't, at least that's what I've found on the internets. The downside of LIO is that it's just one kernel thread (not sure if you could scale it up a bit via /sys knobs), whereas tgtd is multithreaded | 11:04 |
| anskiy | still, I'm using tgt now for my first couple of customers VMs, and I already can't wait till I throw this thing away :) | 11:06 |
| noonedeadpunk | But is discard a required thing for thinpool lvm? My recollections from Xen time does bring memories of free space being reclaimed somehow by default | 11:07 |
| noonedeadpunk | THough, I think that discard was added to boot options | 11:07 |
| noonedeadpunk | but xen was using lvm natively and not through iscsi | 11:08 |
| anskiy | it is required. For thin LVM volume you start small and then it grows until allocated size == max size, unless you can prevent it with discarad/unmap | 11:09 |
| noonedeadpunk | but hm, I think we actually used kvm as well these days... | 11:09 |
| anskiy | and, my bad: the problem is not that tgt does not allow discard/unmap -- it's that it send that command in a way, that thin LVM doesn't understands it | 11:09 |
| noonedeadpunk | but if you have an image that does have discard defined when formating drive - do you also need tgt to pass it as well? | 11:11 |
| noonedeadpunk | tbh I never used lvm for cinder, it was always ceph or nfs... | 11:11 |
| anskiy | and the only suggestion that I've found (without diving into tgt's code) was this: https://github.com/mwilck/discard0#questions-and-anwers. tgt sends FALLOC_FL_PUNCH_HOLE which LVM doesn't understand | 11:11 |
| noonedeadpunk | yeah, well, that sucks then... | 11:11 |
| anskiy | noonedeadpunk: you need four things: first is discard mount option/fstrim in guest, then unmap in libvirt domxml, next is allowing to pass it via your iSCSI implementation and then it's LVM | 11:13 |
| anskiy | and all of these four things should "speak the same language" :) | 11:13 |
| noonedeadpunk | yeah, I knew about 3 of them, but as I said never used lvm and kvm through iscsi | 11:13 |
| noonedeadpunk | good to know | 11:13 |
| anskiy | well, it's more like "that thing would do for now", as at the beginning I wasn't ready to setup and deal with ceph cluster along with openstack itself, which now I would see as a mistake, but well... | 11:15 |
| noonedeadpunk | I wonder if blockio will be same in terms of discard? | 11:16 |
| * noonedeadpunk needs to read more docs | 11:17 | |
| anskiy | besides, nova desperately needs a patch to allow uploading directly to Ceph when you aren't locating emphemeral storage on Ceph too: that thing would just go through Glance's HTTP | 11:17 |
| noonedeadpunk | well, it depends I'd say | 11:18 |
| anskiy | noonedeadpunk: it defaults to fileio and I don't specifically remember, but for some reason I was assured that it was the right setting for LVM+iSCSI | 11:18 |
| noonedeadpunk | as this can be also considered as way to backup things | 11:19 |
| noonedeadpunk | I think it's up to user to define fileio or blockio. from what I read blockio does report directly to lvm, while fileio goes through cache | 11:20 |
| noonedeadpunk | So with blockio you don't get cache and thus loose in performance, but when talking about hypervisor with dozens of vms, likely it doesn't make that much sense | 11:21 |
| noonedeadpunk | anskiy: also why to use epehemeral at all ? :D | 11:21 |
| anskiy | noonedeadpunk: quick grep in cinder's source finds iscsi_iotype only in `cinder/volume/targets/iet.py` | 11:22 |
| noonedeadpunk | Um https://opendev.org/openstack/cinder/src/branch/master/cinder/volume/drivers/lvm.py#L126 | 11:23 |
| anskiy | yeah, but that's just and option declaration | 11:23 |
| anskiy | and it says IET, which, I suppose, some sort of target, which is different from LIO and TGT | 11:24 |
| anskiy | noonedeadpunk: is this some wording joke? :) | 11:25 |
| noonedeadpunk | well, I mean they can pass get_driver_options jsut as kwargs somewhere as well | 11:26 |
| noonedeadpunk | but have no idea if it's used for real | 11:27 |
| anskiy | noonedeadpunk: jokes aside, I still would prefer local storage first (especially when you have working block live migration now), and would add Ceph for it as another option | 11:28 |
| noonedeadpunk | regarding ephemeral? Nah, it's just that usecases for ephemerals are not really wide. While they have some neats and likely for CI/CD and local storage it's nice to have but overall - meh | 11:29 |
| *** dviroel|afk is now known as dviroel | 11:30 | |
| noonedeadpunk | And with ephemerals you need to spawn waaaaay more flavors then jsut use BFV | 11:31 |
| anskiy | well, from quickly poking it, I can only see that one downside for "backups" (image from existent instance), which I hope to rework on my side and upstream one day. And there are many properties like "preserve ephemeral" on rebuild and such | 11:31 |
| noonedeadpunk | But again, why given there's cinder? | 11:32 |
| noonedeadpunk | And also you need to lock compute per storage backend. | 11:32 |
| noonedeadpunk | So if you want to have slow/fast ceph pools - you can't do that with ephemerals (or well only based on computes and aggregates) | 11:33 |
| anskiy | because all the cinder drivers for "local" storage are bad :) You can match flavor property to aggregate | 11:34 |
| noonedeadpunk | yeah, the local storage is only valid usecase I'd say. But for local storage you still will need to upload images through API | 11:34 |
| anskiy | yes, unless, you patch Nova :) | 11:35 |
| anskiy | and then, you don't! :) | 11:35 |
| noonedeadpunk | um, I do know how to do that with ceph. But does lvm also have COW nowadays? | 11:36 |
| KSR | Hello - testing testing | 11:37 |
| jrosser_ | hello KSR | 11:38 |
| KSR | Thanks, just testing if I was able to write without a certificate | 11:38 |
| KSR | We are running openstack ussuri latest 21.2.10 and we have a known bug: https://review.opendev.org/c/openstack/horizon/+/829706 | 11:39 |
| KSR | Are there any plans to add this to a release or can it be cherrypicked by hand? | 11:39 |
| anskiy | noonedeadpunk: you already have this mechanic in nova: https://opendev.org/openstack/nova/src/branch/master/nova/image/glance.py#L227, which bypasses Glance HTTP and goes directly to Ceph to get the desired Image, so, there should be added some `_upload_handlers` here: https://opendev.org/openstack/nova/src/branch/master/nova/image/glance.py#L222, like for download | 11:40 |
| noonedeadpunk | KSR: you can set `horizon_git_install_branch: 920cbb82d20be33074eec52abc6022f5ff97e569` (it's SHA hash of this patch) to your user_variables | 11:41 |
| jrosser_ | KSR: note that ussuri is a release in "extended maintainance" now https://docs.openstack.org/openstack-ansible/latest/ | 11:41 |
| KSR | Ok thanks I'll try that. Any plans to include it in a release? | 11:42 |
| noonedeadpunk | we can't make any new releases of Ussuri because of EM state | 11:42 |
| KSR | And do you know if it's included in future releases? If so we migth just consider an upgrade | 11:42 |
| anskiy | noonedeadpunk: then you just `if` here: https://opendev.org/openstack/nova/src/branch/master/nova/virt/libvirt/driver.py#L3107 for some variable, like `glance.enabled_rbd_upload`, I hope this wouldn't be too hard for me :) | 11:43 |
| KSR | noonedeadpunk: Thanks | 11:43 |
| noonedeadpunk | KSR: I'm quite sure it is included | 11:43 |
| KSR | Thanks guys! | 11:43 |
| jrosser_ | KSR: on the patch you linked see "cherry picks" at the top right, you can see which branches that code is on | 11:43 |
| noonedeadpunk | anskiy: my point returns back to why use epehemerals for ceph as a concept | 11:43 |
| noonedeadpunk | cinder does have ceph driver and it's quite good maintained | 11:44 |
| jrosser_ | but like noonedeadpunk says it is really trivial to update the SHA for horizon in your existing deployment | 11:44 |
| anskiy | noonedeadpunk: nono, ephemeral with local storage, I just need to directly upload images to Glance Ceph pool | 11:44 |
| noonedeadpunk | ahhhhhh | 11:44 |
| noonedeadpunk | ok, gotcha | 11:44 |
| anskiy | hence I would need that patch | 11:44 |
| noonedeadpunk | yes, true, that does make sense indeed | 11:45 |
| KSR | jrosser_: Where to cherry pick openstack horizon source? | 11:45 |
| noonedeadpunk | KSR: you don't need to | 11:45 |
| jrosser_ | you already have the example.... | 11:45 |
| jrosser_ | noonedeadpunk> KSR: you can set `horizon_git_install_branch: 920cbb82d20be33074eec52abc6022f5ff97e569` (it's SHA hash of this patch) to your user_variables | 11:45 |
| noonedeadpunk | patch already merged to stable/ussuri. So using SHA wil ldo the trick | 11:45 |
| KSR | noonedeadpunk: Thanks I'll do that then.. Just wanted to know if there was an alternative. | 11:46 |
| jrosser_ | KSR: in a lot of ways a point release of OSA just updates a bunch of SHA in a file | 11:46 |
| jrosser_ | you can do exactly the same in your user_variables.yml whenever you need | 11:46 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Bump ansible-core version to 2.13.4 https://review.opendev.org/c/openstack/openstack-ansible/+/857506 | 12:19 |
| *** frenzyfriday is now known as frenzyfriday|food | 12:22 | |
| *** ysandeep is now known as ysandeep|out | 14:04 | |
| *** dviroel is now known as dviroel|lunch | 14:50 | |
| noonedeadpunk | #startmeeting openstack_ansible_meeting | 15:00 |
| opendevmeet | Meeting started Tue Oct 4 15:00:14 2022 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| opendevmeet | The meeting name has been set to 'openstack_ansible_meeting' | 15:00 |
| noonedeadpunk | #topic rollcall | 15:00 |
| noonedeadpunk | o/ | 15:00 |
| NeilHanlon | o/ double booked for this but will have it up on the screen :) g'day folks | 15:00 |
| noonedeadpunk | thanks! | 15:01 |
| jrosser_ | o/ hello | 15:01 |
| noonedeadpunk | #topic office hours | 15:05 |
| noonedeadpunk | So first thing - Xena is broken due to cloudsmith dropped erlang version we were using | 15:05 |
| noonedeadpunk | This also blocking Xena as upgrade jobs are also broken | 15:06 |
| noonedeadpunk | *Yoga | 15:06 |
| noonedeadpunk | To unblock we need this https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/860208 | 15:06 |
| noonedeadpunk | But also, what's more important, I'm not sure if we should continue consuming cloudsmith.... | 15:07 |
| noonedeadpunk | I really can't follow their updates and don't really understand policies either | 15:07 |
| noonedeadpunk | As they've dropped completely erlang 23 now but some 22 are still there | 15:07 |
| noonedeadpunk | But what's worse - I'm not sure what alternatives are | 15:08 |
| noonedeadpunk | Next thing - https://review.opendev.org/q/topic:osa%252Ftest_volumes seems passing and functional now. So feel free to review that | 15:10 |
| noonedeadpunk | Question about mariadb version bump: https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/858555 - should we backport it? As sounds like it's worthy thing to backport considering it will allow to sync debian and redhat versions | 15:11 |
| anskiy | I'm running 10.6.9 for quite some time now (since I've found that fix for not backuping redo log, if you remember) and it seems fine (if that helps in any way). | 15:13 |
| noonedeadpunk | yep, that's quite useful info I'd say | 15:15 |
| jrosser_ | sounds reasonable to backport - though it's unusual | 15:16 |
| noonedeadpunk | yeah, we should add release note when backporting | 15:17 |
| jrosser_ | then this is lingering around https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/855985 | 15:19 |
| noonedeadpunk | Yeah. I think we need to drop debian-11 from there | 15:20 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rabbitmq_server master: Update rabbitmq to 3.10.7-1 https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/855985 | 15:20 |
| noonedeadpunk | I kind of wonder when next long-term mariadb will be... | 15:21 |
| noonedeadpunk | As I feel somehow uncomfortable with 10.6 when having 10.10 as current one | 15:22 |
| noonedeadpunk | I'm already unsure if there will be any LTS release in the future.... | 15:23 |
| noonedeadpunk | Ah, found, `A new long-term support MariaDB Server version will be announced for General Availability approximately once every two years` | 15:24 |
| noonedeadpunk | 10.6 was released 6 Jul 2021. So one more year to wait.... | 15:25 |
| noonedeadpunk | I hope there will be upgrade path lol | 15:26 |
| noonedeadpunk | btw, ansible 2.13 now fails on Rocky 9 somehow | 15:27 |
| noonedeadpunk | oh hardening role | 15:29 |
| noonedeadpunk | it also somewhat fails on debian git-core isntallation which is super weird | 15:30 |
| noonedeadpunk | rocky is quite consistant on where it fail: https://zuul.opendev.org/t/openstack/build/6a530b60ab1342baae9adac39368d796/log/job-output.txt#7921 | 15:31 |
| noonedeadpunk | oh my https://opendev.org/openstack/ansible-hardening/src/branch/master/library/get_users | 15:33 |
| noonedeadpunk | lol, TIL about that | 15:34 |
| jrosser_ | oh i was just going to say i bet thats a python3 problem | 15:35 |
| noonedeadpunk | how that works for other distros | 15:36 |
| jrosser_ | $massive_hack ? | 15:36 |
| noonedeadpunk | yeah.... | 15:36 |
| noonedeadpunk | Should we move that to plugins? | 15:36 |
| noonedeadpunk | As hardening used to be quite "stadalone" thing | 15:37 |
| noonedeadpunk | I'm not even sure we used config_tmeplate there | 15:37 |
| jrosser_ | no | 15:37 |
| jrosser_ | i dunno really - theres not really any reuse of that module anywhere | 15:37 |
| jrosser_ | so moving it to plugins is not totally necessary | 15:38 |
| noonedeadpunk | And I can't imagine possible usecase elsewhere | 15:38 |
| NeilHanlon | 🤔 | 15:38 |
| jrosser_ | maybe better to fix it in place first | 15:38 |
| noonedeadpunk | agree | 15:38 |
| noonedeadpunk | does even modules need shebang? | 15:40 |
| noonedeadpunk | how does all stuff in https://opendev.org/openstack/openstack-ansible-plugins/src/branch/master/plugins/modules work lol | 15:40 |
| noonedeadpunk | or it's also $massive_hack.... | 15:41 |
| NeilHanlon | ansible docs for writing modules use shebang.. but not clear if needed | 15:41 |
| noonedeadpunk | shouldn't they be using ansible_python_interpreter.... | 15:43 |
| noonedeadpunk | Oh, and for rocky we don't have discovered_python_interpreter, that's why it falls back to shebang I guess | 15:43 |
| NeilHanlon | https://drop1.neilhanlon.me/irc/uploads/72ea47b1e577fbfc/image.png | 15:43 |
| NeilHanlon | from: https://docs.ansible.com/ansible/latest/dev_guide/developing_program_flow_modules.html#id14 | 15:44 |
| noonedeadpunk | I'd say should be safe to drop shebang then | 15:45 |
| NeilHanlon | agreed | 15:45 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/ansible-hardening master: Remove sebang from get_users https://review.opendev.org/c/openstack/ansible-hardening/+/860276 | 15:46 |
| noonedeadpunk | also sounds like some ansible behaviour has changed with 2.13 regarding to shebangs | 15:46 |
| noonedeadpunk | and they also check for discovered interpreter... | 15:47 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-plugins master: Drop shebangs from ansible modules https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/860277 | 15:49 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Bump ansible-core version to 2.13.4 https://review.opendev.org/c/openstack/openstack-ansible/+/857506 | 15:50 |
| noonedeadpunk | let's see if that helps... | 15:50 |
| noonedeadpunk | though we should really look how to workaround non-defined discovered_python_interpreter for rocky (as we explicitly provide interpreter with env var which disables discovery) | 15:51 |
| noonedeadpunk | in any case that's all I have for you today | 15:53 |
| noonedeadpunk | #endmeeting | 16:00 |
| opendevmeet | Meeting ended Tue Oct 4 16:00:08 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:00 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2022/openstack_ansible_meeting.2022-10-04-15.00.html | 16:00 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/openstack_ansible_meeting/2022/openstack_ansible_meeting.2022-10-04-15.00.txt | 16:00 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2022/openstack_ansible_meeting.2022-10-04-15.00.log.html | 16:00 |
| *** dviroel|lunch is now known as dviroel | 16:21 | |
| psymin | Is there an example like this for centos/rocky ? https://docs.openstack.org/openstack-ansible/yoga/user/network-arch/example.html | 18:25 |
| NeilHanlon | psymin: you can use this as an* example, but I plan to re-do it using systemd-networkd at some point soon. https://git.shrug.pw/linuxadminbooks/AdvancedLinuxAdmin/src/branch/main/ansible/playbooks/files/interfaces.sh | 20:14 |
| NeilHanlon | wherein I use ansible to deploy openstack ansible 🤔 | 20:15 |
| *** dviroel_ is now known as dviroel|afk | 20:37 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
