Monday, 2021-12-13

*** ysandeep|out is now known as ysandeep04:20
*** frenzy_friday is now known as anbanerj|ruck04:35
*** ysandeep is now known as ysandeep|intv06:25
noonedeadpunkadmin1: you should reference env.d for that. I dunno the reason why it's like that, but for tacker it's mano_hosts https://opendev.org/openstack/openstack-ansible/src/branch/master/inventory/env.d/tacker.yml#L3408:03
noonedeadpunkI wonder if it work actually...08:03
jrosserthe really edge-case roles don't get entries in openstack_user_config08:04
jrosserand it is really a very very long time since anyone worked with tacker08:04
noonedeadpunkok, so seems we deploy only api server, not conductor part08:13
noonedeadpunkwhich makes me think it's a legacy deployment https://docs.openstack.org/tacker/latest/user/architecture.html#legacy-tacker-implementation08:14
noonedeadpunkprobably worth impementing also conductor, but it's super hard for me to read their docs as I can hardly understand half of the terminology used....08:15
jrosserlooks like another case where the internal components want to talk to things inside VM08:17
jrosserwonder how thats implemented08:17
noonedeadpunkbut I bet it would need it's own network :)08:23
*** ysandeep|intv is now known as ysandeep08:51
*** ysandeep is now known as ysandeep|lunch08:58
admin1thanks guys .. i literally looked everywhere and the docs .. but forgot to look into the env.d samples :) 09:10
admin1i was looking for a good system to do the l09:19
*** ysandeep|lunch is now known as ysandeep09:21
admin1sorry  for the half sentence 09:25
admin1i was looking for a system that allows me to do logging  ( graylog, filebeat etc ) .. but without effort .. but found this .. https://wazuh.com/   -- works out of the box 09:26
noonedeadpunkI wonder how venus feels nowadays btw...10:28
noonedeadpunkdoesn't look like it has ever released :(10:29
noonedeadpunkoh, it requires agent to be installed - nah....10:29
noonedeadpunkthat not smth that can work for us :(10:30
jrosseri was thinking we had something with wazuh in the ops repo10:31
*** dviroel|out is now known as dviroel|rover10:48
*** dpawlik7 is now known as dpawlik11:56
opendevreviewOpenStack Release Bot proposed openstack/openstack-ansible stable/xena: Update .gitreview for stable/xena  https://review.opendev.org/c/openstack/openstack-ansible/+/82154712:39
opendevreviewOpenStack Release Bot proposed openstack/openstack-ansible stable/xena: Update TOX_CONSTRAINTS_FILE for stable/xena  https://review.opendev.org/c/openstack/openstack-ansible/+/82154812:39
opendevreviewOpenStack Release Bot proposed openstack/openstack-ansible master: Update master for stable/xena  https://review.opendev.org/c/openstack/openstack-ansible/+/82154912:39
*** dviroel is now known as dviroel|rover13:08
jamesdentongood morning14:06
admin1\o 14:12
noonedeadpunk\o/14:12
admin1jamesdenton . any new books coming up ? 14:13
jamesdentonpfft14:13
noonedeadpunkrevisions of old ones ?:D14:13
admin1  i have your signed one 14:13
admin1you can do the whole ovn, complete ipv6,   vxlan breakout to vlans for some "cloud connect" solutions .. lots of stuff :) 14:14
jamesdentonit could be nice, as it's always a good opportunity to learn new stuffs. it comes down to lack of bandwidth these days14:14
jamesdentonadmin1 agreed, would love to see something like that14:14
jamesdentonthings are starting to free up. my kids are getting a little older and i've got more time for blogs, at least14:15
admin1nice  nice . 14:16
jamesdentontrying to close out 2021 by closing the loop on patches started in 2019 :D14:17
opendevreviewMerged openstack/openstack-ansible stable/xena: Update .gitreview for stable/xena  https://review.opendev.org/c/openstack/openstack-ansible/+/82154714:31
*** ysandeep is now known as ysandeep|dinner14:34
opendevreviewMerged openstack/openstack-ansible stable/xena: Update TOX_CONSTRAINTS_FILE for stable/xena  https://review.opendev.org/c/openstack/openstack-ansible/+/82154814:35
opendevreviewMerged openstack/openstack-ansible master: Update master for stable/xena  https://review.opendev.org/c/openstack/openstack-ansible/+/82154914:38
*** dviroel|rover is now known as dviroel|rover|lunch15:01
spateljamesdenton +1 i would like to see new book :) let me know if you need help for any testing for scenario testing :D15:10
jamesdentonyou'll be the first to know15:11
spatel++1 15:13
*** ysandeep|dinner is now known as ysandeep15:14
spatelnoonedeadpunk after upgrading to RabbitMQ 3.8.14 its much stable now. its been a two weeks and still up and running. (i am running with non-HA :) ) 15:15
spatelnoonedeadpunk did you work on patch to create rabbitmq non-HA queue for specific component? 15:17
spatelno rush just curious so i can use that in my X upgrade 15:18
noonedeadpunknope, we haven't done that for X15:19
spatelok15:23
noonedeadpunkthis would require patching all roles as it implements variable for them15:26
noonedeadpunkbut I think you can define rabbitmq policy in group vars with quite same effect15:26
spatelyou are saying make group vars for each component whoever you want to create policy and no policy correct? 15:28
jamesdentonspatel you're using single queues?15:35
spatelsingle queue?15:36
spatelOSA use /vhost to deploy multiple queue 15:36
spateljamesdenton ^ is that what you askin?15:39
jamesdentonsorry, i mean ha vs non-ha queues15:41
noonedeadpunkspatel: you can set no policy overall and set value for specific group, yes15:45
spateljamesdenton i have remove HA policy for all components like nova/neutron/heat etc.. so currently everything is running non-HA in production. (In short my policy section is empty :) )15:46
jamesdentonand what's your feedback on that configuration?15:47
spatelnoonedeadpunk good to know i will give it a try and if works then we should put small snippet in offical doc about how to that 15:47
spateljamesdenton everything works well. i did some failover testing also and it works i didn't see any issue at all. 15:47
spateli have killed rabbitMQ on one of node and automatically queues get created on other remaining nodes. (yes i lost data but RPC calls doesn't have any data :) )15:48
noonedeadpunkI would prefer adding variable for that, but this workaround should still work15:49
spatelif someone trying to create/delete vm and during that time node die then you will see ERROR mesg but if you try again you will get success 15:49
spateli have noticed neutron has 3x queues compare to other services. neutron is biggest consumer of rabbitMQ - If we go OVN then it will reduce lots of issue of rabbitMQ. 15:50
spatelnoonedeadpunk i would also like to have small variable which control HA vs non-HA (for simplicity )15:52
noonedeadpunknah, I don't think we will do that... as eventually you need to fully set a policy for that. 15:52
noonedeadpunkOh, well, at least with current (obsolete) way of ha15:52
spatelwhy are we creating keystone /vhosts ? keystone doesn't use rabbitMQ 15:54
jrosseri think that andrewbonney was taking a look at the rabbitmq connection stuff very similar to how he did for galera15:54
jrosseras the number of connections is kind of surprisingly large15:54
spatelI have 5685 queue for neutron in 250 compute nodes, that is very large number. 15:56
spatel4375 total connection for neutron service 15:56
spatelalmost 20 TCP connection from each compute nodes. 15:59
jrosserwell like i say there are sort of similar concepts of connection pool and per thread/process settings16:01
jrosserand if these are not appropriate then the numbers will get large, quickly16:01
spatelagreed16:02
*** dviroel|rover|lunch is now known as dviroel|rover16:12
noonedeadpunkoh, I never looked into this tbh...16:17
noonedeadpunkwell, I don't experience any rabbit issues nowadays though as well...16:18
jrosserwe're having quite some trouble16:18
jrosserexhaustion of fd somewhere inside/below oslo.messaging in compute nodes16:19
noonedeadpunkouch, yeah, then it's understandable how you started looking into that16:20
jrossernova-compute is doing something that we can't reproduce with a test program yet16:23
admin1anyone using kata containers with openstack ? can provide a few pointers ? 16:28
*** ysandeep is now known as ysandeep|out16:31
spateljrosser i have only basic components in my cloud so not using many fd connection. possible you have all other components chewing up more resources like celiometer, magnum etc.. 16:47
jrosserspatel: it's really about the compute node configuration i think, not to do with the other components16:55
spatelok16:56
jrosseroh this is good https://review.opendev.org/c/openstack/openstack-ansible/+/82147617:25
spatelgood in sense of? 17:51
admin1we will have xena soon :D17:51
spatelits been a while X is put but i can understand, we are little behind.17:52
jrossernot really behind at all https://releases.openstack.org/yoga/schedule.html#y-cycle-trail17:58
*** sshnaidm is now known as sshnaidm|afk19:06
opendevreviewMerged openstack/openstack-ansible-tests stable/xena: Update TOX_CONSTRAINTS_FILE for stable/xena  https://review.opendev.org/c/openstack/openstack-ansible-tests/+/82084619:30
*** dviroel|rover is now known as dviroel|rover|afk21:02
spateljrosser :) good to know23:00

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!