Thursday, 2020-11-19

*** k_mouza has joined #openstack-kolla00:00
*** k_mouza has quit IRC00:05
*** LinPeiWen46 has joined #openstack-kolla00:36
*** ricolin has joined #openstack-kolla00:37
openstackgerritLin PeiWen proposed openstack/kolla-ansible stable/victoria: Add support for GlusterFS NFS Manila backend  https://review.opendev.org/76329200:43
*** wuchunyang has joined #openstack-kolla00:56
*** zzzeek has quit IRC02:17
*** zzzeek has joined #openstack-kolla02:19
*** jobewan has joined #openstack-kolla02:27
*** ivan_lin has joined #openstack-kolla02:30
*** LinPeiWen has quit IRC02:32
*** zzzeek has quit IRC02:35
*** zzzeek has joined #openstack-kolla02:36
*** skramaja has joined #openstack-kolla03:10
*** bsanjeewa has joined #openstack-kolla03:16
*** bsanjeewa has quit IRC03:38
*** imtiazc has quit IRC04:01
*** wuchunyang has quit IRC04:06
*** kevko has quit IRC04:11
*** vishalmanchanda has joined #openstack-kolla04:30
*** jaicaa has quit IRC04:56
*** jaicaa has joined #openstack-kolla04:56
openstackgerritlikui proposed openstack/kolla-ansible master: Update docs for Ubuntu 20.04  https://review.opendev.org/76316705:16
*** evrardjp has quit IRC05:33
*** evrardjp has joined #openstack-kolla05:33
*** ivan_lin has quit IRC05:36
*** ivan_lin has joined #openstack-kolla05:36
*** k_mouza has joined #openstack-kolla06:01
*** JamesBenson has quit IRC06:03
*** JamesBenson has joined #openstack-kolla06:04
*** k_mouza has quit IRC06:06
*** JamesBenson has quit IRC06:09
*** JamesBenson has joined #openstack-kolla06:15
*** kevko has joined #openstack-kolla06:33
*** jaicaa has quit IRC06:37
*** jaicaa has joined #openstack-kolla06:39
*** cah_link has joined #openstack-kolla07:02
*** wuchunyang has joined #openstack-kolla07:08
*** bhagyashris|off is now known as bhagyashris07:55
*** nikparasyr has joined #openstack-kolla07:59
*** rpittau|afk is now known as rpittau08:03
mnasiadkamorning08:27
*** bengates has joined #openstack-kolla08:37
*** imcsk8 has quit IRC08:37
*** imcsk8 has joined #openstack-kolla08:39
*** gfidente|afk is now known as gfidente08:46
*** cah_link1 has joined #openstack-kolla08:51
*** cah_link has quit IRC08:53
*** cah_link1 is now known as cah_link08:53
kevkohi guys, can someone advise me with execution of role in a loop to bunch of servers in several groups ?08:54
kevkoi will explain to detail if someone :) ..08:54
kevkoi really think that ansible can't do this08:55
*** mgoddard has joined #openstack-kolla08:58
*** bsanjeewa has joined #openstack-kolla09:11
bsanjeewamgoddard: about http://paste.openstack.org/show/800169/ yesterday, now both nodes boot into grub> prompt09:12
mnasiadkabsanjeewa: yeah, that's something wrong with DIB (or it's elements), if you type exit in the grub> prompt, you'll get to a menu09:16
mnasiadkabsanjeewa: running grub2-install on a booted OS seems to fix things09:16
*** bsanjeewa_ has joined #openstack-kolla09:18
bsanjeewa_mnasiadka: When type exit on grub>, StartImage failed: Load Error displayed and system does not respond anymore. Will try to locate the kernal and ramdisk images and try the boot command09:20
mnasiadkabsanjeewa_: oh, that's probably some different problem then09:20
*** bsanjeewa has quit IRC09:20
*** skramaja has quit IRC09:29
osmanlicilegimorning09:29
*** bsanjeewa_ has quit IRC09:36
*** ricolin has quit IRC09:38
openstackgerritwu.shiming proposed openstack/kayobe master: Update TOX_CONSTRAINTS_FILE  https://review.opendev.org/76332909:39
openstackgerritMark Goddard proposed openstack/kayobe master: Update TOX_CONSTRAINTS_FILE  https://review.opendev.org/76332909:48
mgoddardmorning09:48
*** bsanjeewa has joined #openstack-kolla09:53
bsanjeewamnasiadka: the /boot/efi folder is empty. will not boot with kernel in the /boot folder. What should I do? Any additional settings in ipa.yml and bifrost.yml?09:55
*** k_mouza has joined #openstack-kolla09:57
openstackgerritwu.chunyang proposed openstack/kolla-ansible master: [CI] Add trove smoke test CI  https://review.opendev.org/76319710:12
openstackgerritMark Goddard proposed openstack/kolla master: docs: add code reviews to contributor guide  https://review.opendev.org/76281710:12
*** bsanjeewa_ has joined #openstack-kolla10:15
*** wuchunyang has quit IRC10:17
*** bsanjeewa has quit IRC10:18
*** luksky has joined #openstack-kolla10:18
openstackgerritMark Goddard proposed openstack/kayobe-config master: Update TOX_CONSTRAINTS_FILE  https://review.opendev.org/76333810:23
openstackgerritMark Goddard proposed openstack/kayobe-config-dev master: Update TOX_CONSTRAINTS_FILE  https://review.opendev.org/76333910:24
*** dardelean has joined #openstack-kolla10:24
openstackgerritMerged openstack/kayobe master: Update master for stable/victoria  https://review.opendev.org/76322110:28
*** bsanjeewa_ has quit IRC10:28
*** e0ne has joined #openstack-kolla10:29
openstackgerritMichal Nasiadka proposed openstack/kolla master: CI: Improve publish playbook  https://review.opendev.org/76314610:34
mnasiadkahrw: commit message updated - happier? :)10:35
openstackgerritMerged openstack/kolla-ansible stable/victoria: kibana: Remove 6.x migration from upgrade  https://review.opendev.org/76321410:36
hrw+210:36
openstackgerritMerged openstack/kolla-ansible master: Fix kolla-ansible to work with pyenv-virtualenv  https://review.opendev.org/76236510:42
openstackgerritMark Goddard proposed openstack/kolla master: Remove footer block from intermediate images  https://review.opendev.org/76302711:03
openstackgerritMark Goddard proposed openstack/kolla master: WIP: CI: revert to public package mirrors after build  https://review.opendev.org/76192811:03
openstackgerritMark Goddard proposed openstack/kolla master: WIP: CI: revert to public package mirrors after build  https://review.opendev.org/76192811:04
openstackgerritMark Goddard proposed openstack/kolla master: WIP: CI: revert to public package mirrors after build  https://review.opendev.org/76192811:07
kevkoanyone for little discussion about kolla-ansible with me please ? :)11:11
openstackgerritMark Goddard proposed openstack/kolla master: DNM: testing mirror revert  https://review.opendev.org/76335111:11
*** cah_link1 has joined #openstack-kolla11:13
rockeykevko: post your question and perhaps someone will have/take the time to answer11:15
*** cah_link has quit IRC11:15
*** cah_link1 is now known as cah_link11:15
kevkook, i would like to implement mysql proxying and database sharding ... we have it on production and working very good, as shard-router and proxy we are using maxscale11:17
kevkomy problem is that i want to write maxscale role , but want to also "recycle" mariadb role11:17
kevkobut if I add servers , or groups to mariadb group in hosts file , site.yml is applying changes to all servers , actually i really need to apply some role to N groups of mariadb clusters11:18
kevkothis can be done by running playbook with limit ... or playbook to specific group .. so i can't include it to site.yml11:19
kevkosecond option is rewrite mariadb role to do both things ..but it is quite hard .. :/ don't know how to deal with it11:19
kevkohope, that it is understandable ... :/11:20
mnasiadkaso, you need to run the maxscale role after the mariadb role, and then it needs to run the mariadb role again?11:24
kevkomaybe I explain problem which we were trying to solve ..11:34
kevkokolla is installing mariadb on some hosts right ? (mariadb group, and to its children , by default controller)11:35
kevkoevery database is after this installed to this cluster, but haproxy is sending always to one server, other servers are backups , right ?11:36
openstackgerritBartosz Bezak proposed openstack/kayobe master: Support for firewalld ZONE option in network interfaces configuration  https://review.opendev.org/76318011:37
kevko^^ this means that most big sense is to have always 2 servers of mariadb, if you have 3,4,N servers, all trafic will be router always to one11:37
kevko^^ this basically means that in some point of cloud usage this one server is not doing his work good ..IO, cpus, network ..etc11:38
kevko(especially neutron DB has big ammount of connections)11:39
kevko^^ So , i want to replace haproxy and place maxscale, which have shardRouter on entrypoint ..and sending all trafic per shard (openstack database_name) to group of mariadb11:40
*** mushrushu has quit IRC11:40
kevkoso, firstly i would like to define SOMEHOW ... install mariadb1_cluster here, mariadb_cluster2 here, mariadb_cluster3 here ..and so on ... and after all clusters(shards) is completly built ..i just deploy maxscale container with all backends configured on controller11:41
kevkopoint is , that i really don't know how to install set of mariadb clusters ...because mariadb role is applying her tasks to all servers :/11:42
kevkois it more understandable now :/ ?11:42
*** mushrushu has joined #openstack-kolla11:43
kevkoin next step i just set shard for openstack service ..and bootstrap of mysql_database will go to specific shard ..while all databases will be accessible via maxscale11:44
*** bsanjeewa_ has joined #openstack-kolla11:46
*** bsanjeewa_ has quit IRC11:53
*** sluna has quit IRC12:10
*** sluna has joined #openstack-kolla12:10
mnasiadkakevko: we understand the motivation to use maxscale, you don't need to explain :)12:11
openstackgerritMerged openstack/kayobe master: Add Python3 wallaby unit tests  https://review.opendev.org/76322212:11
openstackgerritMerged openstack/kayobe-config stable/victoria: Update .gitreview for stable/victoria  https://review.opendev.org/76322312:11
openstackgerritMerged openstack/kayobe-config stable/victoria: Update TOX_CONSTRAINTS_FILE for stable/victoria  https://review.opendev.org/76322412:11
mnasiadkakevko: but I don't understand why do you want to change mariadb role, it's deploying a Galera cluster on a given group of nodes (group in Ansible inventory), right?12:12
openstackgerritMerged openstack/kayobe-config-dev stable/victoria: Update .gitreview for stable/victoria  https://review.opendev.org/76322512:15
openstackgerritMerged openstack/kayobe-config-dev stable/victoria: Update TOX_CONSTRAINTS_FILE for stable/victoria  https://review.opendev.org/76322612:15
*** ricolin has joined #openstack-kolla12:18
mnasiadkakevko: looking at https://mariadb.com/kb/en/mariadb-maxscale-20-maxscale-connection-routing-with-galera-cluster/ - it doesn't seem like you need to do anything with the mariadb role12:20
*** kevinz has joined #openstack-kolla12:36
openstackgerritMark Goddard proposed openstack/kolla master: DNM: testing mirror revert  https://review.opendev.org/76335112:36
*** zzzeek has quit IRC12:45
*** zzzeek has joined #openstack-kolla12:47
openstackgerritVictor Chembaev proposed openstack/kolla-ansible master: RabbitMQ handler refactored to restart services in serial  https://review.opendev.org/76313712:49
kevkomnasiadka: well, imagine then i want to configure neutron database to run on server1,server2 galera cluster, nova on server3,server4 galera cluster right ?12:54
kevkobut i want to have VIP:3306 to see both clusters12:55
kevko*both databases12:55
kevkoi can run:12:57
*** hrw has quit IRC12:57
kevkokolla-ansible -i inventory deploy -t mariadb --limit first_group12:57
kevkokolla-ansible -i inventory deploy -t mariadb --limit second_group12:57
kevkoand then kolla-ansible -i inventory deploy12:57
kevkopoint is ..that i have to do it in separate bunch of playbooks to "prepare" environment ..12:58
*** hrw has joined #openstack-kolla12:59
*** cah_link1 has joined #openstack-kolla13:03
*** cah_link has quit IRC13:05
*** cah_link1 is now known as cah_link13:05
*** jbadiapa has joined #openstack-kolla13:09
*** rpittau is now known as rpittau|brb13:16
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible stable/victoria: Fix kolla-ansible to work with pyenv-virtualenv  https://review.opendev.org/76336613:34
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible stable/ussuri: Fix kolla-ansible to work with pyenv-virtualenv  https://review.opendev.org/76336713:34
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible stable/train: Fix kolla-ansible to work with pyenv-virtualenv  https://review.opendev.org/76336813:34
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible stable/stein: Fix kolla-ansible to work with pyenv-virtualenv  https://review.opendev.org/76336913:34
yoctozeptokevko: one thing to notice - you probably still want 3 servers per shard - to avoid a situation where you lose quorum13:39
yoctozeptokevko: I would say let's do it in small steps13:40
yoctozeptokevko: sharding can be added later13:40
kevkoyoctozepto: ok, doesn't matter if it will be 2, 3, or 4 servers ..13:40
kevkoyoctozepto: but do you understand what I mean ?13:40
yoctozeptokevko: kind of, you want to divide them into shards13:40
kevkoyoctozepto: well, yes, maxscale is very simple to write ..13:40
yoctozeptokevko: what I'm saying is maxscale without shards would be a great boon already13:40
yoctozeptokevko: and then we can think about how to approach sharding13:41
kevkoyoctozepto: yeah, that's true ..13:41
yoctozeptokevko: we would likely want some logic to that13:41
yoctozeptokevko: I've never run sharding in sql databases13:41
yoctozeptoonly ever mongodb afair :D13:41
yoctozeptoso would need to read up on that13:41
yoctozeptono idea about others13:41
kevkoyoctozepto: especially group_by is hell in mariadb :D13:42
yoctozeptokevko: :D13:42
kevkoi mean that group_by in lookup_cluster.yml13:43
kevkothat is reaaaally depresing13:43
yoctozeptokevko: well, it's worky13:44
yoctozeptoand better than what it was :-)13:44
yoctozeptoalways open to improvements13:45
kevkoyoctozepto: agree, it's mostly ansible limitations ..i know13:45
openstackgerritMerged openstack/kolla master: CI: Improve publish playbook  https://review.opendev.org/76314613:46
mnasiadkakevko: simple baby steps, we don't support multiple galera clusters today, so it's totally different feature :)13:46
kevkook, understand .. baby steps  :)13:47
mgoddardkevko: this was started for nova cells: https://review.opendev.org/61975613:49
patchbotpatch 619756 - kolla-ansible - Minimal support for deploying multiple instances o... - 19 patch sets13:49
kevkomgoddard: haha, it looks like author was doing steps similar as me :)13:51
mgoddardkevko: but could you achieve what you want with a single galera cluster, but different active nodes for different DBs?13:52
mgoddardI haven't used maxscale13:52
kevkomgoddard: well, simply said ..on one DB galera cluster you have different databases .. right ?13:54
mgoddardkevko: yes13:54
kevkomgoddard: for example glance is using database very few ... on other side ..there is neutron DB which has on big Openstack clusters very lot connections ...and performance is going down13:56
mgoddardkevko: yes, I understand13:56
kevkoon customer side we have 100+ compute nodes ..13:56
kevkoand we measured that neutron has problem with DB sometimes ..13:56
kevkoso, we build another cluster on different set of barememtals13:56
kevkoand placed neutron database to it ..13:57
mnasiadkaML2/OVS?13:57
mgoddardkevko: as you said earlier, the issue is that haproxy uses one active backend for all DBs. Could different DBs use different active backends?13:57
openstackgerritMark Goddard proposed openstack/kayobe master: Revert "Use OpenStack Victoria release"  https://review.opendev.org/76337413:58
kevkoit's not issue , same functionality can fullfill maxscale .. but  mascale can also do sharding ...13:58
yoctozeptokevko: that's why we would want to add maxscale as the first step13:58
kevkoto be honest ..openstack don't like active-active cluster as it is in kolla ...because second replica isn't always fresh13:59
kevkoso, active-backup has to be as it is now ..13:59
kevkoyes, you are right ...firstly i should provide simple maxcale switch ..and in second step maybe sharding :)14:00
openstackgerritMerged openstack/kayobe stable/stein: Fix filtering of network names set to an empty string  https://review.opendev.org/76133214:01
openstackgerritMark Goddard proposed openstack/kayobe master: Update TOX_CONSTRAINTS_FILE  https://review.opendev.org/76332914:02
yoctozeptokevko: currently it is active-active behind the scenes (haproxy)14:02
yoctozeptobecause it's multimaster14:02
kevkoyes, i really know how it is in kolla ..14:02
kevkobut did you try to turn on active-active in haproxy ? :)14:03
kevkoi mean, loadbalance transactions to mysql backends ?14:03
kevkothere is always small delay between replicas ..14:04
openstackgerritMark Goddard proposed openstack/kayobe master: CI: set previous_release to Victoria  https://review.opendev.org/76337514:04
*** Arador has joined #openstack-kolla14:11
*** TrevorV has joined #openstack-kolla14:16
openstackgerritBartosz Bezak proposed openstack/kayobe master: Support for firewalld ZONE option in network interfaces configuration  https://review.opendev.org/76318014:20
mgoddardkevko, yoctozepto: I think you can specify different URLs for read vs. write. read being load balanced, write being active/backup14:20
kevkomgoddard: and that's problem ...because sometimes it is not synced ..and you want to read .. openstack service fail14:22
kevkolike, there is only small latency between ..but OS services are very sensitive on that14:22
kevkoin that setup you can be sure that data is only on first node in every time ..14:23
kevkothat's the reason why i'm more familiar with ndb-cluster14:23
openstackgerritMark Goddard proposed openstack/kolla master: DNM: testing mirror revert  https://review.opendev.org/76335114:35
*** happyhemant has joined #openstack-kolla14:44
*** mgoddard has quit IRC14:48
*** rpittau|brb is now known as rpittau14:51
openstackgerritMerged openstack/kayobe stable/victoria: Update .gitreview for stable/victoria  https://review.opendev.org/76321915:02
openstackgerritMerged openstack/kayobe stable/victoria: Update TOX_CONSTRAINTS_FILE for stable/victoria  https://review.opendev.org/76322015:02
*** mgoddard has joined #openstack-kolla15:07
*** k_mouza has quit IRC15:15
*** bsanjeewa has joined #openstack-kolla15:20
*** k_mouza has joined #openstack-kolla15:28
openstackgerritMark Goddard proposed openstack/kolla master: WIP: CI: revert to public package mirrors after build  https://review.opendev.org/76192815:37
openstackgerritMark Goddard proposed openstack/kolla master: DNM: testing mirror revert  https://review.opendev.org/76335115:37
*** vishalmanchanda has quit IRC16:00
*** diurnalist has joined #openstack-kolla16:03
*** diurnalist has quit IRC16:04
*** diurnalist has joined #openstack-kolla16:04
*** evrardjp has quit IRC16:17
AradorI am getting stuck on "wait for MariaDB to be available via HAProxy" and I am unable to find a working solution online, does anyone have a suggestion?16:24
Aradormultinode on VM's, installing ussuri on centos 8.216:26
*** nikparasyr has left #openstack-kolla16:32
*** rpittau is now known as rpittau|afk16:52
*** sluna has quit IRC16:53
*** bengates has quit IRC16:57
*** bengates has joined #openstack-kolla16:58
*** bengates has quit IRC17:02
*** happyhemant has quit IRC17:03
openstackgerritVictor Chembaev proposed openstack/kolla-ansible master: RabbitMQ handler refactored to restart services in serial  https://review.opendev.org/76313717:12
*** bsanjeewa has quit IRC17:30
*** JamesBenson has quit IRC17:33
*** diurnalist has quit IRC17:46
*** gfidente is now known as gfidente|afk17:49
*** cah_link has quit IRC17:49
*** mgoddard has quit IRC18:05
*** k_mouza has quit IRC18:34
*** dardelean has quit IRC18:50
*** dciabrin has quit IRC19:05
*** mgoddard has joined #openstack-kolla19:27
*** mgoddard has quit IRC19:39
*** dciabrin has joined #openstack-kolla20:02
*** ysirndjuro has quit IRC20:10
*** bsanjeewa has joined #openstack-kolla20:16
*** bsanjeewa has quit IRC20:21
*** luksky has quit IRC20:26
*** luksky has joined #openstack-kolla20:26
*** ysirndjuro has joined #openstack-kolla20:32
*** k_mouza has joined #openstack-kolla20:35
*** k_mouza has quit IRC20:39
*** jbadiapa has quit IRC20:52
*** TrevorV has quit IRC21:09
*** dciabrin has quit IRC21:11
*** dciabrin has joined #openstack-kolla21:11
*** stingrayza has quit IRC21:11
*** stingrayza has joined #openstack-kolla21:13
*** e0ne has quit IRC21:24
*** luksky has quit IRC21:41
*** luksky has joined #openstack-kolla21:54
*** kevko has quit IRC22:37
*** Arador has quit IRC23:12
*** strobert has quit IRC23:15
*** strobert has joined #openstack-kolla23:16
*** kevko has joined #openstack-kolla23:27

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!