Tuesday, 2019-05-28

*** goldyfruit has quit IRC00:03
*** goldyfruit has joined #openstack-kolla00:07
*** kplant has quit IRC00:58
*** whoami-rajat has joined #openstack-kolla01:06
*** hongbin has joined #openstack-kolla01:10
*** guozijn has joined #openstack-kolla01:44
*** gregoryo has joined #openstack-kolla01:58
*** gmann has quit IRC02:31
openstackgerritChason Chan proposed openstack/kolla-ansible master: Fix the incorrect "ref" tag  https://review.opendev.org/66096502:35
*** goldyfruit has quit IRC02:56
*** noVNC has joined #openstack-kolla03:20
*** shyamb has joined #openstack-kolla03:33
*** noVNC has quit IRC03:54
*** pcaruana has joined #openstack-kolla04:22
*** ivve has quit IRC04:32
*** Sravan has joined #openstack-kolla04:43
*** Sravan has quit IRC04:48
*** shyamb has quit IRC04:48
*** hongbin has quit IRC04:49
*** janki has joined #openstack-kolla04:51
*** shyamb has joined #openstack-kolla05:08
*** jchhatbar has joined #openstack-kolla05:27
*** janki has quit IRC05:30
*** ivve has joined #openstack-kolla05:38
*** Sravan has joined #openstack-kolla05:47
*** Sravan has quit IRC05:52
*** dteselkin has quit IRC06:01
*** shyamb has quit IRC06:02
*** guozijn has quit IRC06:03
*** guozijn has joined #openstack-kolla06:04
*** shyamb has joined #openstack-kolla06:05
*** cah_link has joined #openstack-kolla06:14
*** shyamb has quit IRC06:29
*** shyamb has joined #openstack-kolla06:31
*** dpawlik has joined #openstack-kolla06:32
*** Sravan has joined #openstack-kolla06:51
*** mrunge has quit IRC06:52
*** mrunge has joined #openstack-kolla06:54
*** Sravan has quit IRC06:56
*** gregoryo has quit IRC06:56
mnasiadkamorning07:04
*** shyam89 has joined #openstack-kolla07:05
*** shyamb has quit IRC07:07
openstackgerritMichal Nasiadka proposed openstack/kolla master: Ceph: Update to Nautilus on CentOS and OracleLinux  https://review.opendev.org/65798907:08
*** shyam89 has quit IRC07:24
*** shyamb has joined #openstack-kolla07:24
openstackgerritMichal Nasiadka proposed openstack/kolla stable/stein: Remove nova-placement-api  https://review.opendev.org/66168607:27
*** happyhemant has joined #openstack-kolla07:31
*** logan- has quit IRC07:33
*** Sravan has joined #openstack-kolla07:34
*** logan- has joined #openstack-kolla07:36
openstackgerritRadosÅ‚aw Piliszek proposed openstack/kolla-ansible master: Fix deploy guide build  https://review.opendev.org/66169107:37
*** Sravan has quit IRC07:39
yoctozeptoI wonder why Zuul check/gate in review did not catch this issue early?07:39
*** yoctozepto_ has joined #openstack-kolla07:42
*** yoctozepto has quit IRC07:43
*** shyam89 has joined #openstack-kolla07:43
*** yoctozepto_ is now known as yoctozepto07:43
*** shyamb has quit IRC07:43
yoctozeptook, I see deploy guide is not checked when docs are updated07:52
yoctozepto(docs outside deploy guide)07:53
yoctozeptobecause it does not track includes07:53
yoctozeptoand is not aware of them07:53
yoctozeptoOpenStack-Ansible (OSA) reference deployment guide from main docs but share no code between them07:55
yoctozeptoexcept for doc requirements (so they can still be affected but rarelier)07:55
*** shyam89 has quit IRC08:04
*** dougsz has joined #openstack-kolla08:09
*** k_mouza has joined #openstack-kolla08:11
*** gmann has joined #openstack-kolla08:13
*** yan0s has joined #openstack-kolla08:20
openstackgerritRadosÅ‚aw Piliszek proposed openstack/kolla-ansible master: Fix deploy guide build  https://review.opendev.org/66169108:22
*** dougsz has quit IRC08:23
*** dougsz has joined #openstack-kolla08:23
openstackgerritMerged openstack/kolla master: create missing apache2 directory on Debian/Ubuntu  https://review.opendev.org/66098108:27
*** Sravan has joined #openstack-kolla08:38
*** mgoddard has joined #openstack-kolla08:41
*** Sravan has quit IRC08:43
openstackgerritMerged openstack/kolla-ansible master: Pin ARA to <1.0.0 and use stable setup interface to find callback plugin  https://review.opendev.org/66131908:47
*** gfidente has joined #openstack-kolla08:52
*** k_mouza has quit IRC08:53
*** k_mouza has joined #openstack-kolla08:53
mgoddardhi mnasiadka08:55
mnasiadkaoh, hi mgoddard08:56
mgoddardthe openstack-[nova-]placement-api package issue is because we're still using delorean packages on stable/stein - when we switch to RDO Stein release that package will be available08:56
mnasiadkawe are still using trunk?08:56
mnasiadkahuh08:56
mgoddardI think we ought to keep the old nova placement code in that image08:57
*** priteau has joined #openstack-kolla08:58
mgoddardwe can't switch to RDO stein until ceph nautilus is working08:58
mnasiadkatrue08:58
mnasiadkathe nautilus code should work, waiting for the CI08:58
mnasiadkaseems needs a recheck - failed on: Could not fetch/save url https://download.docker.com/linux/centos/docker-ce.repo to file /etc/yum.repos.d/docker-ce.repo: [Errno 12] Timeout on https://download.docker.com/linux/centos/docker-ce.repo: (28, 'timed out before SSL handshake')09:02
mgoddardivve: replied09:03
*** shyamb has joined #openstack-kolla09:05
ivvemgoddard: woot?09:08
mgoddardivve: almost09:08
ivveah yes, nvm :)09:09
*** yoctozepto has quit IRC09:09
*** yoctozepto has joined #openstack-kolla09:10
openstackgerritMarek Svensson proposed openstack/kolla-ansible master: Add missing timezone mount for watcher (apscheduler).  https://review.opendev.org/66106809:12
ivvemgoddard: done09:12
*** jchhatba_ has joined #openstack-kolla09:18
openstackgerritZijian Guo proposed openstack/kolla-ansible master: Update service_provider for FWaaS v2  https://review.opendev.org/66170409:20
*** jchhatbar has quit IRC09:21
openstackgerritMark Goddard proposed openstack/kolla master: Remove nova-consoleauth  https://review.opendev.org/66125109:22
openstackgerritMark Goddard proposed openstack/kolla master: Remove nova-consoleauth  https://review.opendev.org/66125109:24
*** shyamb has quit IRC09:26
openstackgerritZijian Guo proposed openstack/kolla-ansible master: Update service_provider for FWaaS v2  https://review.opendev.org/66170409:27
*** k_mouza_ has joined #openstack-kolla09:33
*** shyamb has joined #openstack-kolla09:34
*** k_mouza has quit IRC09:37
hrwelo09:38
*** Sravan has joined #openstack-kolla09:39
mgoddardhi hrw09:39
yoctozeptomgoddard: did you have a chance to read my monologue on docs checking? http://paste.openstack.org/show/752146/ any idea on how to proceed? I am afraid the publish process might not be run as well, leaving the deploy guide always behind09:43
*** Sravan has quit IRC09:43
*** hamzaachi has joined #openstack-kolla09:44
mgoddardyoctozepto: we should add a 'files' section to the deploy-guide job config to reference relevant files09:45
mgoddardyoctozepto: or we could just change the link to point to the kolla-ansible docs, and remove the deploy-guide09:45
yoctozeptomgoddard: I somehow feel the latter approach will prove to be less surprising and usually cleaner; not to mention one less check to be done :-)09:48
yoctozeptotripleo link to their docs09:49
mgoddardI don't know the history of why these are separate09:49
yoctozeptoOSA does as I wrote before, however I do not really like their approach09:49
hrwhm. looks like there is a chance for +1 from zuul for py3 patch09:49
hrwbut with failures ;(09:49
yoctozeptoas it splits the docs unnecessarily imho09:50
mgoddardagreed09:50
hrwanyone with Zun experience?09:51
*** shyamb has quit IRC09:52
hrw+ exec apache2 -DFOREGROUND09:52
hrwapache2: Syntax error on line 80 of /etc/apache2/apache2.conf: DefaultRuntimeDir must be a valid directory, absolute or relative to ServerRoot09:52
*** shyam89 has joined #openstack-kolla09:52
hrwok.09:52
hrwhttps://review.opendev.org/660981 (just merged) will get reverted and directory gets added in base image09:53
egonzalezhrw usually i've found you have some conflict with default zun wsgi file, check if the package have its own vhost or something conflicting (ports, process name, etc)10:03
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: create missing apache2 directory on Debian/Ubuntu  https://review.opendev.org/66171310:03
hrwegonzalez: I have a feeling that this patch(set) conflicts with half of openstack nowadays10:04
hrwif it would be something smaller (and not so needed) I would probably abandon it and moved to something else10:05
egonzalezohh, you have the issue with source images, good luck when going to binary ;)10:05
hrwegonzalez: I do not care much about binary ones.10:06
*** dpawlik has quit IRC10:06
hrwI have a feeling that Debian openstack team is usually release delayed, UCA does not care about anyone outside of plain Ubuntu etc10:08
hrwand then OracleLinux decides to lag so much that CI timeouts or turns out that we lack OL maintainer and images stop working because some package needs bump (again)10:09
mnasiadkayeah, we lack OL maintainer I guess - but that's one of the topics for PTG10:09
egonzalezat what time is PTG today?10:11
egonzalezI always screws up with timezones10:11
hrwin 1:4810:12
hrwbtw - https://review.opendev.org/#/c/642375/23/kolla/common/config.py was supposed to be workaround. I am no longer treating it as such.10:13
hrwopenstack/x projects are not maintained mess10:13
openstackgerritChason Chan proposed openstack/kolla-ansible master: Fix the incorrect "ref" directive  https://review.opendev.org/66096510:14
*** dpawlik has joined #openstack-kolla10:18
hrwscrew being polite. sent passive-agressive mail to openstack-discuss.10:19
*** dpawlik has quit IRC10:22
mgoddardegonzalez: it's 12:00 UTC10:28
mnasiadkamgoddard: he just told us he always screws up with timezones :D10:29
mnasiadkaegonzalez: 14:00 CET :)10:30
mgoddardmnasiadka: google says 13:00: https://www.google.com/search?q=12%3A00+utc+in+cet&oq=12%3A00+utc+in+cet&aqs=chrome..69i57.4829j1j7&sourceid=chrome&ie=UTF-810:31
mgoddardCEST is 14:0010:31
mgoddardin 1 hour 30 minutes from now :)10:32
hrwhave to go and look for BT speaker10:33
hrwmy desktop lost sound with mainboard replacement10:33
*** dpawlik has joined #openstack-kolla10:35
hrwlike I thought: https://review.opendev.org/#/c/642375/23 got +1 from zuul. now time to push update and hope that only one CI job will fail10:39
*** dpawlik has quit IRC10:40
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Debian/Ubuntu: move to Python 3 for binary/source images  https://review.opendev.org/64237510:41
hrwhttp://zuul.openstack.org/status/change/642375,24 is one of my pinned cards ;D10:41
hrwjust patch revision gets changed10:41
*** guozijn has quit IRC10:43
*** shyam89 has quit IRC10:47
mgoddardmichaelbarkdoll, ivve, kplant: great to see the love for kolla :D we have our virtual PTG today and tomorrow if you'd like to get involved: https://etherpad.openstack.org/p/kolla-train-ptg10:48
*** k_mouza_ has quit IRC10:51
mnasiadkamgoddard: so it's in 8 minutes, or 1hr and 8 minutes? :D10:52
mnasiadkaah, CEST not CET, whatever10:52
mgoddardmnasiadka: 1h8!10:52
hrw;D10:52
mnasiadkathey should remove this summertime crap10:52
mgoddardthey were considering it IIRC10:52
hrwmnasiadka: in 2002010:53
hrwextra 0 intended10:53
mnasiadkayeah, I will be long dead, and my daughter also ;)10:53
mgoddardof course we won't be in the EU then so maybe we will use our 'sovereignty' to keep BST10:54
mnasiadkawho knows where you will be10:54
*** k_mouza has joined #openstack-kolla10:56
hrwmgoddard: then it will be England Summer Time + Scottish Summer Time + Welsh Summer Time :D10:57
mgoddardwhere does it end? every person having an individual country of their own?10:57
mgoddardactually that sounds quite nice10:57
mgoddardcould have my own currency10:58
mnasiadkatotally unusable in any other country10:58
mnasiadka:D10:58
*** k_mouza has quit IRC10:58
*** k_mouza has joined #openstack-kolla10:59
*** dpawlik has joined #openstack-kolla11:02
*** johnthetubaguy has joined #openstack-kolla11:04
*** k_mouza has quit IRC11:09
mnasiadkamgoddard: https://review.opendev.org/#/c/657989/ - I think we're good11:13
*** guozijn has joined #openstack-kolla11:16
*** shyamb has joined #openstack-kolla11:19
*** k_mouza has joined #openstack-kolla11:21
*** kplant has joined #openstack-kolla11:24
priteauI've noticed that kolla-ansible uses [glance]/api_servers in nova.conf. The configuration reference says: NOTE: The preferred mechanism for endpoint discovery is via keystoneauth1 loading options. Only use api_servers if you need multiple endpoints and are unable to use a load balancer for some reason.11:29
priteauhttps://docs.openstack.org/nova/latest/configuration/config.html#glance11:29
priteauIs this something that we should change?11:29
mnasiadkapriteau: I guess so - create a PR ;)11:34
*** pomac has joined #openstack-kolla11:43
mgoddardKolla virtual PTG in 15 minutes11:46
mgoddardTengu: still on for validations?11:48
mgoddardhttps://etherpad.openstack.org/p/kolla-train-ptg11:48
Tengumgoddard: oh. sh.... that's today11:48
TenguRIGHT11:48
mgoddardTengu: yeah :)11:49
Tengumgoddard: jumping on the meet. let's see how it goes.11:49
Tengumgoddard: just back from my wedding, have some issues with my brain ^^'.11:49
mgoddardTengu: we're starting in 10. You're up once we've done an intro11:50
mgoddardTengu: oh wow, congrats!11:50
Tengumgoddard: care to let me in? apparently some moderator needs to accept me11:52
Tenguwould like to ensure all is working fine :)11:52
mgoddardTengu: ok11:52
*** k_mouza has quit IRC11:53
Tenguhmmm...11:53
hrwinstalling meet app11:55
*** factor has joined #openstack-kolla11:57
Tenguusing a custom firefox profile here. kind of weird, webcam isn't well recognized. Sorry for being just a voice ;)11:57
jrollI'm gonna have to be a little bit late for the PTG :/11:57
mgoddardno worries jroll12:00
*** Wasaac has joined #openstack-kolla12:00
Tenguso apparently my webcam does work in the end. The authorization request was hidden somewhere. yay UI XD12:00
mgoddardhttps://cjeanner.github.io/openstack/tripleo/validations/2019/04/24/validation-framework.html12:02
mgoddardhttps://etherpad.openstack.org/p/kolla-train-ptg12:04
*** k_mouza has joined #openstack-kolla12:11
*** guozijn_ has joined #openstack-kolla12:12
openstackgerritKrzysztof Klimonda proposed openstack/kolla-ansible master: Elevate privileges for copying gnocchi policy.json  https://review.opendev.org/66174312:13
*** priteau has quit IRC12:13
*** priteau has joined #openstack-kolla12:14
*** guozijn has quit IRC12:15
*** guozijn_ is now known as guozijn12:15
*** shyamb has quit IRC12:19
*** k_mouza has quit IRC12:21
*** shyamb has joined #openstack-kolla12:23
openstackgerritKrzysztof Klimonda proposed openstack/kolla-ansible master: Make fluentd-elasticsearch configuration more robust  https://review.opendev.org/66174712:26
openstackgerritKrzysztof Klimonda proposed openstack/kolla-ansible master: Elevate privileges for copying gnocchi policy.json  https://review.opendev.org/66174312:31
*** chason has joined #openstack-kolla12:31
*** shyamb has quit IRC12:34
kklimondaHmmm, ceilometer seems to depend on requests-aws for collecting metrics from ceph, and that library is not available from ubuntu packages, only pip - how to support binary installation in such case? Are there examples from other images?12:36
*** k_mouza has joined #openstack-kolla12:39
*** k_mouza has quit IRC12:43
*** dougsz has quit IRC12:48
*** henriqueof has quit IRC12:56
*** igordc has joined #openstack-kolla12:59
openstackgerritMerged openstack/kolla stable/rocky: Bump service versions Rocky  https://review.opendev.org/66016313:00
openstackgerritMerged openstack/kolla stable/queens: Bump service versions for Queens  https://review.opendev.org/66016013:00
*** dougsz has joined #openstack-kolla13:03
*** henriqueof has joined #openstack-kolla13:04
*** jchhatba_ has quit IRC13:08
*** jchhatba_ has joined #openstack-kolla13:08
*** goldyfruit has joined #openstack-kolla13:11
*** guozijn_ has joined #openstack-kolla13:16
*** guozijn has quit IRC13:19
*** guozijn_ is now known as guozijn13:19
*** priteau has quit IRC13:20
hrwmnasiadka: https://review.opendev.org/#/c/523072/13:21
*** goldyfruit_ has joined #openstack-kolla13:25
*** k_mouza has joined #openstack-kolla13:26
*** goldyfruit has quit IRC13:27
*** goldyfruit__ has joined #openstack-kolla13:35
*** guozijn_ has joined #openstack-kolla13:35
*** goldyfruit_ has quit IRC13:37
*** guozijn has quit IRC13:37
*** guozijn_ is now known as guozijn13:37
*** guozijn_ has joined #openstack-kolla13:40
*** guozijn has quit IRC13:42
*** guozijn_ is now known as guozijn13:42
*** priteau has joined #openstack-kolla13:46
*** k_mouza has quit IRC13:48
*** henriqueof has quit IRC13:49
*** k_mouza has joined #openstack-kolla13:55
*** jchhatba_ has quit IRC13:55
*** k_mouza has quit IRC13:55
*** k_mouza has joined #openstack-kolla14:04
*** k_mouza has quit IRC14:09
*** hamzaachi_ has joined #openstack-kolla14:10
*** k_mouza has joined #openstack-kolla14:14
*** hamzaachi has quit IRC14:14
*** itlinux has quit IRC14:19
*** hamzaachi__ has joined #openstack-kolla14:20
*** henriqueof has joined #openstack-kolla14:20
*** k_mouza has quit IRC14:20
*** k_mouza has joined #openstack-kolla14:21
*** shyamb has joined #openstack-kolla14:22
*** hamzaachi_ has quit IRC14:22
*** guozijn has quit IRC14:24
*** hamzaachi__ has quit IRC14:24
*** hamzaachi has joined #openstack-kolla14:24
*** guozijn has joined #openstack-kolla14:24
*** hamzaachi has quit IRC14:26
*** hamzaachi has joined #openstack-kolla14:26
*** hamzaachi has quit IRC14:28
*** dpawlik has quit IRC14:29
*** hamzaachi has joined #openstack-kolla14:30
v1k0d3nhey folks. have a question re: octavia setup in kolla. i've been playing around with it over the last couple of days, and i have an admin network (`192.168.3.0/24`) and and a "public" network (`192.168.4.0/22`). both of these are external networks, and i have them set up this way in neutron (`neutron_external_interface`).14:31
*** itlinux has joined #openstack-kolla14:33
v1k0d3nonce i have run through the install, i noticed that networking doesn't seem to work in the same broadcast domain; so it seems like i can't ping the gateway for the admin network `192.168.3.1`. i know this is how it's intended, but doesn't the octavia LB need to be able to get out via this "admin" network (if i understand correctly)? how are others doing this setup in Kolla?14:33
*** shyamb has quit IRC14:33
ivvev1k0d3n: are you talking about the octavia management network?14:34
*** shyamb has joined #openstack-kolla14:34
v1k0d3nyes, that's it.14:34
v1k0d3nadmin == octavia management in this case.14:35
*** hamzaachi_ has joined #openstack-kolla14:35
v1k0d3ni have this configured in the globals as `octavia_network_interface: "eno1"` while my external network is actually `eno1`. i'm sure i've misunderstood and am likely making an error somewhere.14:36
*** dasp has quit IRC14:36
ivvethe network name must be lb-mgmt-net, is that so in your case?14:36
ivveis it owned by octavia tenant/user?14:36
v1k0d3noh! no...i didn't name it that. whoops.14:36
v1k0d3nit will be, yes...but i've named it incorrectly as you pointed out.14:37
*** hamzaachi has quit IRC14:37
v1k0d3nso basically all things like the key, the lb-mgmt-net...these things have to be done via the octavia user in the service project (if we just consider the default install of kolla); am i understanding this correctly?14:38
*** k_mouza has quit IRC14:38
ivveoctavia tenant, if that is the chosen one14:39
ivveand octavia user14:39
ivvekeys, sec groups, networks14:39
v1k0d3nit doesn't appear that kolla installs an octavia tenant by default (even when octavia is configured).14:40
ivvehmm14:40
*** k_mouza has joined #openstack-kolla14:40
v1k0d3nso that would all need to be setup afterwards, i guess.14:40
ivveah no, its the admin tenant but octavia user14:40
ivvei believe14:40
v1k0d3nah. ok...that makes sense.14:40
ivvewhen you install you must follow the guide very thoroughly :)14:42
*** hamzaachi_ has quit IRC14:42
*** hamzaachi_ has joined #openstack-kolla14:42
v1k0d3nahh...which guide? i looked through kolla and found...nothing.14:42
ivveoctavia i think14:43
v1k0d3nbut openstack docs are confusing to me sometimes.14:43
ivvethey are for all of us :)14:43
v1k0d3nthat's good to hear at least.14:43
kplanti feel like most of them are written after a 16 hour coding session14:43
kplantlike a "...ugh now i have to document this crap"14:43
ivveprobably :)14:44
*** guozijn has quit IRC14:44
v1k0d3nso true.14:45
v1k0d3ni know i've felt like that before.14:46
v1k0d3nbut the way openstack docs are presented are sort of hard to follow.14:46
*** hamzaachi_ has quit IRC14:46
*** hamzaachi_ has joined #openstack-kolla14:46
v1k0d3nhard to search/find things14:46
kplantyes and the dir structure always changes14:46
kplantso changing 'rocky' to 'stein' usually doesn't work14:47
v1k0d3nthat. or you click on the link and it takes you back to the main docs page. hahaha14:47
v1k0d3nchallenging for new folks.14:47
*** k_mouza has quit IRC14:50
*** hamzaachi_ has quit IRC14:53
yoctozeptothough it would help to report doc bugs/suggestions/questions rather than only grumble about their quality ;-) it does not make them any better this way :P14:55
*** shyam89 has joined #openstack-kolla14:57
yoctozeptoanyways, to join in grumbling: for most fast-moving software the quality of docs leaves a lot to desire... ;-)14:57
*** shyamb has quit IRC15:00
*** ivve has quit IRC15:03
*** hamzaachi has joined #openstack-kolla15:10
*** k_mouza has joined #openstack-kolla15:11
*** yan0s has quit IRC15:11
*** shyam89 has quit IRC15:14
v1k0d3nyoctozepto: i don't think anyone was really knocking the people writing the docs, as much as the tool itself for doc presentation/search.15:16
*** shyamb has joined #openstack-kolla15:17
v1k0d3nbut your suggestion is noted. grumbling over.15:18
*** hamzy has quit IRC15:18
*** k_mouza has quit IRC15:22
*** k_mouza has joined #openstack-kolla15:24
*** guozijn has joined #openstack-kolla15:27
*** itlinux has quit IRC15:35
*** guozijn has quit IRC15:40
*** spiette_ has joined #openstack-kolla15:41
*** spiette has quit IRC15:41
*** shyam89 has joined #openstack-kolla15:43
*** shyamb has quit IRC15:44
*** JamesBenson has joined #openstack-kolla15:44
mgoddardhrw: for the apache config issue, should we just remove the default config file that references /var/run/apache?15:48
*** hamzy has joined #openstack-kolla15:49
*** k_mouza has quit IRC15:52
*** hamzaachi_ has joined #openstack-kolla15:53
*** Wasaac has quit IRC15:54
*** hamzaachi has quit IRC15:55
*** shyam89 has quit IRC15:58
*** shyamb has joined #openstack-kolla16:03
*** k_mouza has joined #openstack-kolla16:15
*** ivve has joined #openstack-kolla16:22
*** goldyfruit__ has quit IRC16:22
ivveanyone here using freezer?16:23
*** shyamb has quit IRC16:26
*** shyamb has joined #openstack-kolla16:27
ivveanyways, seems like when updating jobs doesn't work properly, im guessing the json is illegal somewhere16:27
ivve(via horizon)16:27
*** hamzaachi_ has quit IRC16:29
*** itlinux has joined #openstack-kolla16:35
*** shyam89 has joined #openstack-kolla16:36
*** shyamb has quit IRC16:37
*** shyam89 has quit IRC16:41
*** itlinux has quit IRC16:43
*** itlinux has joined #openstack-kolla16:46
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Test Ceph upgrade in CI  https://review.opendev.org/65813216:47
*** dpawlik has joined #openstack-kolla16:48
*** jonaspaulo has joined #openstack-kolla16:49
*** k_mouza has quit IRC16:51
*** itlinux has quit IRC16:58
*** dougsz has quit IRC17:00
*** itlinux has joined #openstack-kolla17:01
*** eandersson has joined #openstack-kolla17:11
*** goldyfruit has joined #openstack-kolla17:14
*** irclogbot_3 has quit IRC17:17
*** irclogbot_3 has joined #openstack-kolla17:18
v1k0d3nhey ivve ... i'm super sorry to bother ... but your octavia setup on the network side; how do you have this configured in globals.yml?17:24
v1k0d3nso in my case, i have essentially three networks. api (192.168.70.0/24), an admin...which is also where MaaS is (192.168.3.0/24) <<< i want to use this for octavia, and my external (192.168.4.0/22).17:25
v1k0d3nthose are all my actual networks...prior to creating any flat nets.17:25
v1k0d3nthe config that i have currently just doesn't seem to work, because once it's deployed, the 192.168.3.x net goes dark...i can't ping the gateway anymore.17:26
v1k0d3nso that means that my single beefy server (aio) can't talk to that admin net anymore where the Amphora server gets spun up.17:27
ivvei don't configure it in globals at all17:29
ivvei use a vxlan and create interfaces on br-int for my controller nodes17:29
ivveit doesn't have to be an external network17:30
ivvebut okay, lets give it a go17:31
ivveoctavia will use the admin tenant but octavia user to create amphorae17:32
ivveso i need a bit more info.. how is 192.168.3.0/24 configured on the controller and how is it configured in openstack?17:34
*** igordc has quit IRC17:34
ivveyou see when an amphora is created, it needs a port on lb-mgmt-net17:36
ivveand the controller should already be connected on that network17:37
*** igordc has joined #openstack-kolla17:55
*** igordc has quit IRC17:55
*** igordc has joined #openstack-kolla17:55
*** priteau has quit IRC17:56
v1k0d3noh that's right...you did say that actually.17:57
v1k0d3ni can actually create a vxlan. that's what i've done for a couple of years...this is the first time that i've used a MaaS provided network (which presents it's own challenges).17:58
v1k0d3nso 192.168.3.0/24 is directly connected via eno1 (ip 192.168.3.35/24), eno2 is the api interface (192.179.70.20/24 and 25 for vip when using hap/kad), and eno3 is the "public" access, external network (192.168.4.0/2218:00
*** stackedsax has joined #openstack-kolla18:00
*** itlinux has quit IRC18:02
stackedsaxshucks, I guess I missed the google hangout18:05
ivvev1k0d3n: im guessing you will have to create a flat external network 192.168.3.0/24 and choose a smaller range for your amphorae18:06
ivveor well, a range not taken by other stuff18:06
ivveand you will have to define eno1 as the external interface18:07
ivvewhere br-ext is bound18:07
ivvebr-ex*18:07
ivveand configure neutron accordingly18:07
v1k0d3ninteresting. i guess i could create a virtual interface, bridge them and use that?18:08
ivvewell the "issue" is that both the controller and the amphorae needs to be on the same network, and that network needs to exist atleast in the OS world so to speak, but it doesn't have requirements of being external18:09
ivveso that is why i chose vxlan, the infra already exists and its easy to create a port on br-int and assign it some config in openstack and then i just conf it on the controller and let dhclient do its work18:09
v1k0d3nlet me show you my old config...it needed cleaned up a little but i could use this if it works.18:11
ivveshoot18:11
ivvethe path to magnum is long and dangerous :P18:12
*** gdwornicki has joined #openstack-kolla18:15
*** itlinux has joined #openstack-kolla18:17
*** gdwornicki has quit IRC18:22
v1k0d3nsorry got distracted.18:23
v1k0d3none thing that's changed quite a bit is ubuntu's netplan in 18.04, and i haven't quite figured out how to do the same implementation that i had before. :-/18:23
*** Sravan has joined #openstack-kolla18:27
*** gdwornicki has joined #openstack-kolla18:36
openstackgerritMerged openstack/kolla-ansible stable/rocky: Pin ARA to <1.0.0 and use stable setup interface to find callback plugin  https://review.opendev.org/66132618:39
openstackgerritMerged openstack/kolla stable/rocky: Bump Monasca Fluentd plugin version  https://review.opendev.org/66101618:44
*** gdwornicki has quit IRC18:49
*** gdwornicki has joined #openstack-kolla18:54
*** Sravan has quit IRC19:01
ivvemgoddard: regarding https://review.opendev.org/#/c/661485, i have some questions. 1) is the deployer intended to search in group_vars or defaults or globals.yml for the possible options (if we disregard docs). 2) if a default fails a deployment by "default" should it still be there as a blank/null value? would default in the role dir suffice?19:02
openstackgerritMerged openstack/kolla master: Install fluentd gem on debian/ubuntu for non x86_64  https://review.opendev.org/63869219:04
*** Sravan has joined #openstack-kolla19:05
*** Sravan has quit IRC19:06
ivvethe entire reason is that kolla-ansible deployed elasticsearch will never work due to version incompability. 2.1.x - 2.3.0 (roughly) work for freezer19:08
ivveif a deployer would simply switch backend and go along, it would "break". not knowing about or setting the external 2.3.0 ES backend19:09
*** itlinux has quit IRC19:09
ivvei guess if default is undefined, ansible would spit it out19:09
ivveis that acceptable?19:10
*** Sravan has joined #openstack-kolla19:10
*** itlinux has joined #openstack-kolla19:13
*** Sravan has quit IRC19:23
*** Sravan has joined #openstack-kolla19:27
*** pcaruana has quit IRC19:30
*** goldyfruit has quit IRC19:31
*** henriqueof has quit IRC19:32
*** goldyfruit has joined #openstack-kolla19:34
*** kplant has quit IRC19:44
*** Sravan has quit IRC20:02
openstackgerritMerged openstack/kolla stable/rocky: Add CI trigger for ceph jobs  https://review.opendev.org/66095520:02
openstackgerritMichal Nasiadka proposed openstack/kolla master: Ceph: Update to Nautilus on CentOS and OracleLinux  https://review.opendev.org/65798920:03
openstackgerritMichal Nasiadka proposed openstack/kolla master: Ceph: Update to Nautilus on CentOS and OracleLinux  https://review.opendev.org/65798920:07
*** Sravan has joined #openstack-kolla20:12
*** dasp has joined #openstack-kolla20:20
*** hamzy has quit IRC20:21
*** gdwornicki has quit IRC20:30
*** dpawlik has quit IRC20:31
openstackgerritMarek Svensson proposed openstack/kolla-ansible master: Fix default deployment of freezer, use mariadb.  https://review.opendev.org/66148520:37
*** dasp has quit IRC20:38
*** dasp has joined #openstack-kolla20:38
*** gfidente is now known as gfidente|afk20:45
*** cah_link has quit IRC20:59
ChewieBeardyheya everyone! I was looking at this page: https://docs.openstack.org/neutron/stein/admin/config-bgp-dynamic-routing.html and was wondering if such a deployment would be possible within kolla-ansible21:03
ChewieBeardybasically, my idea is to do away with the network nodes, and instead having floating IPs advertised by the hypervisor running the instance directly21:05
ChewieBeardyand something similar for API endpoints and others like dashboards, doing load balancing at the BGP level on the controller nodes21:06
ivveChewieBeardy: dvr does that. i mean if you want direct north-south traffic21:06
ivve(if thats what you are trying to solve)21:06
ChewieBeardyseems like it, but I've read on some pages of the docs that network nodes were still required in some cases21:07
ChewieBeardywith DVR, every compute node runs the L3 agent, and as such a BGP speaker, would that be correct?21:08
ivveyes21:08
*** dave-mccowan has joined #openstack-kolla21:09
ivvebut you won't get rid of network nodes21:09
ChewieBeardyaww, a shame21:10
ivvewell, what exactly do you mean by getting rid of them should be the question21:10
ivveyou can also enable ha to alleviate some SPoF for say, dhcp21:10
ChewieBeardyI guess I meant fully distributing them, instead of having dedicated nodes that did just that (and were necessarily kept in their own subnet for HA)21:11
ivvewell if east west traffic is direct and north south too. and you have atleast 3 networknodes. whats the problem?21:12
ChewieBeardyyeah, maybe I'm just drowning in a cup of water21:12
ChewieBeardymy main "problem" in my head was that since in our current design each rack had its own subnet, and that I wanted to have network nodes on each rack to spread the failure domain, I couldn't use L2 based HA mechanisms like vrrp without resorting to ugliness21:13
ivveah21:14
ChewieBeardybut maybe I'm confusing "internal" HA that doesn't care about that with the haproxy cluster that is running keepalived21:14
ivvethats for the api's21:16
ivvewell, and some other services21:17
ivve:)21:17
ChewieBeardyright, I should separate those two problems in my mind21:17
ivveyes21:17
ivvedvr-ha mainly solves your main concern, i think :)21:17
ChewieBeardyyou're right, I'll study that path in more detail21:18
ChewieBeardyand for the APIs themselves, either keep with the basic haproxy cluster for now, or maybe even its own set of bgp shenanigans to advertise the endpoints21:19
ivveas long as you have 3 controllernodes and 3 networknodes (which could be co-located) failure domain-wise. you're safe21:19
ivvewell haproxy does the job perfectly fine, imo. but you could solve it with a f5 LB or something too21:19
ivveor other hw21:20
ivvef5 just an example21:20
*** itlinux has quit IRC21:20
ChewieBeardyyeah, I was thinking of reutilizing the same idea : each controller advertises the same IP through BGP and the routers above loadbalance through that21:20
ivvebut you shouldn't try to solve any stuff in the OS world with components from it, thats basically sawing off a branch while sitting on it21:21
ChewieBeardybut it migt be a bit overkill for a first tryout, no need to be more royalist than the king like we say in french21:21
ChewieBeardya good point21:22
ivveif anything with an external component, but bgp isn't the answer here. loadbalancing is21:22
ivve(if we're talking haproxy and external/internal api's)21:22
*** dave-mccowan has quit IRC21:23
ChewieBeardyyeah, BGP solely for loadbalancing, or any other protocol. It's a practice I've seen here https://www.xenon.com.au/wp-content/uploads/dlm_uploads/2017/10/BGP_Oreilly-2.pdf that I wanted to give a try to21:23
*** itlinux has joined #openstack-kolla21:23
ivveand really for the north-south or east-west, dvr-ha solves it. either that or ml2 plugin which takes over everything networkwise from neutron21:23
ChewieBeardynot really its original purpose, but it works surprisingly well21:24
ivve(except for the api part)21:24
ChewieBeardysounds good21:24
ChewieBeardyso just for recap: dvr-ha for everything that concerns instances, and haproxy for the api?21:24
ivvethen you have your problem in the hardware instead :)21:24
ivvewhich has other solutions21:25
ivvehaproxy+keepalived21:25
ivveyea21:25
ChewieBeardyawesome21:25
ivve:)21:25
ivvejust be careful with the neutron rabbithole21:25
ChewieBeardyjust a last silly question if I may: by default the haproxy runs on the network nodes, but I suppose this can be changed without too much problem?21:25
ivveits a deep one ;)21:26
ChewieBeardyyeah, feels like I opened pandora's box here21:26
ivveits not called after a quantum particle for nothing21:26
ivvenamed after*21:26
ivvebut check the dvr-ha architecture, its a good start21:26
ChewieBeardythank you so much for all your guidance21:27
ChewieBeardyit's a great help :)21:27
ivvewell haproxy is deployed on controllernodes by kolla-ansible by default if im not mistaken21:27
ivvebut you can turn it off if you have your own solution21:27
ivve(like a physical lb or some other software)21:28
ivvebut you would have to preconfigure everything.. and its a lot21:28
ChewieBeardyindeed21:28
ChewieBeardyat least the haproxy path is more or less "self contained"21:28
ivvewell, haproxy is quite reliable imo21:29
ChewieBeardyotherwise I'd need to open the pandora's box hidden within the pandora's box with a global SDN and... let's not go there :D21:29
ivveyou could always deploy it with helm ;)21:30
ChewieBeardyyeah, haproxy works wonder, only reason I started thinking to remove it was because I wanted to loadbalance them without VRRP but with ECMP, and at that point I thought "might as well ECMP directly the services behind it"21:31
ivvebut then you're back to where you started21:31
ChewieBeardyindeed21:32
ivvei wouldn't bet my life on it, but a lot of products that are "black boxes" use keepalived, haproxy or pacemaker to solve stuff like that21:32
ivvesome of them i know for sure, other a good guess21:33
ChewieBeardyyeah, you're right21:33
ivveand some use quorum based solutions21:33
ChewieBeardyI'm no fan of magic godbox appliances, I'm much more in favor of distributed commodity hardware doing stuff intelligently21:33
ivvethen haproxy and keepalived is your friend, possibly pacemaker too :)21:34
ivveatleast you can configure them yourself :P21:34
ChewieBeardyhence the reason I fell in love with the leaf-spine design, it's such an elegant solution21:34
ivveand vrrp21:34
ChewieBeardybasically each switch on the rack is its own AS, and they all peer with a common mesh of routers above that interconnect them21:35
ChewieBeardyno more L2 craziness, guaranteed 3 hops at worst, no other protocols than BGP and the linux kernel to load balance the routes, what more could I ask for? :D21:35
ivvevxlans for l2 crazyness? :D21:36
ChewieBeardyoh god21:36
ivve:D21:36
ivvehey, it runs well on the bgp leaf spine backbone21:37
ChewieBeardymight need to look into that21:37
ivvewell, you will quickly find that thats how neutron work ;)21:37
ChewieBeardyin my mind, I only associated vxlans with "the magic tunnels that make tenant networks work" and nothing else21:37
ivveand that most deployments out there probably use that architecture21:38
ivvewell you can also used it for external networks if that is a requirement (if segmented networks isn't applicable)21:39
ivvei mean in conjunction with openstack21:39
ChewieBeardyso basically, deploy machines however I want, then spread vxlans on top to simulate the L2 fabrics where I need?21:40
ivvewell, if you wanna have the same L2 in say a few DC's spread over a city21:40
ivveand you have dark fiber21:40
ivvevxlan and bgp is the way to go21:41
ivvewell, in my opinion21:41
ivvei should say21:41
ChewieBeardydefinitely something I need to study more then21:42
ivvesome guys will probably say something about complexity and stretching vlans is a great idea21:42
ChewieBeardyyeah, I guess I'm biased against vlans and other L2 stuff, I usually try to minimize them as much as possible21:42
emccormickI dream of days when I can have a nice ECMP / BGP / VXLAN mesh21:43
ChewieBeardyat least with IP every appliance more of less speaks the same thing21:43
emccormickonce we hire a real network admin I'm all for it :D21:43
ivvehaha21:43
ivve:D21:43
ChewieBeardyhahaha21:43
ChewieBeardyhave a look at the pdf I linked earlier, the design is actually super simple21:44
emccormickI can theoretically accomplish it, but my little clouds don't really need it21:44
ChewieBeardythat could be said of so much stuff we play with :D21:45
emccormickjust a lot of retrofit. I started messing with it in a Cumulus VX setup a while ago and never got it finished21:45
emccormicknext region maybe ;)21:45
ivveyou can always play in a lab with openvswitch21:45
ChewieBeardyin our case we have the luxury of being a school and students helping on it, so it's as much for "real" needs as it is a pedagogical project for fun, we can go way overboard just to study it21:45
ChewieBeardy(let's be honest, we're nowhere near the level of machines where big vlans become a problem)21:46
emccormickThe Cumulus virtual switches work better for me because I run Cumulus IRL21:46
ivvefair enough21:46
ChewieBeardyyeah, if only they had a docker version21:47
emccormickyep agreed21:47
ivvethere is this thing called gns3, never tried it though21:47
ChewieBeardywould fit so well with GNS321:47
ChewieBeardyI use it a lot for teaching, it's awesome21:47
ivveyou can load switch firmware into it21:47
ivvei have stuff downloaded for it, but i also have access to the real things.. so :)21:47
ChewieBeardyI don't use real appliances or vms with it, only docker images like openvswitch and custom linux images for hosts and routers21:48
emccormickon another subject, anyone notice the need to restart fluentd after a day or so of logging at debug level? They just stop sending logs for no good reason I can see21:49
ChewieBeardyit's in french, but I've made a few teaching videos with GNS3 if you want to see what it looks like:  https://www.youtube.com/watch?v=NgodL4Ll6Tc21:49
emccormickI simply use ansible to bulk-restart fluentd and it all goes along nicely again21:49
emccormickChewieBeardy cool, thanks!21:49
ivvenot really, had many of the core services run at debug for nearly 6 weeks in a setup using mostly rocky at the time. filled up quite a few GB's of logs :P21:50
ivverunning ubuntu, source fluentd21:51
emccormickCentos source here21:52
emccormickDon't have the problem in a Queens deploy, but do in a Rocky one. Weird21:52
emccormickRE: Cumulus and Docker, they have their own kernel versions so they can't really run as a container on a regular box21:52
emccormickMaybe they need Kata ;)21:52
ivveim not sure anymore which is more "bugfree" centos or ubuntu21:53
ivveit seems to flip back n forth alot. there should be vars like <service>_tag: "" in kolla for container os :P21:54
ivvethere is for source/binary and container tag, but not for os :)21:56
ivveor i missed it21:56
*** JamesBenson has quit IRC21:57
*** niedbalski has joined #openstack-kolla22:00
*** Sravan has quit IRC22:01
*** itlinux has quit IRC22:02
*** whoami-rajat has quit IRC22:11
*** jonaspaulo has quit IRC22:12
*** absubram has joined #openstack-kolla22:21
*** Talion has joined #openstack-kolla22:24
*** Sravan has joined #openstack-kolla22:28
*** gdwornicki has joined #openstack-kolla22:38
*** hamzaachi has joined #openstack-kolla22:42
*** hamzaachi has quit IRC22:46
*** hamzaachi has joined #openstack-kolla22:46
*** hamzaachi has quit IRC22:47
*** hamzaachi has joined #openstack-kolla22:48
*** goldyfruit has quit IRC22:49
*** Sravan has quit IRC22:49
*** hamzaachi has quit IRC22:59
*** hamzaachi has joined #openstack-kolla22:59
*** goldyfruit has joined #openstack-kolla23:03
*** dave-mccowan has joined #openstack-kolla23:12
*** absubram has quit IRC23:22
*** hamzaachi has quit IRC23:24
*** Sravan has joined #openstack-kolla23:25
*** goldyfruit has quit IRC23:28
*** Sravan has quit IRC23:29
*** chrizl has quit IRC23:30
*** goldyfruit has joined #openstack-kolla23:32
*** dave-mccowan has quit IRC23:33
*** JamesBenson has joined #openstack-kolla23:36
openstackgerritMerged openstack/kolla stable/stein: Install iputils package in neutron containers  https://review.opendev.org/66139523:37
*** jbadiapa has quit IRC23:37
*** jbadiapa has joined #openstack-kolla23:38
*** JamesBenson has quit IRC23:40
*** dave-mccowan has joined #openstack-kolla23:47
*** dave-mccowan has quit IRC23:53

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!