Tuesday, 2019-05-21

*** sshnaidm|afk has quit IRC00:04
*** sshnaidm has joined #openstack-kolla00:06
*** Sravan has joined #openstack-kolla00:23
*** Sravan has quit IRC00:28
*** gregoryo has joined #openstack-kolla01:50
*** Sravan has joined #openstack-kolla01:57
*** lei-zh has joined #openstack-kolla02:00
*** whoami-rajat has joined #openstack-kolla02:00
*** Sravan has quit IRC02:08
*** Sravan has joined #openstack-kolla02:10
*** Sravan_ has joined #openstack-kolla02:18
*** Sravan has quit IRC02:21
*** lei-zh has quit IRC02:32
*** lei-zh has joined #openstack-kolla02:33
*** hongbin has joined #openstack-kolla03:00
*** hrw has quit IRC03:01
*** hrw has joined #openstack-kolla03:02
*** Sravan_ has quit IRC03:15
*** Sravan has joined #openstack-kolla03:16
*** Sravan has quit IRC03:20
*** hrw has quit IRC03:37
*** hrw has joined #openstack-kolla03:38
*** Sravan has joined #openstack-kolla03:53
*** lei-zh has quit IRC04:06
*** hongbin has quit IRC04:21
*** Sravan has quit IRC04:31
*** Sravan has joined #openstack-kolla04:35
*** janki has joined #openstack-kolla04:37
*** itlinux has quit IRC04:46
*** pcaruana has joined #openstack-kolla05:02
*** Sravan has quit IRC05:02
*** pcaruana has quit IRC05:11
*** lei-zh has joined #openstack-kolla05:27
*** ivve has quit IRC05:34
*** radeks_ has joined #openstack-kolla05:36
*** Luzi has joined #openstack-kolla05:43
*** cah_link has joined #openstack-kolla05:54
*** zbr_ has quit IRC06:05
*** jistr is now known as jistr|mtg06:20
*** skramaja has joined #openstack-kolla06:25
*** ivve has joined #openstack-kolla06:43
*** luksky has joined #openstack-kolla07:00
mnasiadkamorning07:02
*** dpawlik has joined #openstack-kolla07:03
*** lemko has joined #openstack-kolla07:20
*** tolisbar has joined #openstack-kolla07:21
*** pcaruana has joined #openstack-kolla07:21
*** k_mouza has joined #openstack-kolla07:25
*** k_mouza has quit IRC07:26
*** k_mouza has joined #openstack-kolla07:26
*** k_mouza has quit IRC07:31
*** pcaruana has quit IRC07:39
*** k_mouza has joined #openstack-kolla07:39
*** k_mouza has quit IRC07:43
openstackgerritMerged openstack/kolla master: Add Qinling containers  https://review.opendev.org/65942207:46
*** tolisbar has quit IRC07:48
openstackgerritMerged openstack/kolla-ansible stable/stein: Fix issues obtaining Keystone token with Monasca Grafana  https://review.opendev.org/65799107:54
*** happyhemant has joined #openstack-kolla07:58
*** hamzaachi has joined #openstack-kolla08:00
*** JangwonLee has joined #openstack-kolla08:04
*** skramaja_ has joined #openstack-kolla08:05
*** skramaja has quit IRC08:07
openstackgerritMerged openstack/kolla stable/pike: Add sqlite package to manila-share  https://review.opendev.org/65798708:08
*** k_mouza has joined #openstack-kolla08:11
*** luksky has quit IRC08:11
*** k_mouza has quit IRC08:15
hrwelo08:17
*** priteau has joined #openstack-kolla08:20
*** k_mouza has joined #openstack-kolla08:21
openstackgerritMerged openstack/kolla-ansible stable/stein: Fix periodic CI jobs  https://review.opendev.org/66018008:23
*** guozijn has joined #openstack-kolla08:24
*** priteau has quit IRC08:24
*** k_mouza has quit IRC08:25
*** k_mouza has joined #openstack-kolla08:26
openstackgerritMark Goddard proposed openstack/kolla stable/queens: Bump service versions for Queens  https://review.opendev.org/66016008:29
*** priteau has joined #openstack-kolla08:29
*** k_mouza has quit IRC08:30
openstackgerritMark Goddard proposed openstack/kolla stable/pike: Bump service versions for Pike EM  https://review.opendev.org/66015908:32
*** Chaserjim has quit IRC08:34
*** Chaserjim has joined #openstack-kolla08:35
mgoddard\o08:43
mnasiadkahi mgoddard08:44
*** luksky has joined #openstack-kolla08:46
*** jistr|mtg is now known as jistr08:46
mnasiadkamgoddard: ceph is getting there, need to alter the osd bootstrap a bit and we should be home - then I'll check the upgrade08:47
mgoddardmnasiadka: great :)08:48
*** k_mouza has joined #openstack-kolla08:52
*** k_mouza_ has joined #openstack-kolla08:54
*** k_mouza_ has quit IRC08:56
*** k_mouza has quit IRC08:56
*** k_mouza has joined #openstack-kolla08:56
*** tolisbar has joined #openstack-kolla08:57
*** k_mouza has quit IRC08:57
*** k_mouza has joined #openstack-kolla08:58
*** tolisbar has quit IRC09:01
*** k_mouza_ has joined #openstack-kolla09:01
*** k_mouza has quit IRC09:02
*** cah_link has quit IRC09:08
*** openstackstatus has quit IRC09:13
*** openstackstatus has joined #openstack-kolla09:14
*** ChanServ sets mode: +v openstackstatus09:14
*** gregoryo has quit IRC09:15
*** yan0s has joined #openstack-kolla09:15
*** k_mouza_ has quit IRC09:25
*** guozijn_ has joined #openstack-kolla09:27
openstackgerritJakub Bielecki proposed openstack/kolla-ansible master: bugfix hostname for kolla-ansible repository  https://review.opendev.org/66031409:29
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Fix quickstart for virtual environments  https://review.opendev.org/66031509:30
*** guozijn has quit IRC09:31
*** guozijn_ is now known as guozijn09:31
*** lei-zh has quit IRC09:35
*** k_mouza has joined #openstack-kolla09:35
*** k_mouza has quit IRC09:39
*** k_mouza has joined #openstack-kolla09:41
*** skramaja has joined #openstack-kolla09:42
*** skramaja_ has quit IRC09:43
*** k_mouza_ has joined #openstack-kolla09:43
*** guozijn has quit IRC09:46
*** k_mouza has quit IRC09:46
*** k_mouza has joined #openstack-kolla09:46
*** k_mouza_ has quit IRC09:48
*** k_mouza has quit IRC09:51
*** k_mouza has joined #openstack-kolla10:03
*** dpawlik has quit IRC10:09
*** dpawlik has joined #openstack-kolla10:09
*** k_mouza has quit IRC10:18
*** k_mouza has joined #openstack-kolla10:32
hrwfailed: [localhost] (item=['br-ex', 'eth1']) => {"changed": false, "cmd": ["docker", "exec", "openvswitch_db", "/usr/local/bin/kolla_ensure_openvswitch_configured", "br-ex", "eth1"], "delta": "0:00:00.588335", "end": "2019-05-21 11:07:12.037645", "item": ["br-ex", "eth1"], "msg": "non-zero return code", "rc": 1, "start": "2019-05-21 11:07:11.449310", "stderr": "Device \"eth1\" does not exist.",10:34
hrw"stderr_lines": ["Device \"eth1\" does not exist."], "stdout": "", "stdout_lines": []}10:34
hrwmeh.10:34
hrwall-in-one gets more and more complicated10:35
* hrw digs for usb ethernet10:35
*** k_mouza has quit IRC10:36
hrwneutron_external_interface: "enx00248c005801"10:37
*** k_mouza has joined #openstack-kolla10:39
*** k_mouza has quit IRC10:41
*** k_mouza_ has joined #openstack-kolla10:41
mgoddardhrw: in CI we use a dummy interface10:43
*** k_mouza_ has quit IRC10:43
*** k_mouza has joined #openstack-kolla10:43
mgoddardhrw: 'ip l a fake_interface type dummy'10:44
hrwmgoddard: looks like good addon to 'bootstrap-servers'10:44
hrwalso prechecks need expansion to check for neutron_external_interface10:44
mgoddardhrw: well its just for testing - you won't get external network access10:44
mgoddardprecheck would be good10:45
hrwok. init-runonce started10:45
*** k_mouza has quit IRC10:45
hrwok. master ubuntu/source install works10:48
*** k_mouza has joined #openstack-kolla10:49
*** Chaserjim has quit IRC10:50
hrwnow master debian/source, then py3 ubuntu/source, py3 debian/source10:52
hrwsimple 'deploy and run vm'10:52
*** k_mouza has quit IRC10:54
openstackgerritJakub Bielecki proposed openstack/kolla-ansible master: bugfix hostname for kolla-ansible repository  https://review.opendev.org/66031410:56
hrwmnasiadka: https://review.opendev.org/#/c/657370/ https://review.opendev.org/#/c/660089/11:06
*** Chaserjim has joined #openstack-kolla11:15
*** Chaserjim has quit IRC11:20
openstackgerritMerged openstack/kolla stable/rocky: Fix potential Monasca-thresh start failure  https://review.opendev.org/65585711:22
hrwok. master debian/source skipped11:23
*** Chaserjim has joined #openstack-kolla11:23
openstackgerritRafael Weingärtner proposed openstack/kolla-ansible master: Method to override the default ceilometer meters.yaml via Kolla-ansible  https://review.opendev.org/65698111:24
*** k_mouza has joined #openstack-kolla11:26
*** dave-mccowan has joined #openstack-kolla11:28
*** kplant has joined #openstack-kolla11:29
*** k_mouza has quit IRC11:30
*** k_mouza_ has joined #openstack-kolla11:30
*** k_mouza_ has quit IRC11:34
hrwhm. "Cannot update service status on host "gossamer" since it is not registered.: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host gossamer could not be found."11:35
hrw'hypervisor list' lists it as gossamer.lan as it was called...11:36
kplanthow are the nova containers doing on the hypervisor itself?11:36
hrwchanged, redeploy...11:36
kplantcould also be rabbit11:44
*** henriqueof has joined #openstack-kolla11:44
openstackgerritJakub Bielecki proposed openstack/kolla-ansible master: bugfix hostname for kolla-ansible repository  https://review.opendev.org/66031411:45
*** Sravan has joined #openstack-kolla11:50
openstackgerritChason Chan proposed openstack/kolla-ansible master: Update fluentd filter for consistencies of log  https://review.opendev.org/66035911:54
*** Sravan has quit IRC11:54
*** janki has quit IRC11:54
*** k_mouza has joined #openstack-kolla11:58
*** k_mouza_ has joined #openstack-kolla12:00
*** k_mouza_ has quit IRC12:01
*** k_mouza__ has joined #openstack-kolla12:01
*** k_mouza__ has quit IRC12:02
hrwhate deployments12:02
*** k_mouza_ has joined #openstack-kolla12:02
*** k_mouza has quit IRC12:02
*** henriqueof has quit IRC12:04
*** k_mouza_ has quit IRC12:07
hrwhttps://github.com/ansible/ansible/issues/4210512:21
hrwyay..12:21
hrwok. time to patch kolla-toolbox to use recent ansible not ancient 2.2 one12:22
mnasiadkahrw: tried that, but will not be so easy - some modules need some small rewriting to fit the newer Ansible API12:23
mnasiadkahrw: it's on my radar, but currently working on bloody ceph12:23
hrwmnasiadka: shit.12:24
*** k_mouza has joined #openstack-kolla12:26
*** k_mouza_ has joined #openstack-kolla12:30
*** k_mouza has quit IRC12:30
hrwmnasiadka: any wip patch?12:31
mnasiadkahrw: https://review.opendev.org/#/c/642416/12:32
hrwthx12:32
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: Bump up pkg versions in kolla-toolbox  https://review.opendev.org/64241612:34
hrwjust added closes-bug info12:35
mnasiadkagood :)12:36
*** priteau has quit IRC12:36
hrw14:36 (845s) hrw@gossamer:kolla-toolbox$ git review -x 64241612:37
hrw(venv3) 14:36 (5526s) hrw@gossamer:kolla$ ~/devel/docker-utils/kolla-build-images.sh 'debian' source ' --push --registry gossamer.lan:2000 kolla-toolbox' py312:37
hrwlet's check12:37
*** k_mouza_ has quit IRC12:37
*** k_mouza has joined #openstack-kolla12:39
*** priteau has joined #openstack-kolla12:40
*** cah_link has joined #openstack-kolla12:50
*** priteau has quit IRC12:51
erolgIs there anyone who is using keystone "reader" or "observer" role here? I think reader and observer roles are not working according to this spec https://specs.openstack.org/openstack/keystone-specs/specs/keystone/rocky/define-default-roles.html12:53
erolgWhen I try these roles there is no diffrence from member12:53
erolgI think other projects weren't implement these roles yet.12:54
*** skramaja has quit IRC13:06
*** janki has joined #openstack-kolla13:11
openstackgerritMerged openstack/kolla master: nova-libvirt: in Debian qemu-kvm is x86 only  https://review.opendev.org/65737013:19
*** goldyfruit has joined #openstack-kolla13:23
openstackgerritMerged openstack/kolla-ansible master: prechecks: be nice, use print in Py3 way  https://review.opendev.org/66008913:24
*** k_mouza has quit IRC13:27
*** k_mouza has joined #openstack-kolla13:29
*** michaelbarkdoll has joined #openstack-kolla13:32
*** k_mouza has quit IRC13:33
michaelbarkdollHi, I've successfully deployed kolla-ansible to an all-in-one configuration and have external networking working.  I'd now like to setup multiple external vlans for the external networking, but I'm seeing alot of people face issues with it on ask.openstack.org, so I created a question regarding the best procedure: https://ask.openstack.org/en/question/122223/kolla-ansible-multiple-external-vlans-configuration/13:34
michaelbarkdollI'm curious if I should modify the kolla-anisble/tools/init-runonce script and change the openstack network and subnet creations or if I first need to modifiy the deployment with an custom /etc/kolla/config/neutron/ml2_conf.ini with something like:13:34
michaelbarkdoll[ml2_type_vlan]  network_vlan_ranges = physnet1:10:20013:34
*** k_mouza has joined #openstack-kolla13:35
kplanti changed mine to: 'network_vlan_ranges = physnet1'13:35
kplantthat allows all tags13:35
kplantand i'm overriding that in /etc/kolla/config/neutron/ml2_conf.ini13:36
*** k_mouza_ has joined #openstack-kolla13:38
michaelbarkdollIs that the only modification needed then I can just `openstack network create --external --provider-physical-network physnet1 --provider-network-type vlan public1` ?13:39
michaelbarkdollAfter deploment13:39
kplantyep, as long as you have an interface added to br-ex and not a sub-interface (barring qinq)13:39
*** k_mouza_ has quit IRC13:39
*** k_mouza has quit IRC13:40
kplantyou also need to specify the tag in the network create13:40
kplantopenstack network create blah --external --no-share --provider-physical-network physnet1 --provider-network-type vlan --provider-segment 102413:41
kplantyou want something like that13:41
michaelbarkdollSo, I run that for each vlan?13:41
kplantyes13:42
*** k_mouza has joined #openstack-kolla13:42
*** itlinux has joined #openstack-kolla13:42
michaelbarkdollSorry, last question.  kolla-ansible seems to add my neutron_external_interface: "enp0s8" to br-ex -- when you say "as long as you have an interface added to br-ex and not a sub-interface (barring qinq)" is there an additional step that I need to do like setup some type of OVSport or something?13:43
kplantnope k-a will do that for you13:44
*** cah_link has quit IRC13:44
kplantjust make sure you have no configuration on that interface you care about13:44
michaelbarkdollThanks so much, I'll update my answer with this information on the forums to help others.  I really appreciate your help on this matter.13:44
kplantnot a problem13:44
*** henriqueof has joined #openstack-kolla13:47
michaelbarkdollkplant, br-ex should accept something like a bonded interface as well right?13:50
kplantyep13:50
michaelbarkdollGreat, thanks again.13:50
kplantneutron_external_interface: "bond0"13:51
kplantthat's exactly what i use13:51
*** erolg has quit IRC13:53
openstackgerritRafael Weingärtner proposed openstack/kolla-ansible master: Method to override the default ceilometer meters.yaml via Kolla-ansible  https://review.opendev.org/65698113:55
*** jchhatbar has joined #openstack-kolla13:57
*** calbers_ has joined #openstack-kolla13:58
*** spiette_ has joined #openstack-kolla13:59
*** gary_perkins_ has joined #openstack-kolla14:00
*** goldyfruit has quit IRC14:00
*** gfidente|afk is now known as gfidente14:00
*** Reepicheep has joined #openstack-kolla14:00
*** mrunge_ has joined #openstack-kolla14:01
*** quoth_ has joined #openstack-kolla14:02
*** goldyfruit has joined #openstack-kolla14:03
*** janki has quit IRC14:06
*** markmcclain has quit IRC14:06
*** mrunge has quit IRC14:06
*** dosaboy has quit IRC14:06
*** Reepicheep_ has quit IRC14:06
*** henriqueof has quit IRC14:06
*** sshnaidm has quit IRC14:06
*** spiette has quit IRC14:06
*** calbers has quit IRC14:06
*** johnthetubaguy has quit IRC14:06
*** gary_perkins has quit IRC14:06
*** mjturek has quit IRC14:06
*** strigazi has quit IRC14:06
*** calbers_ is now known as calbers14:06
*** yankcrime has quit IRC14:07
*** Sravan has joined #openstack-kolla14:07
*** Luzi has quit IRC14:09
*** sshnaidm has joined #openstack-kolla14:12
*** itlinux has quit IRC14:13
*** Sravan has quit IRC14:13
hrw    "msg": "The conditional check '(keystone_bootstrap.stdout | from_json).changed' failed. The error was: Invalid control character at: line 1 column 295 (char 294)"14:27
hrwtime to debug ;(14:27
*** erolg has joined #openstack-kolla14:39
*** erolg has left #openstack-kolla14:39
*** erolg has joined #openstack-kolla14:40
*** k_mouza has quit IRC14:44
*** k_mouza has joined #openstack-kolla14:46
*** luksky has quit IRC14:49
*** k_mouza_ has joined #openstack-kolla14:50
*** k_mouza has quit IRC14:51
*** k_mouza_ has quit IRC14:54
*** Chaserjim has quit IRC14:55
*** Chaserjim has joined #openstack-kolla14:55
hrwheh. instead of giving readable traceback I got 15639 chars long line14:56
*** yankcrime has joined #openstack-kolla14:56
hrw"Table 'keystone.project' doesn't exist"14:57
hrwha!14:57
*** k_mouza has joined #openstack-kolla15:00
*** jchhatbar is now known as janki15:02
*** hamzy has quit IRC15:07
*** yan0s has quit IRC15:13
mgoddardhrw: sounds like you are having all the fun :)15:14
*** priteau has joined #openstack-kolla15:24
hrwo yes!15:24
hrwmgoddard: py3 ubuntu/source deploys fine but has issue with creating VMs15:25
hrwmgoddard: py3 debian/source fails due to python 3.7 being too new for our very old ansible version.15:25
hrwmgoddard: and updating to newer ansible requires rewriting parts of kolla-ansible15:25
mgoddardhrw: I thought containers were supposed to fix dependency hell? :p15:26
hrwmgoddard: ha ha ha15:26
openstackgerritMark Goddard proposed openstack/kolla master: Update contributor guide  https://review.opendev.org/66042615:30
mgoddardFinally got round to writing up PTL responsibilities ^15:31
hrwmgoddard, mnasiadka: which Ansible version we require now?15:33
hrwand can we bump it to 2.7.7+?15:33
mgoddardhrw: ansible_version_min: '2.4.0'15:34
mgoddardhrw: why bump?15:34
hrwmgoddard: python 3.7 in Debian images15:34
mgoddardhrw: is that the earliest version that supports py3.7?15:34
hrwmgoddard: have to check15:35
*** hamzy has joined #openstack-kolla15:35
priteauI noticed Kolla-Ansible sets many endpoints with a `/%(tenant_id)s` prefix. Is this still useful? I just checked the Nova install guide for stein and they recommend using `http://controller:8774/v2.1`15:38
mgoddardpriteau: not sure. If nova recommends without then I guess we should remove it.15:39
priteauWe should also retire legacy Nova endpoints: https://opendev.org/openstack/kolla-ansible/src/branch/master/ansible/roles/nova/defaults/main.yml#L341-L34315:43
priteauThe v2 API code was removed in Newton15:43
hrw17:42 < hrw> which version of ansible is the first one working with Python 3.7?15:43
hrw17:42 <@sivel> hrw: Ansible 2.5 iirc, due to the reserved async keyword15:43
hrw17:43 <@sivel> 2.5 is no unsupported anyway, so 2.6+ should all work and support Python 3.715:43
hrwmgoddard: so 2.6.0 should be new minimal15:43
*** k_mouza has quit IRC15:45
mgoddardhrw: seems reasonable, it's been out for a while now. Should we say it is minimum for py3 or py2 as well?15:46
hrwmgoddard: 2.6.0 minimum.15:49
*** ivve has quit IRC15:49
hrwno py3 or py2. no ubuntu or centos or debian15:49
hrwjust ansible_version_min: 2.6.015:49
mgoddarddo all supported distros provide that as a minimum?15:50
kplantcentos7 is still shipping 2.4 in the repo15:51
hrwDebian has 2.7.5 (stretch-backports) 2.7.7 (buster), CentOS has even 2.8 in ovirt repos15:51
kplantthough epel-release and others have 2.815:51
kplant^ what he said15:51
hrwubuntu 18.04 has 2.5.115:51
mgoddardhmm15:52
hrwwhich is already unsupported by upstream15:52
hrwso they can always 'pip install ansible==2.6.0' or sth15:52
mgoddardbut presumably supported downstream?15:52
hrwmgoddard: ansible is in universe part not main15:53
mgoddardok15:53
*** k_mouza has joined #openstack-kolla15:53
hrwhttps://bugs.launchpad.net/ubuntu/+source/ansible/+bug/1780278 is an old bug saying 2.5.1 is unusable. completelly ignored15:54
openstackLaunchpad bug 1780278 in ansible (Ubuntu) "ansible 2.5.1 need upgrade to 2.5.2 or 2.5.3 (bug with set_facts)" [Undecided,Confirmed]15:54
*** k_mouza has quit IRC15:54
mgoddardhrw: https://docs.openstack.org/kolla-ansible/latest/user/quickstart.html#install-dependencies-not-using-a-virtual-environment15:55
*** k_mouza has joined #openstack-kolla15:55
hrwrest of their bugs can be summaried: 2.5.1 is broken15:55
*** janki has quit IRC15:55
mgoddardok15:55
mgoddardlet's go for 2.6 then15:56
hrwmgoddard: in other words: docs needs update too15:56
hrwmnasiadka: ping15:56
*** k_mouza_ has joined #openstack-kolla15:56
mnasiadka2.6 is a safe go I guess15:57
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Update contributor guide  https://review.opendev.org/66044215:57
mnasiadkabut about kolla-toolbox - maybe it's better to go 2.8 in the image, so we don't have to move when 2.6 gets dropped15:57
openstackgerritMerged openstack/kolla stable/rocky: Add monasca user to Monasca thresh container  https://review.opendev.org/65585815:57
hrwmnasiadka: so 2.6 as minimum, kolla-toolbox to 2.8 and then update playbooks/etc according to 2.8 porting guide?15:58
hrwmnasiadka: or according to 2.6 porting guide to be safe?15:58
*** k_mouza has quit IRC15:58
*** hamzaachi_ has joined #openstack-kolla16:00
*** k_mouza_ has quit IRC16:01
mnasiadkahrw: ansible support lasts for 3 releases, so when 2.9 goes out (in 4-5 months) - 2.6 is dead16:01
mnasiadkamgoddard: ideas?16:01
hrw2.7.0?16:01
mnasiadkaprobably that's the safe approach16:02
*** hamzaachi has quit IRC16:02
mnasiadkamaybe let's add it as a topic for tomorrow's meeting16:02
mgoddard2.7 is probably a good compromise16:02
mgoddardalthough we're only at the start of the release so 2.8 might be more stable by the end16:03
hrwhttps://bugs.launchpad.net/kolla-ansible/+bug/182989516:04
openstackLaunchpad bug 1829895 in kolla-ansible train "Update Ansible to 2.6+ due to Python 3.7" [High,New]16:04
hrwmgoddard: 2.7 is in Debian now.16:04
hrw2.8 is not yet there16:04
mgoddardhrw: I meant for kolla-toolbox, where we install via pip16:05
hrwok16:05
*** hamzaachi_ has quit IRC16:05
hrwadded discussion to the bug16:06
*** luksky has joined #openstack-kolla16:06
mnasiadkamgoddard: for kolla-toolbox we should go with 2.8, and probably master playbooks can be also adjusted for 2.8 - 2.9 will be close to release when we release train16:07
*** hamzaachi_ has joined #openstack-kolla16:08
*** erolg has quit IRC16:08
mnasiadkamgoddard: 2.7 is also safe, but we'll need to bump sooner then :)16:11
mnasiadkamgoddard: or we need to take care about bumping ansible version every release :)16:11
*** graysonh has joined #openstack-kolla16:12
*** igordc has joined #openstack-kolla16:20
*** mgoddard has quit IRC16:25
*** mgoddard has joined #openstack-kolla16:28
mnasiadkayay, all ceph services deploy properly with nautilus16:31
*** Sravan has joined #openstack-kolla16:31
hrwmnasiadka: so you are free for ansible part? :D16:37
mnasiadkahrw: I guess I will be tomorrow ;)16:37
hrwmnasiadka: bosko16:37
*** mgoddard has quit IRC16:42
*** mgoddard has joined #openstack-kolla16:43
*** hamzaachi_ has quit IRC16:46
*** Sravan has quit IRC16:48
*** whoami-rajat has quit IRC16:49
*** itlinux has joined #openstack-kolla16:57
*** roukoswarf has joined #openstack-kolla16:58
*** ivve has joined #openstack-kolla16:59
roukoswarfso im testing my way through a rocky->stein upgrade, and doing a synced flush of elasticsearch seems pretty much like a whack a mole? it gets a pending operations status from a few entries and fails out.17:00
*** dpawlik has quit IRC17:08
roukoswarfshould the flush be tuned to wait longer?17:09
*** igordc has quit IRC17:12
*** igordc has joined #openstack-kolla17:13
*** whoami-rajat has joined #openstack-kolla17:15
*** dpawlik has joined #openstack-kolla17:20
*** itlinux has quit IRC17:22
openstackgerritPierre Riteau proposed openstack/kolla-ansible master: Add ansible_nodename (system hostname) to /etc/hosts  https://review.opendev.org/66046317:24
*** dpawlik has quit IRC17:24
*** Sravan has joined #openstack-kolla17:27
*** priteau has quit IRC17:42
*** itlinux has joined #openstack-kolla17:44
*** igordc has quit IRC17:46
*** Sravan has quit IRC18:01
*** Sravan has joined #openstack-kolla18:02
*** Sravan has quit IRC18:04
*** Sravan has joined #openstack-kolla18:12
*** Sravan has quit IRC18:15
*** igordc has joined #openstack-kolla18:18
*** Sravan has joined #openstack-kolla18:19
*** Sravan has quit IRC18:22
*** Sravan has joined #openstack-kolla18:25
*** itlinux has quit IRC18:40
*** unicell has joined #openstack-kolla18:43
v1k0d3nhey everyone...sorry if this is a pretty basic question, but is there a document page for octavia? i guess the expectation is for users to have certificates created in advance (which I have done following Octavia's docs), but where do i configure kolla-ansible to look for, and copy these certs?18:44
openstackgerritMerged openstack/kolla stable/rocky: Use alembic migrations to deploy Monasca DB  https://review.opendev.org/65585918:45
v1k0d3ni thought i saw some docs around this at one point, but i can't seem to find it in the master docs (i'm using development on master branch right now, since i'm on ubuntu 18.04)18:45
ivvev1k0d3n: its in {{ config_dir }}/octavia/18:47
v1k0d3nah, i guess sticking them in `/etc/kolla/octavia` would make the most sense.18:47
ivveit will look for ca_01.pemm cakey.pem and client.pem18:47
ivveno, /etc/kolla/config/octavia18:48
ivveif you're going with default18:48
v1k0d3noh! good call. ok.18:48
ivvejust assuming..18:48
ivvethere is a pretty ok guide in the bin dir in octavia project18:49
ivvetip: use absolute paths18:49
ivve:)18:49
v1k0d3nthat's really helpful.  thanks ivve!18:49
*** goldyfruit has quit IRC18:50
ivveno worries18:50
ivveanyone here got a working watcher/freezer running under ubuntu 16/18(.04) ?18:51
*** Sravan has quit IRC18:52
ivvegot timezone errors from schedulers18:53
ivvesuch as https://bugs.launchpad.net/kolla/+bug/182159218:53
openstackLaunchpad bug 1821592 in kolla "Kolla Watcher Service - Unable to determine the name of the local timezone" [Undecided,Incomplete]18:53
*** Sravan has joined #openstack-kolla18:54
openstackgerritMerged openstack/kolla stable/rocky: Downgrade Logstash for Monasca  https://review.opendev.org/65595318:55
*** sshnaidm is now known as sshnaidm|afk18:56
openstackgerritMerged openstack/kolla stable/rocky: Build and install Fluentd Monasca plugin  https://review.opendev.org/65595418:57
*** graysonh_ has joined #openstack-kolla19:02
*** jonaspaulo has joined #openstack-kolla19:03
*** graysonh has quit IRC19:04
*** jonaspaulo has quit IRC19:09
*** jonaspaulo has joined #openstack-kolla19:12
v1k0d3nhey ivve ... sorry one more question. things appear to be set up correctly (famous words), but the octavia (load-balancer) horizon dashboard appears to be empty. anything that you're aware of that's preventing this from appearing?19:21
ivvewhat did you run? (kolla-ansible)19:22
v1k0d3ncorrect. i ran a reconfigure since this was a basic previous installation.19:23
ivvewith tags or a full?19:23
v1k0d3nhmmm...when you say with tags, what do you mean exactly?19:23
ivve--tags octavia19:23
ivveansible tags19:24
v1k0d3nah, no...i just ran a reconfigure. i guess that was incorrect?19:24
v1k0d3ni did full19:24
ivvewhat verions are you running?19:24
v1k0d3nmaster from yesterday19:25
v1k0d3nlast commit was `Merge "Add support for Ceilometer custom gnocchi_resources.yaml` @ `Thu May 16 11:00:26 2019 +0000`19:25
ivvedoes the api work? can you run openstackclient?19:27
v1k0d3nyes, everything seems to work really well.19:27
v1k0d3nlet me specifically call the octavia apis. one sec.19:27
*** whoami-rajat has quit IRC19:29
v1k0d3ndo i need to enable the neutron-lbaas as well? that seems like it's intended for the older non-octavia lbaas, is that the right assumption?19:30
ivveno, not unless you plan to use it (with proxy)19:30
ivvei don't use it at all19:31
v1k0d3nultimately, this will be for kubernetes installations in openstack.19:31
v1k0d3nok, good to know.19:31
ivveso do i19:31
ivvehehe19:31
ivveyou're in for a run there..19:31
v1k0d3noh boy.19:31
*** Sravan has quit IRC19:31
ivveoh boy doesn't even describe 1%19:32
v1k0d3nwell, we have customers using it...so it's good to get back in. when i deployed k8s in os a long time ago...it was very different.19:32
v1k0d3nthat must be why i've been asked to look at it (i'm with tigera).19:32
*** Sravan has joined #openstack-kolla19:32
v1k0d3nthis is going to be fun :)19:32
ivvehehe19:32
kplantmagnum is pretty good at deploying k8s19:33
kplantare you using magnum?19:33
ivveyes19:33
v1k0d3nso what's the best approach here then, based on your experiences?19:33
ivvebut there are many "bugs"19:33
v1k0d3ni would imagine that magnum is still using octavia, correct?19:33
kplantit can19:33
v1k0d3nthere's _always_ been many bugs.19:33
kplantyou can also write your own heat stack and use kubespray and haproxy19:33
v1k0d3nsince the very beginning. and nobody's fault either...it's just...complex.19:33
ivveyeah well, k8s is nearly as complex as a bunch of openstack services together19:34
ivveso its inception in complexity19:34
v1k0d3nlol19:35
*** Sravan has quit IRC19:35
ivveyou have to figure out how you want health_managers to talk to eachother19:35
ivve(octavia)19:35
ivvealso there are some old deprecated settings that screw up the entire config, here they are19:36
v1k0d3nso openstack appears to be fine with loadbalancer commands.19:36
ivveyou need to set those both19:36
v1k0d3ni should try to create something to see if the octavia services are responsive.19:36
ivvehealth_update_threads19:36
ivvestats_update_threads19:36
ivveif you don't nothing will work..19:36
*** Sravan has joined #openstack-kolla19:37
v1k0d3ni'm cheating at the moment with an AIO w/o haproxy/keepalived19:37
ivvesince all 3 settings (including the deprecated) defaults to 'none'19:37
v1k0d3nbeefy server, but still not exactly multi-node complex.19:37
ivveokay, well if you're going to use multi-controller setup. i suggest you use vxlan for octavia.. i link my nodes through a vxlan on the int bridge19:38
ivveoh yea, i use openvswitch btw.19:38
v1k0d3nthat makes sense. i do that typically for multi-node as well. and same on ovs.19:38
ivveso basically assign ports to the controller nodes19:39
ivvedhclient to get your stuff, remember port id, mac and also to assign them in neutron so that they aren't available :)19:39
ivveone thing i noticed, if you misconfigure octavia a bit and run keepalived/haproxy the health_manager can kill haproxy19:40
ivveso basically all your services stop and you're left trying to figure out whats going on19:41
ivveso if the entire stack decides to flip, kill health_manager :)19:41
v1k0d3noh my19:42
ivveand test it throughly before putting it into your prod19:43
v1k0d3nso create health manager. is this per lb-pool, or k8s cluster in our case?19:43
*** kplant has quit IRC19:43
ivveoctavia uses "a network" to communicate with amphorae19:44
ivvemore or less api calls to them as well as heartbeats and checks once they pop up19:44
ivveto configure them19:44
v1k0d3nhmm..19:44
*** goldyfruit has joined #openstack-kolla19:45
ivvethere is a good doc on this in the project19:45
v1k0d3nok...so in my case, i'm going fairly simple with a flat network on physnet1 with a connected external network.19:45
v1k0d3nhmm...one sec. i'm in that repo...looking now.19:45
ivveyeah in that case just remember the name of the network must be lb-mgmt-net and you need to configure it in octavia.conf once it is setup19:46
ivveim guessing you can use the octavia network var in kolla if you wish. since i use a vxlan19:47
*** Sravan has quit IRC19:47
v1k0d3nah, so maybe it's best to create a new network, and legitimately use the public/external network i already have configured as floating ip's.19:47
ivvei dont, since i use a vxlan19:47
*** radeks_ has quit IRC19:48
ivvethe amphoras will be spawned directly on the network, it is a management network for them and the controllers19:48
ivveand they both need to communicate in that network, if im not mistaken it must also be L219:49
openstackgerritJim Rollenhagen proposed openstack/kolla-ansible master: Use follow: no when creating node_config_directory  https://review.opendev.org/66050219:50
ivveoctavia utilizes udp and vrrp as well in that network19:50
ivvejust so you know19:50
v1k0d3nthat's good to know.19:51
ivveheh yeah :D19:51
v1k0d3nstill curious why the horizon dashboard isn't showing anything for lbaas.19:51
v1k0d3nstrange.19:51
ivvedid you try a fresh browser?19:51
v1k0d3njust did yes19:52
ivveanything from the horizon logs?19:53
v1k0d3njust started digging into them.19:53
v1k0d3ni'll post what i find.19:53
v1k0d3nthanks for the help btw.19:53
ivveno worries, feel free to ask if you get stuck19:54
*** logan- has quit IRC19:55
v1k0d3ncrap...even roles/groups aren't showing up now.19:56
v1k0d3nthat's interesting (even for the admin). wonder if the reconfigure jacked something up.19:56
v1k0d3nit's a new environment...so it's fine if i blow it away and recreate it. luckily users/roles, etc are all scripted out.19:56
v1k0d3nhorizon looks like it's sending information just fine. no real problems.19:57
ivvehmm19:57
ivvelemme think19:58
ivvelocal_settings perhaps19:58
ivvesometimes some settings simply won't get swallowed and then alot of settings go default or just bork19:59
ivvebut i guess it could be something with the horizon dash.. can you try stein?20:00
ivvei run stein and it works perfectly20:01
v1k0d3nredeploying now.20:13
*** Chaserjim has quit IRC20:13
*** Chaserjim has joined #openstack-kolla20:14
*** lemko has quit IRC20:20
*** henriqueof has joined #openstack-kolla20:35
v1k0d3nivve: you're right...dashboard is available now.20:37
v1k0d3nusing stein20:37
ivvev1k0d3n: cool20:40
ivvei use stein for all containers except neutron and magnum20:41
ivveneutron = rocky and magnum = master20:41
ivveneutron has some nasty external network bug atm, they had a huge commit that changed around a lot of stuff in that area20:42
*** Talion has joined #openstack-kolla20:43
*** hamzy has quit IRC20:45
*** dave-mccowan has quit IRC20:50
*** dave-mccowan has joined #openstack-kolla20:56
*** gfidente has quit IRC21:02
*** gfidente has joined #openstack-kolla21:03
v1k0d3noh no...21:04
v1k0d3nthat's really bad, because most of what i will be diving into is the neutron configs.21:04
*** unicell has left #openstack-kolla21:05
v1k0d3ni don't recall, but how do you override individual services like that? it's been 3 years at least since i've really gone that deep on kolla-ansible.21:05
v1k0d3nbasically since we started openstack-helm. :-/21:06
v1k0d3ni'll search docs. been coming up short on some docs in kolla lately (like what items to configure in globals for magnum).21:06
*** logan- has joined #openstack-kolla21:13
*** graysonh_ has quit IRC21:20
*** henriqueof has quit IRC21:21
ivvemgoddard: you awake? did some short troubleshooting.. it seems apscheduler needs /etc/timezone as well as /etc/localtime. so that needs to be included (https://bugs.launchpad.net/kolla/+bug/1821592)21:22
openstackLaunchpad bug 1821592 in kolla "Kolla Watcher Service - Unable to determine the name of the local timezone" [Undecided,Incomplete]21:22
*** Sravan has joined #openstack-kolla21:23
ivvetested this under ubuntu 18.04, source, stein, kolla-ansible 8.0.0.0rc121:23
*** Sravan has quit IRC21:35
*** Sravan has joined #openstack-kolla21:36
*** Sravan has quit IRC21:39
*** dave-mccowan has quit IRC21:40
*** Sravan has joined #openstack-kolla21:40
*** unicell has joined #openstack-kolla22:00
*** gfidente is now known as gfidente|afk22:05
*** dciabrin_ has quit IRC22:10
*** dciabrin_ has joined #openstack-kolla22:11
*** hamzaachi has joined #openstack-kolla22:25
*** itlinux has joined #openstack-kolla22:26
*** hamzaachi has quit IRC22:26
*** hamzaachi has joined #openstack-kolla22:27
*** hamzaachi has quit IRC22:28
*** hamzaachi has joined #openstack-kolla22:28
*** unicell has left #openstack-kolla22:29
*** Sravan has quit IRC22:30
*** goldyfruit has quit IRC22:41
*** itlinux has quit IRC22:43
*** hamzaachi has quit IRC22:44
*** hamzaachi has joined #openstack-kolla22:45
*** hamzaachi has quit IRC22:45
*** hamzaachi has joined #openstack-kolla22:46
*** Talion has quit IRC22:52
*** goldyfruit has joined #openstack-kolla22:54
*** hamzaachi has quit IRC22:54
*** jonaspaulo has quit IRC22:59
*** calbers has quit IRC23:07
*** Sravan has joined #openstack-kolla23:26
*** luksky has quit IRC23:26
*** calbers has joined #openstack-kolla23:26
*** Sravan has quit IRC23:35
*** Sravan has joined #openstack-kolla23:39
*** Sravan has quit IRC23:43
*** itlinux has joined #openstack-kolla23:45
*** itlinux_ has joined #openstack-kolla23:47
*** itlinux has quit IRC23:49
*** itlinux_ has quit IRC23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!