Monday, 2020-04-06

openstackgerritHongbin Lu proposed openstack/kolla-ansible master: [WIP] Zun: Add zun-cni-daemon to compute node  https://review.opendev.org/70821300:14
*** k_mouza has joined #openstack-kolla01:38
*** k_mouza has quit IRC01:43
openstackgerritjacky06 proposed openstack/kolla-cli master: Update hacking for Python3  https://review.opendev.org/71764501:52
*** sorin-mihai has quit IRC01:56
openstackgerritjacky06 proposed openstack/kolla-cli master: Fix some pep8 failed  https://review.opendev.org/71765002:33
openstackgerritjacky06 proposed openstack/kolla-cli master: Fix some pep8 failed  https://review.opendev.org/71765002:35
openstackgerritjacky06 proposed openstack/kolla-cli master: Use unittest.mock instead of third party mock  https://review.opendev.org/71765303:11
openstackgerritjacky06 proposed openstack/kolla-cli master: Use unittest.mock instead of third party mock  https://review.opendev.org/71765303:13
openstackgerritMerged openstack/kolla-cli master: Fix some pep8 failed  https://review.opendev.org/71765003:23
*** k_mouza has joined #openstack-kolla03:39
*** k_mouza has quit IRC03:44
*** cah_link has joined #openstack-kolla03:44
*** cah_link1 has joined #openstack-kolla03:55
*** cah_link has quit IRC03:56
*** cah_link1 is now known as cah_link03:56
*** skramaja has joined #openstack-kolla04:07
*** ykarel|away is now known as ykarel04:12
openstackgerritjacky06 proposed openstack/kolla-cli master: Use unittest.mock instead of third party mock  https://review.opendev.org/71765304:13
openstackgerritHongbin Lu proposed openstack/kolla-ansible master: [WIP] Zun: Add zun-cni-daemon to compute node  https://review.opendev.org/70821304:22
openstackgerritjacky06 proposed openstack/kolla-cli master: Use unittest.mock instead of third party mock  https://review.opendev.org/71765304:33
*** evrardjp has quit IRC04:36
*** evrardjp has joined #openstack-kolla04:37
openstackgerritHongbin Lu proposed openstack/kolla-ansible master: [WIP] Zun: Add zun-cni-daemon to compute node  https://review.opendev.org/70821304:40
openstackgerritjacky06 proposed openstack/kayobe master: Use unittest.mock instead of third party mock  https://review.opendev.org/71765905:08
*** diga has joined #openstack-kolla05:09
*** shyamb has joined #openstack-kolla05:12
openstackgerritjacky06 proposed openstack/kolla-cli master: Update hacking for Python3  https://review.opendev.org/71764505:22
openstackgerritDincer Celik proposed openstack/kolla-ansible master: Introduce /etc/timezone to Debian/Ubuntu containers  https://review.opendev.org/70425805:30
cosmicsoundGoood day05:30
osmanlicilegimorning05:36
*** shyamb has quit IRC05:43
openstackgerritMerged openstack/kolla-cli master: Use unittest.mock instead of third party mock  https://review.opendev.org/71765305:48
*** cah_link1 has joined #openstack-kolla05:52
*** cah_link has quit IRC05:54
*** cah_link1 is now known as cah_link05:54
*** shyamb has joined #openstack-kolla05:55
openstackgerritjacky06 proposed openstack/kolla-cli master: Update hacking for Python3  https://review.opendev.org/71764505:56
openstackgerritjacky06 proposed openstack/kolla-cli master: Update hacking for Python3  https://review.opendev.org/71764505:57
*** abdysn has joined #openstack-kolla06:10
*** bjolo_ has joined #openstack-kolla06:14
bjolo_morning06:14
*** bjolo_ has left #openstack-kolla06:16
*** bjolo_ has joined #openstack-kolla06:22
openstackgerritMerged openstack/kolla-cli master: Update hacking for Python3  https://review.opendev.org/71764506:25
*** jbadiapa has joined #openstack-kolla06:27
*** rpittau|afk is now known as rpittau06:35
rpittauhrw: that was not me :)06:37
openstackgerritMerged openstack/kolla-ansible master: Update hacking for Python3  https://review.opendev.org/71738206:43
*** iniazi has quit IRC06:48
*** shyam89 has joined #openstack-kolla06:55
*** shyamb has quit IRC06:58
*** zhanglong has joined #openstack-kolla07:03
*** bengates has joined #openstack-kolla07:07
*** bengates has quit IRC07:09
*** bengates has joined #openstack-kolla07:09
*** shyam89 has quit IRC07:11
*** e0ne has joined #openstack-kolla07:13
*** pbing19 has joined #openstack-kolla07:28
*** born2bake has joined #openstack-kolla07:31
*** cah_link1 has joined #openstack-kolla07:31
*** cah_link has quit IRC07:32
*** cah_link1 is now known as cah_link07:32
*** iniazi has joined #openstack-kolla07:32
*** shyamb has joined #openstack-kolla07:43
*** happyhemant has joined #openstack-kolla07:46
cosmicsoundhey osmanlicilegi07:48
*** spsurya_ has joined #openstack-kolla07:59
*** ktibi has joined #openstack-kolla08:01
osmanlicilegihey cosmicsound08:04
*** Abdallahyas has joined #openstack-kolla08:09
*** abdysn has quit IRC08:12
*** shyamb has quit IRC08:16
*** shyamb has joined #openstack-kolla08:17
hrwrpittau: sorry08:20
mgoddardmorning08:25
*** priteau has joined #openstack-kolla08:32
priteauGood morning Kolla folks. Please review a couple of patches which will help unbreak kayobe CI:08:34
priteauhttps://review.opendev.org/#/c/717276/08:34
patchbotpatch 717276 - kolla-ansible (stable/queens) - Use alternative mirrors for legacy Docker packages - 6 patch sets08:34
priteauhttps://review.opendev.org/#/c/717302/08:34
patchbotpatch 717302 - kolla-ansible (stable/queens) - CI: Use upper constraints to install clients - 7 patch sets08:34
hrwred red red all the way...08:37
priteauThere are two separate issues, the second patch is also needed to make kolla-ansible-centos-source green08:39
priteauubuntu images have other issues I haven't looked into. We don't need it for Kayobe.08:40
hrwImportError: No module named cryptography.hazmat.backends08:41
hrwouch08:42
priteauSince ubuntu jobs are non-voting as well, they could be fixed in another patch08:44
*** zhanglong has quit IRC08:46
hrwpriteau: https://review.opendev.org/#/c/688745/1/tests/pre.yml may fix that crypto part08:49
patchbotpatch 688745 - kolla-ansible (stable/rocky) - Fix CI failures (MERGED) - 1 patch set08:49
hrwpriteau: so bindep will be run and libffi-dev installed08:49
*** ykarel is now known as ykarel|lunch08:50
hrwignore me. looks like wheel got used. I forgot that its on x8608:51
*** abdysn has joined #openstack-kolla08:57
*** Abdallahyas has quit IRC08:59
*** k_mouza has joined #openstack-kolla09:04
*** tonythomas has joined #openstack-kolla09:06
cosmicsoundanyone knows how do i set the tag to latest version of horizon dashboard?09:07
hrwok. rocky/stein/train/master images got published today. hope that today's recheck series will finally pass09:11
hrwcause it gets depressing. one CI failure takes a week to fix09:12
*** e0ne has quit IRC09:13
*** e0ne has joined #openstack-kolla09:14
*** shyamb has quit IRC09:21
*** shyamb has joined #openstack-kolla09:22
*** vishalmanchanda has joined #openstack-kolla09:24
brtknrhello ka folks, what is the variable for modifying admin and public endpoint for services?09:31
openstackgerritPiotr Rabiega proposed openstack/kolla stable/train: add dpdk_telemetry and logparser, use centos-release-opstools for C8  https://review.opendev.org/71769709:33
priteaubrtknr: Can you be more precise? How do you want to modify them?09:34
brtknrSo I want to use a particular interface for public and admin endpoints09:35
brtknrat the moment, all public, admin and internal are using the same endpoints09:35
priteauYou're looking for this? https://docs.openstack.org/kolla-ansible/latest/admin/production-architecture-guide.html#interface-configuration09:36
mgoddardbrtknr: why do you want to use separate networks?09:36
*** ykarel|lunch is now known as ykarel09:36
*** pbing19 has quit IRC09:36
brtknrmgoddard: I want keystone service to be exposed publicly so that it can be used by a standalone monasca deployment09:37
mgoddardbrtknr: ok, sounds complicated09:38
brtknrmgoddard: my current guess is kolla_external_vip_interface is my friend09:38
mgoddardbrtknr: in CI we setup a VXLAN overlay network between multinode hosts09:38
mgoddardthat is one way to connect the bridges together09:38
mgoddardalternatively you could use NAT, but that might end up a bit weird09:39
hrwbrtknr: one auth center for two separate deployments?09:39
brtknrhrw: yeah thats right09:40
cosmicsoundbrtknr , maybe only setting up services manually ? openstack endpoint set \09:40
mgoddardbrtknr: https://opendev.org/openstack/kolla-ansible/src/branch/master/tests/pre.yml#L909:40
brtknrmgoddard: that sounds more complicated than what I'm trying to do lol09:40
mgoddardbrtknr: what you are trying to do is complicated09:41
mgoddardbrtknr: I suppose another option is to set the node's public IP as your public interface?09:42
mgoddardis that what you were asking originally?09:42
brtknrmgoddard: yes!09:42
mgoddardok09:42
brtknris that a more complicated option?09:42
mgoddardit gets tricky with HAProxy as your cloud provider will probably filter out the VIP09:42
cosmicsoundmgoddard , i got issue when not using haproxy my external vip wont work to link to my local.09:43
mgoddardso you'd need some kind of NAT to access the VIP from another host09:43
cosmicsoundso i have to keep haproxy running to use both external and internal09:43
mgoddardcosmicsound: you can't use a VIP without HAProxy09:44
cosmicsoundright09:44
cosmicsoundand how can we expose openstack publicly without external vip09:44
brtknrmgoddard: am I on the right track with kolla_external_vip_interface?09:45
cosmicsoundbrtknr , if you want to allow external ip to services yes09:45
mgoddardcosmicsound: you have to just use a single internal IP for everything without haproxy09:45
cosmicsoundso no external09:46
mgoddardbrtknr: in kayobe it's public_net_name / public_interface09:46
mgoddardbrtknr: but you will hit the problems I've described09:46
mgoddardbrtknr: seriously I think we can setup a tunnel quickly09:46
mgoddardsudo ip link add vxlan0 type vxlan id 10000 local <IP of bond0> dstport 479009:48
mgoddardsudo ip link set vxlan0 mtu <bond0 MTU - 50>09:49
mgoddardsudo bridge fdb append 00:00:00:00:00:00 dev vxlan0 dst <IP of other node's bond0>09:50
mgoddardsudo ip link set vxlan0 up09:50
mgoddardsudo ip link set vxlan0 master breth109:50
mgoddardbrtknr: ^ done09:50
mgoddardbrtknr: make sure each host has a different IP on breth109:51
mgoddardthen try pinging between the breth1 IPs09:51
mgoddard(you have to do that on each node)09:51
born2bakeHi, I am trying to add Swift(ceph rgw) to my openstack cluster. https://docs.ceph.com/docs/luminous/radosgw/keystone/ - I ve added rgw-keystone to my ceph.conf files, restarted rgw, created service, created endpoints, I can see object store tab appeared in horizon but it cant be used, cant retrieve data. I ve opened API access tab, "Object Store - " it cant see swift url I  ve added. I can see swift in services, in endpoint list its09:52
born2bakeenabled as well.09:52
cosmicsoundborn2bake , you run reconfigure?09:54
cosmicsoundi had to run reconfigure to have it seen in horizon09:54
cosmicsoundthen its visible09:54
born2bakeno, I didnt.09:54
born2bakeit appeared itself after some time09:54
cosmicsoundgive it a try09:54
cosmicsoundok09:54
watersjHello, Does masakari have nova-compute (hostmonitoring) in it? I have instance HA, where I can kill -9 qemu instance and it comes back, but if the host is killed, nothing09:57
hrwcurl: (22) The requested URL returned error: 503 Service Unavailable10:01
hrwegy3e8o7t435to784to;3oT^@{"$%2%!#%$@#^%@%@10:01
hrwI think that it is time for me to read more about smartnic stuff and move10:01
hrwopenstack starts to be unfixable mess10:02
hrw4 branches, 3 distros, 2 install types, 2 projects using gazillion of other projects.10:02
hrwkolla's recipe for doom10:02
hrwwhen something breaks in stein then stein and train are doomed. we have to fix stein, wait 1-2 days, fix train, wait 1-2 days. whole development is stuck at gerrit and everyone goes to fight with fire.10:04
hrwwhen train is doomed then master may or may not be10:05
*** shyamb has quit IRC10:06
mgoddardhrw: we're an integration project, pulling hundreds of things together. Things break10:10
mgoddardhrw: what do propose we should do to improve it?10:10
*** cah_link has quit IRC10:11
hrwmgoddard: I am out of ideas for any improvements10:13
*** cah_link has joined #openstack-kolla10:13
hrwamount of chained fuckups gets me closer to burnout then usual10:14
mgoddardhrw: we have ideas about primary/secondary/tertiary images, so that we can ignore failure of tertiary images10:15
mgoddardhrw: we just need someone to implement it10:15
mgoddardhrw: master branch has been pretty stable, and that is where our development is done10:16
hrwmgoddard: to get some changes into master I/we need working train. which needs working stein. which needs working rocky.10:17
*** shyamb has joined #openstack-kolla10:17
mgoddardhrw: not true. branches only depend on the previous release10:17
mgoddardhrw: master is working fine10:17
mgoddardtrain is only blocked on upgrade due to stein10:18
hrwmgoddard: https://review.opendev.org/713854 needs train images and https://review.opendev.org/716574 to land in train.10:18
patchbotpatch 713854 - kolla-ansible - CI: add Debian source upgrade job on AArch64 - 3 patch sets10:18
patchbotpatch 716574 - kolla (stable/train) - CI: Use network mode = host - 2 patch sets10:18
hrwkolla-ansible-ubuntu-source-upgrade in train is broken10:19
mgoddardok, but that's only blocked because you're patching train10:19
mgoddardanyway10:20
mgoddardI hear your point. We spend a lot of time fighting CI fires10:20
mgoddardand yes it's a PITA10:20
hrwI am out of ideas for any development on master10:20
mgoddardhrw: we have ideas. If you have time to implement them, that would be great10:21
hrwand with release being so close I wonder how many months/weeks we will be delayed with ussuri release10:21
hrwmgoddard: any new ideas should be now postponed to Vicky10:21
hrwor whatever V was named10:22
mgoddardyes, but they can still be worked on10:22
hrwwe need to get master shiny and ready. without being loaded with work on older releases10:22
hrwand I have kevinz and xinliang working on rocky->stein->train upgrade so I want train->ussuri upgrade job working on aarch64 so next time they will get something rather working than maybe working10:23
brtknrmgoddard: so same on the other host?10:24
mgoddardbrtknr: yes, but swap IPs around10:24
hrwanyway, I have some online training to finish so end of kolla's work for me today10:24
hrwINFO:kolla.common.utils.cinder-volume:1mhttp://mirror.iad.rax.openstack.org/centos/7/cloud/x86_64/openstack-train/Packages/p/python2-cinderclient-5.0.1-1.el7.noarch.rpm: [Errno 14] HTTP Error 404 - Not Found10:25
hrwheh. infra out of sync10:25
mgoddardhrw: let me know if you want to talk about potential improvements10:28
hrwmgoddard: I got proposals to move to work on IoT/Edge or SmartNICs.10:29
brtknrmgoddard: ok they seem to be talking to each other10:29
brtknrbut how do i verify them via tcpdump?10:29
mgoddardbrtknr: great :)10:29
brtknrtcpdump -n icmp?10:30
mgoddardbrtknr: why tcpdump?10:30
brtknronly thing i could think of10:30
brtknras in, if i set the interface down, i cant ping the other node10:30
yoctozeptohey fellow kollars, I will most likely (as in: I hope) be able to wear my firefighting equipment later, most probably this evening10:31
*** sean-k-mooney has joined #openstack-kolla10:34
*** hjensas|afk is now known as hjensas10:55
watersjwhat generates COMPUTE_HOST failure notification?11:01
*** shyamb has quit IRC11:02
hrwok, so who has any idea why horizon failes in train->stein upgrade jobs?11:11
mgoddardhrw: not sure. Just pulling down a stein horizon image11:13
mgoddardhrw: docker logs look fine. No horizon logs though. HAProxy says 50311:13
watersjIf I manually create COMPUTE_HOST notification, masakari goes ahead and evacuates the instance and powers it on. Is haproxy supposed to create notification when host goes down?11:15
hrwmgoddard: that's the problem11:15
hrwanyway... k-a in stein is borned11:15
hrwhttps://review.opendev.org/#/c/717259 got another recheck11:16
patchbotpatch 717259 - kolla-ansible (stable/stein) - CI: Use upper constraints to install clients - 3 patch sets11:16
hrwuntil it gets in stein is dead11:16
hrwmgoddard: when do you plan to make new stein/train releases?11:17
*** shyamb has joined #openstack-kolla11:17
mgoddardhrw: no plan currently but can do if someone needs them11:17
mgoddardhrw: also, doesn't have to be me11:18
hrwmgoddard: I think that once we get them green on ci then would be good to release11:18
mgoddardsure11:18
hrwas they will have 'use u-c' patch in every branch11:18
hrwit would cut potencial support question11:19
hrws11:19
mgoddard+111:19
*** zhanglong has joined #openstack-kolla11:25
mgoddardhrw: managed to run stein horizon locally. Let's see what zuul says11:28
mgoddardalthough train jobs have already failed11:29
*** JamesBenson has joined #openstack-kolla11:36
mgoddardlooks like our disk is filling up in the centos8 ceph-ansible upgrade job: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_dc0/694799/36/check/kolla-ansible-centos8-source-upgrade-ceph-ansible/dc059bb/primary/logs/system_logs/df.txt11:39
*** shyamb has quit IRC11:44
hrwhttps://review.opendev.org/#/c/717259/ - k-a-ubuntu-{binary,source} look like not even started horizon...11:51
patchbotpatch 717259 - kolla-ansible (stable/stein) - CI: Use upper constraints to install clients - 3 patch sets11:51
openstackgerritMark Goddard proposed openstack/kolla stable/rocky: Make deploy jobs voting again  https://review.opendev.org/71772611:52
hrw+211:53
mgoddardhrw: https://9b719019225fc3f1a633-9c3ff8aa7dcabcee88b15fa3c77a7faa.ssl.cf2.rackcdn.com/717259/3/check/kolla-ansible-ubuntu-source/afd3ae4/primary/logs/docker_logs/horizon.txt11:53
mgoddardhrw: although, now I look at that log I can see it is still compressing static files11:53
mgoddardhrw: and when I started the container locally it took AGES to compress11:54
mgoddardhrw: so maybe we need to wait longer?11:54
*** shyamb has joined #openstack-kolla11:56
mgoddardhrw: took over a minute on my laptop, probably longer in CI11:57
mgoddardhrw: :o using 100% CPU11:57
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: CI: Use upper constraints to install clients  https://review.opendev.org/71725911:57
mgoddard^ bumped horizon timeout to 300s, let's see how it goes11:58
hrwmgoddard: why we not compress that during build...12:00
*** gfidente has joined #openstack-kolla12:07
*** shyam89 has joined #openstack-kolla12:11
openstackgerritDincer Celik proposed openstack/kolla master: Upgrade RabbitMQ to 3.8  https://review.opendev.org/69479912:14
*** shyamb has quit IRC12:15
*** priteau has quit IRC12:35
*** shyam89 has quit IRC12:43
*** Luzi has joined #openstack-kolla13:01
mgoddardhrw: good question13:04
mgoddardhrw: after 7 minutes container uptime, horizon still compressing:13:04
mgoddardroot     31399 31333 31399 99.1  3.7 336828 302972 /var/lib/kolla/venv/bin/python /var/lib/kolla/venv/bin/manage.py compress --force13:04
mgoddardhrw: maybe due to runtime customisation we need to compress at runtime?13:05
*** ykarel is now known as ykarel|afk13:11
*** jimcrowleyibm has joined #openstack-kolla13:33
*** also_stingrayza is now known as stingrayza13:39
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: CI: Use upper constraints to install clients  https://review.opendev.org/71725913:39
hrwmgoddard: maybe add 10s delay between attempts?13:41
mgoddardhrw: already do13:42
hrwops, right ;d13:42
mgoddardgeneralfuzz: around?13:43
*** Blinkiz has joined #openstack-kolla13:53
*** zhanglong has quit IRC13:54
*** TrevorV has joined #openstack-kolla13:56
*** Blinkiz has quit IRC13:57
*** cdearborn has joined #openstack-kolla14:00
*** Blinkiz has joined #openstack-kolla14:01
*** lennyb has quit IRC14:04
*** ykarel|afk is now known as ykarel14:06
*** Luzi has quit IRC14:15
*** gary_perkins has quit IRC14:30
*** gary_perkins has joined #openstack-kolla14:32
born2bakeHi, I am trying to add Swift(ceph rgw) to my openstack cluster. I ve added rgw-keystone to my ceph.conf files, restarted rgw, created service, created endpoints, I can see object store tab appeared in horizon but it cant be used - https://pastebin.com/qKJ83QM9 AccessDenied error14:33
born2bakeany ideas on that one?14:33
watersjnot familiar with swift but to get external ceph working I needed to export my ceph.client.cinder.keyring and like keys into their corresponding /etc/kolla/config/[cinder,nova,...] directories14:42
*** bengates_ has joined #openstack-kolla14:46
*** bengates has quit IRC14:47
kplantborn2bake: what did you use for the endpoints14:54
kplanti have ceph-rgw backed swift working in mine14:54
born2bakeopenstack endpoint create --region RegionOne swift public http://10.0.2.1:8080/swift/v114:55
born2bakeopenstack service create --name=swift \14:55
born2bake                         --description="Swift Service" \14:55
born2bake                         object-store14:55
born2bakecurl http://10.0.2.1:808014:55
born2bake<?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>anonymous</ID><DisplayName></DisplayName></Owner><Buckets></Buckets></ListAllMyBucketsResult>14:55
kplantah14:56
born2bakecurl http://10.0.2.1:8080/swift/v1 - NoSuchKey14:56
kplantyou need /swift/v1/AUTH_$(tenant_id)s14:56
born2bakeneed that where? in endpoints?14:56
kplantyes14:56
born2bakedelete my previous one first right?14:56
kplantyes14:56
born2bakethank you mate14:56
kplantyw14:57
kplantlet me know how it goes14:57
noxoidive always been curious why AUTH_ is needed. it works fine for me without it14:58
kplantdepends on your ceph config14:58
kplantthere's a line to use the keystone tenant id as the ceph "user"14:59
openstackgerritMark Goddard proposed openstack/kolla stable/stein: DNM: Test horizon & pyScss==1.3.4 compress  https://review.opendev.org/71777615:00
noxoidrgw_keystone_implicit_tenants ?15:00
noxoidor rgw_swift_account_in_url15:00
born2bakeI dont have it "keystone tenant id" in my ceph.conf15:01
born2bakerunning reconfigure, will see how it goes15:01
mgoddardhrw: horizon works with a timeout of 1000s!15:03
mgoddardlooks like pyScss==1.3.7 might be the culprit15:03
cosmicsoundwhat pools you folks have for rgw in ceph for swift15:04
hrwmgoddard: o!15:11
hrwmgoddard: for py2 1.3.4 is fine15:11
hrwmgoddard: py3 will fetch setuptools 46+ so 1.3.7 is needed15:11
mgoddardhrw: hmm, maybe 1.3.7 doesn't work well with py2?15:12
mgoddardbecause we don't seem to hit this on master15:12
hrwmgoddard: master uses 1.3.715:13
hrwmgoddard: horizon in master uses 1.3.7, in train iirc too15:14
mgoddardhrw: right, but also py315:14
hrwmgoddard: pyscss 1.3.6 was '1.3.5 + setuptools v46+', 1.3.7 was 'let build without compiler'15:14
mgoddardhrw: centos7 train image has 1.3.415:14
hrwmgoddard: so 1.3.7 can be used with py2 but horizon in stein- stick to 1.3.415:15
mgoddardhrw: 1.3.7 is already in stein upper-constraints15:15
*** tonythomas has quit IRC15:15
hrwo.15:16
*** abdysn has quit IRC15:16
hrwstill fine with py2/py315:16
hrwmgoddard: ok, I know then what we may need to fix to get 1.3.7 pyscss faster. have to check logs15:17
hrwINFO:kolla.common.utils.horizon:  Created wheel for pyScss: filename=pyScss-1.3.7-cp27-cp27mu-linux_x86_64.whl15:19
hrwso it built that faster part15:19
born2bakekplant didnt help :/15:22
kplantcan i see your ceph rgw config?15:24
kplantthe "rgw keystone *" lines15:26
*** ykarel is now known as ykarel|away15:28
noxoidcosmicsound: if you give the radosgw daemon a key with the write mon permission (iirc) itll create the pools necessary itself15:29
noxoidunless you're talking about if we use erasure/replication15:30
openstackgerritMerged openstack/kolla stable/rocky: Make deploy jobs voting again  https://review.opendev.org/71772615:31
*** vishalmanchanda has quit IRC15:33
cosmicsoundnoxoid , i was thinnking i miss some pools, i was able to create a bucker, yet it failed when tried to create a folder in it15:33
cosmicsoundhttps://mdb.uhlhost.net/uploads/078780d235c83ae4/image.png15:34
noxoidmay want to try creating it from the cli and see what error its actually returning15:35
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/train: CI: Bump dashboard access timeout to 1000s  https://review.opendev.org/71778215:37
cosmicsoundnoxoid , will try that15:46
openstackgerritMark Goddard proposed openstack/kayobe master: Add kolla_enable_openstack_core variable  https://review.opendev.org/71778515:49
openstackgerritMark Goddard proposed openstack/kolla stable/stein: DNM: Test horizon & pyScss==1.3.4 compress  https://review.opendev.org/71777615:50
mgoddardhrw, yoctozepto, osmanlicilegi: could use a review of https://review.opendev.org/#/c/717276 which blocks kayobe CI15:51
patchbotpatch 717276 - kolla-ansible (stable/queens) - Use alternative mirrors for legacy Docker packages - 6 patch sets15:51
mgoddardhrw, yoctozepto, osmanlicilegi: also https://review.opendev.org/#/c/717259 will unblock stein15:53
patchbotpatch 717259 - kolla-ansible (stable/stein) - CI: Use upper constraints to install clients - 5 patch sets15:53
mgoddardand https://review.opendev.org/717782 should unblock train15:53
patchbotpatch 717782 - kolla-ansible (stable/train) - CI: Bump dashboard access timeout to 1000s - 1 patch set15:53
*** tonythomas has joined #openstack-kolla16:00
osmanlicilegimgoddard: done16:02
mgoddardthanks osmanlicilegi16:03
*** rpittau is now known as rpittau|afk16:08
openstackgerritMerged openstack/kolla-ansible stable/queens: Use alternative mirrors for legacy Docker packages  https://review.opendev.org/71727616:14
openstackgerritMerged openstack/kolla-ansible stable/queens: CI: Use upper constraints to install clients  https://review.opendev.org/71730216:14
openstackgerritPiotr Rabiega proposed openstack/kolla stable/train: add dpdk_telemetry and logparser for C8, use centos-release-opstools for C8  https://review.opendev.org/71769716:27
openstackgerritMark Goddard proposed openstack/kolla master: Add docs and release note for CentOS 8  https://review.opendev.org/71779616:30
openstackgerritMark Goddard proposed openstack/kayobe master: Add kolla_enable_openstack_core variable  https://review.opendev.org/71778516:32
*** evrardjp has quit IRC16:36
*** evrardjp has joined #openstack-kolla16:37
openstackgerritMark Goddard proposed openstack/kolla master: Add docs and release note for CentOS 8  https://review.opendev.org/71779616:37
yoctozeptomgoddard: so stein has newer pyscss installed?16:39
mgoddardyoctozepto: yep!16:39
yoctozeptomgoddard: woot16:39
yoctozepto;D16:39
mgoddardwell thankfully it did or I might not have found a diff16:40
yoctozeptowhatever, deprecate horizon16:40
kplantlol16:41
yoctozeptokplant: what? it's clumsy16:41
kplanti agree16:41
kplanti don't even know what's good for web anymore16:44
kplantbut it's definitely not django16:44
*** k_mouza has quit IRC16:45
*** skramaja has quit IRC16:52
openstackgerritPedro Henrique Pereira Martins proposed openstack/kolla-ansible master: Add suport to OpenID Connect Authentication flow  https://review.opendev.org/69543216:54
*** k_mouza has joined #openstack-kolla16:57
yoctozeptokplant: django or not, openstack begs to be consumed via browser-side requests to services APIs, not clumsy middleman16:58
*** e0ne has quit IRC16:59
openstackgerritMark Goddard proposed openstack/kolla stable/train: Add docs and release note for CentOS 8  https://review.opendev.org/71780717:04
kplantthen horizon could turn into a little turd service that just fires js to the client17:07
kplantis that what you're saying?17:07
mgoddardhrw, yoctozepto: verified, PyScss 1.3.4 in stein reverts to sane compression time: https://ac68d2db0e7c0ff52223-7f8a4c585e620728c399ab8dbffa7ef2.ssl.cf2.rackcdn.com/717776/2/check/kolla-ansible-centos-source/2272ca9/primary/logs/ansible/test-openstack17:07
*** k_mouza has quit IRC17:07
mgoddardtested via https://review.opendev.org/71777617:08
patchbotpatch 717776 - kolla (stable/stein) - DNM: Test horizon & pyScss==1.3.4 compress - 2 patch sets17:08
*** bengates has joined #openstack-kolla17:10
*** bengates_ has quit IRC17:14
hrwmgoddard: can you create an issue on https://github.com/Kronuz/pyScss tracker?17:17
* hrw off17:18
mgoddardhrw: https://github.com/Kronuz/pyScss/issues/39817:19
mgoddardjust linked to our issue17:19
yoctozeptokplant: another spin-off project but yeah17:19
kplantthat would be great17:20
kplantis there any function that _needs_ to hit the admin endpoint?17:21
*** k_mouza has joined #openstack-kolla17:23
*** k_mouza has quit IRC17:27
*** bengates has quit IRC17:29
*** ktibi has quit IRC17:34
openstackgerritMerged openstack/kolla-ansible stable/stein: CI: Use upper constraints to install clients  https://review.opendev.org/71725917:43
born2bakecosmicsound object store disappeared in horizon for me :) what you did to make it back?17:44
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Add support for encrypting backend Keystone HAProxy traffic  https://review.opendev.org/71398617:48
cosmicsoundborn2bake , well until now nothing :D17:49
cosmicsoundsince it never dissapeared yet17:49
cosmicsoundjust check make sure you have properly made the service urls17:50
cosmicsoundpublic admin internal17:50
yoctozeptomgoddard: wdyt regarding zun cni as highlight?17:57
*** k_mouza has joined #openstack-kolla17:58
*** crindi_ has joined #openstack-kolla18:04
openstackgerritRadosÅ‚aw Piliszek proposed openstack/kolla-ansible stable/train: Fix ovs fw driver for the other ovs agent  https://review.opendev.org/71617518:05
*** crindi has quit IRC18:08
*** thatcher has quit IRC18:08
*** crindi_ is now known as crindi18:08
*** admin0 has quit IRC18:09
*** k_mouza has quit IRC18:16
*** diga has quit IRC18:19
*** gfidente is now known as gfidente|afk18:42
*** kukacz has quit IRC18:55
*** kukacz has joined #openstack-kolla18:57
openstackgerritRadosÅ‚aw Piliszek proposed openstack/kolla stable/train: add dpdk_telemetry and logparser for C8, use centos-release-opstools for C8  https://review.opendev.org/71769718:58
openstackgerritMerged openstack/kolla-ansible stable/train: CI: Bump dashboard access timeout to 1000s  https://review.opendev.org/71778218:59
openstackgerritMerged openstack/kolla master: Upgrade RabbitMQ to 3.8  https://review.opendev.org/69479919:16
*** dking_desktop has quit IRC19:20
born2bakekplant nah, its still failing.. tried everything19:24
born2bakehttps://pastebin.com/QrUWPCNw19:24
born2bakesame thing19:24
kplantthat's so weird19:25
kplantcan i get your output from `openstack endpoint list --service swift` ?19:25
cosmicsoundhow do i create buckets via cli19:26
cosmicsoundin swift19:26
kplantit's a container19:26
kplantopenstack container create19:26
born2bakehttp://paste.openstack.org/show/791700/19:27
kplantyou're missing AUTH_$(tenant_id)s19:27
kplantyou can hit the endpoint manually19:27
kplantget the uuid of your admin project19:27
kplantand plug it in yourself with curl19:27
kplantshould work19:27
born2bakeI had it before when I ve added those tenant things and it didnt work...so I decided to revert it19:28
born2bakeeven without tenant you can see in the log -  https://pastebin.com/QrUWPCNw19:28
born2bakeit goest the same way without swift url = true19:28
kplantis your rgw endpoint reachable from keystone?19:28
kplantmore importantly, is keystone reachable from your rgw(s)19:29
born2bakecurl swift/v1 - 404, curl swift/v1/AUTH_tenantid - HTTP/1.1 301 Moved Permanently19:30
born2bake<kplant> is your rgw endpoint reachable from keystone? - how do I check that? from keystone container?19:31
kplantyou can check from the host the container is on19:31
kplanthttp://paste.openstack.org/show/pxgWwjK8f8mcrZSarC9Y/19:32
kplanti have :8443 as a service in haproxy for https and loadbalancing19:32
kplantother than that, it's the same19:32
born2bakeI can curl my rgw from keystone container, from host19:35
born2bakeand I can see requests in rgw logs.19:36
kplantwith your accepted roles only being "admin"19:36
kplanti assume the user you're trynig with has the admin role for that project?19:36
born2bake. /etc/kolla/admin-(post-deploy-script)19:37
born2bakeso I ve added same admin and password in rgw keystone19:38
*** keper7 has joined #openstack-kolla19:40
kplanti don't know what else to say really, i just created the swift service, those three endpoints19:43
kplantconfigured the rgws19:43
kplantand it worked19:43
kplantthen i made it fancy with haproxy19:43
born2bakewhat output do you have when you curl your swift?19:57
openstackgerritDincer Celik proposed openstack/kolla master: Upgrade Elasticsearch and Kibana to 6.x  https://review.opendev.org/64774820:07
openstackgerritDincer Celik proposed openstack/kolla master: Follow up on RabbitMQ 3.8 upgrade  https://review.opendev.org/71784520:14
bjolo_mgoddard: i just saw you mention centos8 containters for train 9.1 release. when is that release coming?20:17
yoctozeptobjolo_: "soon" :-) but you can grab kolla from stable/train branch as it's quite... well... stable :-)20:22
*** cah_link has quit IRC20:42
*** e0ne has joined #openstack-kolla20:50
openstackgerritJason Anderson proposed openstack/kolla-ansible master: [elasticsearch] Update config for 6.x  https://review.opendev.org/70962420:59
openstackgerritJason Anderson proposed openstack/kolla-ansible master: [elasticsearch] Add migration for Kibana 6.x index  https://review.opendev.org/70962520:59
openstackgerritPierre Riteau proposed openstack/kayobe stable/queens: Fix multiple CI failures  https://review.opendev.org/71720821:00
openstackgerritJason Anderson proposed openstack/kolla-ansible master: [elasticsearch] Add migration for Kibana 6.x index  https://review.opendev.org/70962521:09
*** e0ne has quit IRC21:16
*** JamesBenson has quit IRC22:00
*** k_mouza has joined #openstack-kolla22:16
*** born2bake has quit IRC22:17
*** k_mouza has quit IRC22:21
*** TrevorV has quit IRC22:24
*** k_mouza has joined #openstack-kolla22:46
*** k_mouza has quit IRC22:47
*** k_mouza has joined #openstack-kolla22:48
*** k_mouza has quit IRC22:52
*** jimcrowleyibm has quit IRC23:05

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!