Friday, 2018-09-28

*** strigazi has quit IRC00:01
*** vabada has quit IRC00:01
*** strigazi has joined #openstack-kolla00:02
*** vabada has joined #openstack-kolla00:02
*** JamesBenson has quit IRC00:10
*** hamzy has joined #openstack-kolla00:25
*** dpascheka has quit IRC00:28
*** shardy has quit IRC01:10
*** shardy has joined #openstack-kolla01:18
*** goldyfruit has quit IRC01:40
*** itlinux has joined #openstack-kolla01:44
*** itlinux has quit IRC01:44
*** zzzeek has quit IRC01:48
*** zzzeek has joined #openstack-kolla01:49
openstackgerritChason Chan proposed openstack/kolla master: Remove '.. end' comments  https://review.openstack.org/60590302:01
openstackgerritChason Chan proposed openstack/kolla-ansible master: Remove '.. end' comments  https://review.openstack.org/60590602:18
*** hongbin has joined #openstack-kolla02:20
*** unicell has joined #openstack-kolla02:35
*** unicell has quit IRC02:40
*** jongwooh has joined #openstack-kolla02:53
*** skramaja has joined #openstack-kolla02:53
*** unicell has joined #openstack-kolla02:57
*** unicell has quit IRC02:57
*** donghm_ has joined #openstack-kolla03:03
*** jongwooh has quit IRC03:24
*** hongbin has quit IRC03:38
spsuryamorning guys04:19
donghmspsurya, good morning04:20
*** negronjl has joined #openstack-kolla04:37
*** itlinux has joined #openstack-kolla05:12
*** gfidente has joined #openstack-kolla06:07
*** pcaruana has joined #openstack-kolla06:11
*** negronjl has quit IRC06:53
*** Luzi has joined #openstack-kolla06:53
*** negronjl has joined #openstack-kolla06:55
*** chandankumar has joined #openstack-kolla06:56
*** shardy has quit IRC07:01
*** shardy has joined #openstack-kolla07:02
*** gkadam has joined #openstack-kolla07:23
*** negronjl has quit IRC07:28
*** mattgo has joined #openstack-kolla07:28
openstackgerritXinliang Liu proposed openstack/kolla master: Add kolla-build-debian-source-arm64 experimental job  https://review.openstack.org/55765907:32
*** negronjl has joined #openstack-kolla07:32
*** hamzaachi has joined #openstack-kolla07:36
openstackgerritMerged openstack/kolla-ansible master: Remove '.. end' comments  https://review.openstack.org/60590607:38
*** negronjl has quit IRC07:41
openstackgerritMerged openstack/kolla master: Remove '.. end' comments  https://review.openstack.org/60590307:41
*** donghm has quit IRC07:43
*** ducnv_ has quit IRC07:43
*** hoangcx has quit IRC07:43
*** ducnv has joined #openstack-kolla07:43
*** negronjl has joined #openstack-kolla07:45
*** ducnv has quit IRC07:52
*** ducnv has joined #openstack-kolla07:55
*** hoangcx has joined #openstack-kolla07:57
*** ktibi has joined #openstack-kolla08:01
*** dougsz has joined #openstack-kolla08:05
Tengumandre: heya! I saw you commented out one of my patches about the modprobe thingy in kolla containers - care to provide a feedback on that one? https://review.openstack.org/#/c/605624/  - it's the kolla-ansible patch :). Note: not finished, need to integrate all the modules.08:05
*** donghm has joined #openstack-kolla08:06
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Don't use 'become' in CI tests  https://review.openstack.org/60569308:16
*** mgoddard has joined #openstack-kolla08:16
*** serlex has joined #openstack-kolla08:16
openstackgerritDoug Szumski proposed openstack/kolla master: Add Monasca template override file  https://review.openstack.org/60575908:17
egonzalezTengu also replied, i'm a bit worried about removing the modprobe to other projects using k8s or an orchestrator engine08:21
Tenguegonzalez: juist saw your comment, preparing an update.08:22
xinlianggrowroot still not working for arm64 node: http://logs.openstack.org/59/557659/25/experimental/kolla-build-debian-source-arm64/ff00a12/job-output.txt.gz#_2018-09-28_07_57_15_53952808:22
Tenguegonzalez: well, to be honnest, modprobe should *never* be done within containers as this action requires really elevated privileges. This works only because selinux separation is disabled in docker....08:22
Tenguegonzalez: for tripleo, we must still support that, but only for 3rd parties, meaning we can limit the impact to a few containers. As a first step. next one would be to do the things in a more secure way, without involving lower selinux settings and the like.08:23
openstackgerritWill Szumski proposed openstack/kolla-ansible master: Use iptables as section header instead of firewall  https://review.openstack.org/60599508:24
mnasiadkamorning08:26
spsuryasup mnasiadka08:27
mnasiadkaLast day of work before 2 weeks long vacation :)08:27
spsuryanice enjoy !!08:27
mnasiadkaI will, first long vacation this year ;)08:28
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: WIP: Add Monasca documentation  https://review.openstack.org/60575108:28
Tenguegonzalez: in addition - according to some side discussions I read, modprobe should be launched by infra containers only. Orchestrator would hence manage that properly.08:30
Tengumgoddard: small thought regarding the "become": is it really, really needed? because according to the doc, it's launched with "sudo kolla-ansible deploy", hence already root... ?08:44
egonzalezTengu not in the remote nodes, unless the user to ssh is root08:45
Tenguoh, ok.08:45
Tengulet's patch that then :).08:45
Tengualso adding support for module unloading. not a big deal.08:45
Tengumight be useful. or not. but it's there. Testing it now before pushing.08:46
*** shyamb has joined #openstack-kolla08:52
*** rgogunskiy has joined #openstack-kolla08:54
*** hamzaachi has quit IRC08:58
openstackgerritCédric Jeanneret proposed openstack/kolla-ansible master: Load known, standard modules from the host, not within containers  https://review.openstack.org/60562409:14
Tenguegonzalez, mgoddard -^^  :)09:14
*** pbourke has quit IRC09:24
*** pbourke has joined #openstack-kolla09:25
hrwmorning09:31
hrwhttps://review.openstack.org/#/c/605441/ anyone? egonzalez pbourke spsurya?09:31
openstackgerritMerged openstack/kolla stable/rocky: ceph: stick to luminous on Ubuntu (in proper way)  https://review.openstack.org/60577209:33
openstackgerritMerged openstack/kolla master: base: remove pinning of dh-python for ubuntu  https://review.openstack.org/60560809:36
*** ktibi has quit IRC09:38
*** JamesBenson has joined #openstack-kolla09:43
*** JamesBenson has quit IRC09:47
hrwthanks09:53
*** shyamb has quit IRC09:56
*** JamesBenson has joined #openstack-kolla09:57
*** shyamb has joined #openstack-kolla09:58
*** JamesBenson has quit IRC10:01
*** donghm has quit IRC10:03
openstackgerritCédric Jeanneret proposed openstack/kolla-ansible master: Load known, standard modules from the host, not within containers  https://review.openstack.org/60562410:09
Tenguegonzalez, mgoddard: just added a release-note -^ :). Once the process is OK for both of you, I'll add the 2-3 other modules.10:09
*** xinliang has quit IRC10:16
*** hoangcx has quit IRC10:23
*** shyamb has quit IRC10:27
*** hamzaachi has joined #openstack-kolla10:27
openstackgerritCédric Jeanneret proposed openstack/kolla-ansible master: Load known, standard modules from the host, not within containers  https://review.openstack.org/60562410:41
Tengumgoddard: thanks for your feedback - now I think we're good :).10:41
Tengulemme know if there are other things to change - I've loaded the 3 missing modules as well.10:41
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Don't use 'become' in CI tests  https://review.openstack.org/60569310:49
openstackgerritDavid Rabel proposed openstack/kolla-ansible master: openvswitch: always run handler to to ensure OVS bridges are up  https://review.openstack.org/60603311:01
*** shyamb has joined #openstack-kolla11:01
openstackgerritCédric Jeanneret proposed openstack/kolla-ansible master: Load known, standard modules from the host, not within containers  https://review.openstack.org/60562411:01
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Don't use 'become' in CI tests  https://review.openstack.org/60569311:02
*** hjensas has joined #openstack-kolla11:05
*** xinliang has joined #openstack-kolla11:09
hrwwho is using oraclelinux?11:22
hrwhttp://logs.openstack.org/41/605441/4/gate/kolla-build-oraclelinux-source/85997be/logs/build/000_FAILED_base.txt.gz11:22
egonzalezhrw that should been fixed by https://review.openstack.org/#/c/600328/4/docker/base/Dockerfile.j2 line 4811:29
hrwegonzalez: INFO:kolla.common.utils.base:1mhttps://yum.oracle.com/repo/OracleLinux/OL7/optional/latest/x86_64/repodata/updateinfo.xml.gz: [Errno -1] Metadata file does not match checksum11:30
egonzalezunless is a aarm64 which i havent tested it11:30
openstackgerritDavid Rabel proposed openstack/kolla-ansible master: openvswitch: always run handler to to ensure OVS bridges are up  https://review.openstack.org/60603311:30
hrwgeneric ol7 issue11:30
egonzalezmany times, when is an rpm promotion in the repos, are unabailable until repodata.xml is synced11:31
openstackgerritDavid Rabel proposed openstack/kolla-ansible master: openvswitch: always run handler to to ensure OVS bridges are up  https://review.openstack.org/60603311:31
hrwegonzalez: yep. bad luck11:32
*** hamzaachi has quit IRC11:40
*** shyamb has quit IRC11:41
*** shyamb has joined #openstack-kolla11:41
*** dpascheka has joined #openstack-kolla11:46
*** mgoddard has quit IRC11:54
*** negronjl has quit IRC11:58
*** Bhujay has joined #openstack-kolla11:58
*** mgoddard has joined #openstack-kolla12:00
*** negronjl has joined #openstack-kolla12:00
*** shyamb has quit IRC12:02
*** shyamb has joined #openstack-kolla12:03
*** serlex has quit IRC12:11
*** ducnv has quit IRC12:13
*** rgogunskiy has quit IRC12:22
*** goldyfruit has joined #openstack-kolla12:24
*** skramaja has quit IRC12:26
emccormickHey, I have a deployment with a mariadb container so severely of sync it won't start (it thinks it's ahead but it's not. Kinda weird)12:27
emccormickthink it's safe to delete it and its volume and run mariadb_recovery? should it recreate it?12:28
*** nicolasbock_ has joined #openstack-kolla12:32
*** goldyfruit has quit IRC12:35
*** shyamb has quit IRC12:49
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Don't use 'become' in CI tests  https://review.openstack.org/60569312:53
*** serlex has joined #openstack-kolla12:56
*** jaosorior has quit IRC13:00
*** serlex has quit IRC13:03
*** goldyfruit has joined #openstack-kolla13:22
*** Luzi has quit IRC13:30
*** zzzeek has quit IRC13:43
*** zzzeek has joined #openstack-kolla13:45
mgoddardrm_work: about?13:45
*** zzzeek has quit IRC13:48
*** zzzeek has joined #openstack-kolla13:49
*** mystery_smith has joined #openstack-kolla13:50
*** negronjl has quit IRC13:59
*** negronjl has joined #openstack-kolla14:03
*** JamesBenson has joined #openstack-kolla14:04
*** emccormickva has joined #openstack-kolla14:08
openstackgerritMark Goddard proposed openstack/kolla master: Use rabbitmq package from OS distro for Bifrost  https://review.openstack.org/60609114:10
openstackgerritMark Goddard proposed openstack/kolla-ansible master: DNM: Test bifrost  https://review.openstack.org/60609214:11
*** emccormick has quit IRC14:12
*** edleafe has quit IRC14:29
*** emccormickva has quit IRC14:29
*** emccormick has joined #openstack-kolla14:30
*** edleafe has joined #openstack-kolla14:30
*** mjturek has joined #openstack-kolla14:32
*** emccormickva has joined #openstack-kolla14:34
*** emccormick has quit IRC14:35
*** emccormickva has quit IRC14:36
*** mattgo has quit IRC14:38
*** emccormickva has joined #openstack-kolla14:39
*** gfidente is now known as gfidenteN00b14:46
mystery_smithIs there an easy way to verify rabbitmq is working? I see in several nova logs it's unable to connect to rabbitmq and I get a weird status from rabbitmqctl cluster_status in the rabbitmq container.14:46
goldyfruitmystery_smith: rabbitmqctl status / rabbitmqctl cluster_status14:47
goldyfruitRabbitMQ logs ?14:48
goldyfruitcheck if 5672 port is reachable14:48
mystery_smithgoldyfruit: it tells me status is nodedown and gives me the following error: TCP connection succeeded but Erlang distribution failed14:48
mystery_smithI can check that with netstat / telnet / nc but sadly they're not available in the container - how would I check it otherwise?14:49
goldyfruitfrom outside the container14:49
goldyfruithttps://stackoverflow.com/questions/47874958/rabbitmq-failed-to-start-tcp-connection-succeeded-but-erlang-distribution-faile14:50
mystery_smithConnection refused from outside the container14:51
mystery_smithIs there any way to rerun just the rabbitmq playbooks or is that a viable option I wonder?14:52
*** munimeha1 has joined #openstack-kolla14:54
goldyfruitkolla-ansible reconfigure -t rabbitmq ?14:55
mystery_smithfails on a random mariadb check :-/14:58
*** Bhujay has quit IRC14:58
*** emccormickva has quit IRC15:00
*** dave-mccowan has joined #openstack-kolla15:08
*** dave-mccowan has quit IRC15:13
*** mystery_smith has quit IRC15:15
*** mgoddard has quit IRC15:16
*** mgoddard has joined #openstack-kolla15:47
*** hamzaachi has joined #openstack-kolla15:58
*** genek has joined #openstack-kolla16:00
*** unicell has joined #openstack-kolla16:03
*** unicell has quit IRC16:08
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Don't use 'become' in CI tests  https://review.openstack.org/60569316:16
*** manjeets has quit IRC16:17
*** shyamb has joined #openstack-kolla16:30
*** shyamb has quit IRC16:35
*** shyamb has joined #openstack-kolla16:35
*** hamzaachi has quit IRC16:36
*** gkadam has quit IRC16:42
*** munimeha1 has quit IRC16:43
*** genek has quit IRC16:50
*** genek has joined #openstack-kolla16:51
*** genek has quit IRC16:58
*** dougsz has quit IRC17:00
*** shyamb has quit IRC17:03
*** hamzaachi has joined #openstack-kolla17:07
*** gfidenteN00b has quit IRC17:10
*** goldyfruit has quit IRC17:23
rm_workmgoddard: i only show up about now-o-clock usually T_T17:31
rm_worksorry17:31
mgoddardrm_work: no problem17:33
mgoddardwe saw a haproxy issue in kayobe CI17:33
rm_workok, i can probably help with that17:34
rm_workshoot me the details17:34
rm_workunless you already resolved it17:35
mgoddardhttp://logs.openstack.org/04/568804/19/check/kayobe-overcloud-centos/b1a8bc8/primary/docker_logs/haproxy.txt.gz17:35
rm_workok, i need to afk for 15m but i'll look when i get back17:35
mgoddardsadly we don't have the generated config for haproxy17:35
rm_workhmm errors in memcached cfg? weird17:35
rm_workthats one of the ones i explicitly remember testing17:35
rm_workweird, why not? i see the rest of them there....17:36
rm_workwhy didn't it copy the haproxy folder :/17:36
mgoddardI think those are the input configs, rather than the generated configs17:37
*** genek has joined #openstack-kolla17:39
rm_workthey aren't j2s17:52
rm_workah but maybe?17:52
*** unicell has joined #openstack-kolla17:53
rm_worki'll look at it again later today, i have to drive somewhere presently17:53
rm_workbut yeah it worked fine for me before, and it's hard without seeing the config it generated17:54
rm_workcan we make it hold a node or something? or update the job to dump the configs?17:54
rm_workmaybe even just PRINT the generated haproxy config post-copy17:54
mgoddardrm_work: I'll update it to dump the configs18:00
rm_workthanks, i'll be able to check it in a few hours -- just drop me a link, i'm on a bouncer so i'll get whatever you send me18:01
rm_workyou should get on a bouncer too :P I am always sad when I can't message you because you're offline, lol18:01
rm_workor maybe everything is according to plan ;)18:01
*** TheJulia is now known as needssleep18:02
*** unicell has quit IRC18:02
*** unicell has joined #openstack-kolla18:02
mgoddardrm_work: I have a bouncer (ZNC), but it didn't play well with my old IRC client (hexchat) so I stopped using it18:13
mgoddardrm_work: just switched to weechat, if I like it I'll try the bouncer again18:13
mgoddardrm_work: so far it seems less good at notifying me of messages, but it might just need a config tweak18:14
*** mgoddard has quit IRC18:47
*** hamzaachi has quit IRC18:56
*** hamzaachi has joined #openstack-kolla18:56
*** hamzaachi has quit IRC19:12
*** Chaserjim has joined #openstack-kolla19:16
*** negronjl has quit IRC19:34
*** negronjl has joined #openstack-kolla19:37
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support Octavia custom policy  https://review.openstack.org/60080019:38
*** dpascheka has quit IRC19:42
*** hamzaachi has joined #openstack-kolla19:42
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Support Octavia custom policy  https://review.openstack.org/60080019:44
*** unicell has quit IRC19:46
*** unicell has joined #openstack-kolla19:46
*** unicell has quit IRC19:46
*** unicell has joined #openstack-kolla19:46
*** openstackgerrit has quit IRC20:07
*** Chaserjim has quit IRC20:10
*** negronjl has quit IRC20:13
*** negronjl has joined #openstack-kolla20:15
*** emccormick has joined #openstack-kolla20:18
*** mjturek has quit IRC20:18
*** negronjl has quit IRC20:20
*** negronjl has joined #openstack-kolla20:21
*** negronjl has quit IRC20:30
*** negronjl has joined #openstack-kolla20:32
*** emccormick has quit IRC20:56
*** JamesBen_ has joined #openstack-kolla20:58
*** shardy has quit IRC21:00
*** JamesBenson has quit IRC21:02
*** genek has quit IRC21:05
JamesBen_Can you reconfigure and remove neutron-dvr?21:13
JamesBen_I enabled it and things just aren't working... :-(21:14
*** negronjl has quit IRC21:16
JamesBen_Do you need to enable "enable_neutron_provider_networks" this as well with DVR?21:17
*** negronjl has joined #openstack-kolla21:19
*** mgoddard has joined #openstack-kolla21:21
mgoddardrm_work: here's the haproxy config for memcached: http://logs.openstack.org/04/568804/20/check/kayobe-overcloud-centos/591b447/primary/kolla_node_configs/haproxy/services.d/memcached.cfg21:33
rm_workooo21:34
rm_workok21:34
*** genek has joined #openstack-kolla21:34
mgoddardI think the problem is it's using the inventory_hostname rather than the IP of api_interface21:34
rm_workmgoddard: also, if you do have a bouncer, it's not very good at staying online :P21:34
mgoddardno bouncer yet21:34
rm_workah so another candidate for custom members? :/21:34
rm_workah but that shouldn't cause haproxy to barf21:34
mgoddardit's using custom members21:34
rm_workthat'd start up and just not work21:34
mgoddardI think that's the problem21:35
rm_workoh unless it was blank21:35
rm_workahhh21:35
rm_workor a hostname we don't have?21:35
rm_workyeah >_>21:35
rm_workmine must have worked because the hostname existed21:35
rm_workor something21:35
mgoddardcontroller0 isn't a resolvable hostname, just the inventory hostname21:35
rm_workk, do you know which variable has the IP we need?21:35
rm_workwhy did this pass the regular gates?21:35
rm_workdo they also have the hostnames set?21:36
mgoddardthey must be setting the hostname21:36
rm_worki can propose it really quick so you can +2 it, if you know what var it is that'd help tho21:36
mgoddardneeds to use the same logic as the haproxy-config template21:37
rm_workah i guess matching the other external stuff?21:37
mgoddardi.e. IP address of api_interface21:37
rm_work{{ hostvars[host]['ansible_hostname'] }} {{ hostvars[host]['ansible_' + hostvars[host]['api_interface']]['ipv4']['address'] }}21:37
mgoddard+121:37
mgoddardtime for bed21:38
rm_workkk21:38
rm_worki'll post that now21:38
*** genek has quit IRC21:39
rm_workmgoddard: vv21:39
*** openstackgerrit has joined #openstack-kolla21:39
openstackgerritAdam Harwell proposed openstack/kolla-ansible master: Fix haproxy server config for memcached backends  https://review.openstack.org/60621221:39
rm_workmgoddard: ^^21:39
*** hamzaachi has quit IRC21:43
*** mgoddard has quit IRC21:43
*** bbezak has joined #openstack-kolla21:50
bbezakquiot21:54
*** bbezak has quit IRC21:54
*** pbourke has quit IRC22:56
*** pbourke has joined #openstack-kolla22:56
*** pbourke has quit IRC22:59
*** pbourke has joined #openstack-kolla22:59
*** genek has joined #openstack-kolla23:35
*** genek has quit IRC23:40
*** JamesBen_ has quit IRC23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!