Monday, 2021-05-17

*** zbr has quit IRC00:17
*** zbr has joined #openstack-ansible00:17
*** rh-jlabarre has joined #openstack-ansible01:27
*** rh-jlabarre has quit IRC02:53
*** akahat is now known as akahat|ruck04:26
*** macz_ has joined #openstack-ansible04:53
*** macz_ has quit IRC04:57
*** shyamb has joined #openstack-ansible05:54
*** shyamb has quit IRC05:57
*** shyamb has joined #openstack-ansible05:58
*** macz_ has joined #openstack-ansible06:00
*** macz_ has quit IRC06:05
*** logan- has quit IRC06:12
*** logan- has joined #openstack-ansible06:15
*** shyamb has quit IRC06:42
*** shyamb has joined #openstack-ansible06:42
*** shyamb has quit IRC06:55
*** miloa has joined #openstack-ansible07:03
*** jawad_axd has joined #openstack-ansible07:05
*** miloa has quit IRC07:11
noonedeadpunkmorning07:18
noonedeadpunkmasakari failure for centos is super weird. I believe there is python3-libvirt as rpm package, as we also install it for nova...07:20
*** andrewbonney has joined #openstack-ansible07:21
*** kleini has quit IRC07:22
*** macz_ has joined #openstack-ansible07:26
jrossermorning07:28
*** macz_ has quit IRC07:30
*** tosky has joined #openstack-ansible07:32
openstackgerritOpenStack Proposal Bot proposed openstack/openstack-ansible master: Imported Translations from Zanata  https://review.opendev.org/c/openstack/openstack-ansible/+/79167407:43
*** kleini has joined #openstack-ansible07:43
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_nova master: Add galera port to nova config and database template  https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/79167507:44
*** macz_ has joined #openstack-ansible07:47
*** macz_ has quit IRC07:51
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_aodh master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_aodh/+/79167607:54
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_barbican master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_barbican/+/79167707:55
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_blazar master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_blazar/+/79167807:56
*** jbadiapa has joined #openstack-ansible08:32
openstackgerritlikui proposed openstack/openstack-ansible master: Replace deprecated UPPER_CONSTRAINTS_FILE variable  https://review.opendev.org/c/openstack/openstack-ansible/+/79169308:39
openstackgerritlikui proposed openstack/openstack-ansible master: Replace deprecated UPPER_CONSTRAINTS_FILE variable  https://review.opendev.org/c/openstack/openstack-ansible/+/79169308:44
*** pto has joined #openstack-ansible08:51
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_ceilometer/+/79170009:24
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_cloudkitty master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_cloudkitty/+/79170109:24
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_designate master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_designate/+/79170309:26
sakharkarnoonedeadpunk: I am trying to deploy OSA Victoria with all endpoint SSL/TLS with options haproxy_user_ssl_cert: /etc/openstack_deploy/ssl/tvm.cert.pem09:35
sakharkarhaproxy_user_ssl_key: /etc/openstack_deploy/ssl/tvm.key.pem09:35
sakharkarhaproxy_user_ssl_ca_cert: /etc/openstack_deploy/ssl/ca-chain.cert.pem09:35
sakharkarstill it is failing with the error http://paste.openstack.org/show/805420/09:35
noonedeadpunkoh, well, previously it was another error09:36
sakharkarnoonedeadpunk: could you please let us know what could be the issue and changes required09:36
noonedeadpunksakharkar: ok, so I think it's worth taking a look at haproxy config09:38
jrossersakharkar: is this a self signed certificate (internal CA) or is it a certificate you have purchased from a real CA09:41
sakharkarjrosser: Its a self signed certificate09:41
jrosserok, so there is no reason that your systems will trust this CA09:42
jrosserand i think that is the root cause of the trouble09:42
sakharkarjrosser: The same certificates works fine when used for Public endpoints only and fails with all endpoint on SSL/TLS09:43
jrosserwhat do you contact the public endpoint with?09:43
jrosseri.e, if you are using a browser for horizon for example, and you have imported the CA (or told the browser to trust the cert) then it will work from the point of view of the browser09:45
jrosserso the same sort of thing needs to be done for the internal services to trust your self-signed certificate when they contact the internal endpoint09:45
jrosserit is correct behaviour that the services will reject your self-signed certificate unless some provision has been made to install the CA into the system trust store09:46
noonedeadpunkthis wouldn't cause `SSL exception connecting to` though?09:47
noonedeadpunkso it feels more like haproxy serves it as http endpoint?09:47
noonedeadpunkoh, wait09:48
noonedeadpunkyeah09:48
noonedeadpunkI looked too down the stack trace09:48
jrosserhmm, well i'm thinking `certificate verify failed` points to "there is a cert, but i don't trust it"09:48
jrosserthough i could be wrong there09:48
noonedeadpunkyeah, you're right09:48
jrosserusing wget will prove this very quickly09:48
noonedeadpunksakharkar: I think you can set variables to make services not to check certificates validity09:49
jrossersakharkar: for a cloud using a company CA rather than a public one, the most correct approach is to put the company CA certificate in the system trust store of all your hosts09:50
sakharkarjrosser: But when we try to curl the endpoint after adding the ca.crt to trusted it works fine for us09:50
jrosserright, but this is python09:50
noonedeadpunkie - keystone_service_internaluri_insecure and keystone_service_adminuri_insecure to true09:50
jrosserso you need to tell python to use your CA as well09:50
sakharkarjrosser: we have added the entry haproxy_user_ssl_ca_cert: /etc/openstack_deploy/ssl/ca-chain.cert.pem09:51
jrosserthe python clients connect to haproxy and they do not understand the certificate they find there09:52
jrosserthat is becasue the python requests library uses a thing called the 'certifi' package which is a static bundle of CA certs09:52
jrossercertifi does not get extended *ever* when you add your own CA to the system trust store09:53
jrosserso i can totally understand curl working but not a python program09:53
jrosseryou can change the CA bundle that the underlying python libraries use by setting slmething like this in /etc/environment (or ubuntu example) `REQUESTS_CA_BUNDLE="/etc/ssl/certs/ca-certificates.crt"`09:54
jrossersakharkar: if you want a simple way to see if this is the issue then you can very quickly do this http://paste.openstack.org/show/805421/10:00
jrosserreplace with your internal https url of course10:00
sakharkarjrosser: Thank you..will try it out and let you know the output.10:01
openstackgerritMerged openstack/openstack-ansible-galera_server master: Include galera_devel into main  https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/79104510:24
openstackgerritMerged openstack/openstack-ansible-os_swift master: Do not collect gnocchi project ID  https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/79125010:36
openstackgerritMerged openstack/openstack-ansible-os_ceilometer master: Remove deprecated options from config  https://review.opendev.org/c/openstack/openstack-ansible-os_ceilometer/+/79125910:48
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_heat master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_heat/+/79171610:48
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_ironic master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/79171710:49
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_magnum master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/79171810:50
admin0morning10:55
openstackgerritMerged openstack/openstack-ansible-os_gnocchi master: Switch gnocchi service name to service  https://review.opendev.org/c/openstack/openstack-ansible-os_gnocchi/+/79125410:58
noonedeadpunko/11:15
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_manila master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/79172211:22
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_masakari master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_masakari/+/79172311:24
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_mistral master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_mistral/+/79172511:28
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_murano master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_murano/+/79172611:29
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_murano master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_murano/+/79172611:32
*** macz_ has joined #openstack-ansible11:36
*** macz_ has quit IRC11:40
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_octavia master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/79173111:41
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_sahara master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_sahara/+/79173211:42
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_senlin master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_senlin/+/79173411:43
openstackgerritMerged openstack/openstack-ansible master: Decrease manila tempest coverage  https://review.opendev.org/c/openstack/openstack-ansible/+/79120211:45
*** macz_ has joined #openstack-ansible11:57
*** macz_ has quit IRC12:02
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_swift master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/79174012:10
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_tacker master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_tacker/+/79174112:11
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_zun master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_zun/+/79174212:12
*** rh-jlabarre has joined #openstack-ansible12:14
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_ceilometer/+/79170012:16
openstackgerritMerged openstack/openstack-ansible master: Add option to remove group from inventory  https://review.opendev.org/c/openstack/openstack-ansible/+/79127712:28
*** macz_ has joined #openstack-ansible12:35
*** pcaruana has quit IRC12:37
*** macz_ has quit IRC12:40
*** pcaruana has joined #openstack-ansible12:42
openstackgerritMerged openstack/openstack-ansible-os_nova master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/79003712:43
openstackgerritMerged openstack/openstack-ansible-os_adjutant master: Install mysql client libraries  https://review.opendev.org/c/openstack/openstack-ansible-os_adjutant/+/77760712:44
*** mgariepy has joined #openstack-ansible12:59
openstackgerritMerged openstack/openstack-ansible-os_keystone master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/79003413:05
openstackgerritMerged openstack/openstack-ansible-os_cinder master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/79003513:09
*** dave-mccowan has quit IRC13:13
*** dave-mccowan has joined #openstack-ansible13:16
openstackgerritMerged openstack/openstack-ansible-os_glance master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_glance/+/79000613:22
*** jamesdenton has joined #openstack-ansible13:27
*** jamesden_ has quit IRC13:30
*** spatel_ has joined #openstack-ansible13:46
*** spatel_ is now known as spatel13:46
admin0spatel, here ?13:48
spatelyes13:48
admin0i was reading your ovn doc .. "make sure both vm01 and vm02 endup on two different compute nodes, if not then delete and re-create. :)" -- you can crate an anti-affinity rule13:49
admin0which ensures the vms are always on diff hosts13:49
admin0i am following your ovn build in my lab today13:49
admin0how about ovn troubleshooting .. how is 1:1 NAT handled ?13:50
mgariepyyou can migrate the vm after the creation also.13:50
admin0how/where is 1:1 nat handled13:50
spatelYes anti-affinity is good idea, i try to make it easy in this doc13:50
spatelwhat do you mean 1:1 NAT handled?  compute nodes is your network node and your router to do NAT13:51
admin0ignore that question :)13:51
spatelLike true DVR setup13:51
admin0i was thinking something else13:51
spateloh13:52
admin0do you have it in prod or in use .13:52
spatelstill doing testing on lab, i need to setup 3 node OVN controller to test all redundancy and troubleshooting of cluster failure13:52
spatelI don't think osa playbook provide OVN controller clustering, you need to do some hand work until playbook are ready13:53
*** jamesden_ has joined #openstack-ansible14:10
*** pto_ has joined #openstack-ansible14:10
openstackgerritJonathan Herlin proposed openstack/openstack-ansible-os_adjutant master: Remove incorrect horizon_post_install example  https://review.opendev.org/c/openstack/openstack-ansible-os_adjutant/+/78442014:13
*** jamesdenton has quit IRC14:14
*** pto has quit IRC14:14
*** pto_ has quit IRC14:15
*** chandankumar is now known as raukadah14:16
openstackgerritMerged openstack/openstack-ansible-os_manila master: [goal] Deprecate the JSON formatted policy file  https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/78224414:16
jrosserspatel: i don't think anyone is looking at the OVN controller stuff, if you have patches it would be great14:23
spateljrosser yes, soon i will have patch, I have build lab with 3 controller now going to start work on clustering part.14:24
jrosserawesome14:24
admin0if a db already exists, does openstack add new users14:43
admin0here is a use case14:43
admin0i have one openstack cluster using a very very old osa-inspired   custom ansible  script ..  its on linuxbridge and simple .. my thought is .. let me take just the database backup,   and run the osa playbooks on top of it, so that i can have all running using that same database14:44
admin0how feasible/possible is this ?14:44
admin0the thing i have to do is delete the mysql.user entries, so that osa can add entries again based on generated passwords14:45
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_ceilometer/+/79170014:49
admin0how are you guys doing logging ?14:52
admin0using beats ? any way integrated into osa ?14:52
spatelBeats is way to go..14:53
openstackgerritMerged openstack/openstack-ansible-os_neutron master: Use ansible_facts[] instead of fact variables  https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/77765014:56
jonherwe are testing fluentd -> graylog and that seems to do the job too14:56
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_magnum master: Add variables for rabbitmq ssl configuration  https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/79171814:57
*** macz_ has joined #openstack-ansible15:06
*** mgariepy has quit IRC15:08
*** spatel has quit IRC15:30
*** spatel_ has joined #openstack-ansible15:31
*** spatel_ is now known as spatel15:31
*** dave-mccowan has quit IRC15:33
*** dave-mccowan has joined #openstack-ansible15:37
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server master: Add debian bullseye support  https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/79176915:37
*** jawad_axd has quit IRC15:46
*** jawad_axd has joined #openstack-ansible15:46
jrosseroh here we go again E: Failed to fetch https://packages.erlang-solutions.com/debian/dists/buster/contrib/binary-amd64/Packages.bz2  File has unexpected size (479447 != 479445). Mirror sync in progress? [IP: 99.84.39.78 443]15:46
*** mgariepy has joined #openstack-ansible15:51
*** jawad_axd has quit IRC15:52
noonedeadpunkdoh:(15:54
noonedeadpunkbtw had the same issue localy recently with apt-cacher-ng...15:55
*** jawad_axd has joined #openstack-ansible16:17
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server master: Add debian bullseye support  https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/79176916:18
*** gyee has joined #openstack-ansible16:44
*** jawad_axd has quit IRC16:50
jrosserlooks reasonable on bullseye now16:56
jrossernext patches should get past setup-infrastructure16:56
*** andrewbonney has quit IRC17:46
*** jawad_axd has joined #openstack-ansible18:01
*** jawad_axd has quit IRC18:06
*** pto has joined #openstack-ansible18:27
*** pto has quit IRC18:27
*** pto has joined #openstack-ansible18:28
*** pto has quit IRC18:37
openstackgerritMerged openstack/ansible-config_template master: Remove references to unsupported operating systems  https://review.opendev.org/c/openstack/ansible-config_template/+/78075219:00
*** spatel has quit IRC20:06
*** jbadiapa has quit IRC20:08
openstackgerritMerged openstack/openstack-ansible master: Change order of swift and gnocchi installation  https://review.opendev.org/c/openstack/openstack-ansible/+/79126120:52
-openstackstatus- NOTICE: The Zuul service at zuul.opendev.org will be offline for a few minutes (starting now) in order for us to make some needed filesystem changes; if the outage lasts longer than anticipated we'll issue further notices21:31
*** macz_ has quit IRC23:32
*** tosky has quit IRC23:43

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!