| opendevreview | Michal Nasiadka proposed openstack/kayobe master: ovn: add network group to neutron-ovn-metadata-agent https://review.opendev.org/c/openstack/kayobe/+/850608 | 06:18 |
|---|---|---|
| opendevreview | Konstantin Yarovoy proposed openstack/kolla master: Refactor image module https://review.opendev.org/c/openstack/kolla/+/828716 | 07:31 |
| opendevreview | Konstantin Yarovoy proposed openstack/kolla master: Refactor image module https://review.opendev.org/c/openstack/kolla/+/828716 | 07:54 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: clustercheck: move from xinetd to socat https://review.opendev.org/c/openstack/kolla-ansible/+/850611 | 08:29 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: clustercheck: move from xinetd to socat https://review.opendev.org/c/openstack/kolla-ansible/+/850611 | 08:42 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: clustercheck: move from xinetd to socat https://review.opendev.org/c/openstack/kolla-ansible/+/850611 | 08:42 |
| EugenMayer | Hello. Trying to upgrade with the xena series. Updated to the latest xena kolla/kolla-ansible, but when i run deploy it fails with the sasl password https://gist.github.com/EugenMayer/9fa2521af059933fac0a39ff71314ee5 - is this a known issue? | 08:45 |
| EugenMayer | s/with/within | 08:46 |
| mnasiadka | EugenMayer: have you merged the passwords.yml changes? like here: https://docs.openstack.org/kolla-ansible/latest/user/operating-kolla.html#preparation-the-real-deal | 08:47 |
| EugenMayer | hmm i was looking on https://docs.openstack.org/kolla-ansible/latest/user/operating-kolla.html#upgrade-procedure .. i seem to not been aware sorry. checking your docs | 08:48 |
| EugenMayer | mnasiadka that was surely it, i did not follow those docs. Will do better next time, thank you for the hint! | 08:54 |
| mnasiadka | EugenMayer: np | 09:01 |
| EugenMayer | mnasiadka seems like the upgrade went fine at first sight, the ansible task completed without issues. now on all computes nova seem to have issues to start the docker container: Restarting (0) 6 seconds ago nova_compute | 09:07 |
| EugenMayer | am i usually forced to restart the controllers/computes after a in-series upgrade= | 09:08 |
| mnasiadka | no, it should work | 09:09 |
| mnasiadka | check docker logs, nova-compute logs | 09:09 |
| EugenMayer | docker logs looked non problematic, checking the nova logs now | 09:10 |
| EugenMayer | 2022-07-21 11:10:56.775 7 INFO nova.virt.libvirt.driver [-] Connection event '0' reason 'Failed to connect to libvirt: authentication failed: authentication failed' | 09:11 |
| EugenMayer | https://gist.github.com/EugenMayer/bad693ff3a4a989b54abbce641e7c0d1 | 09:12 |
| mnasiadka | yeah, that's related to sasl for sure | 09:17 |
| mnasiadka | you can check libvirt logs for anything meaningful | 09:17 |
| EugenMayer | mnasiadka where do i find those? | 09:19 |
| EugenMayer | found those | 09:20 |
| EugenMayer | 2022-07-21 09:19:23.586+0000: 6539: error : virNetSASLSessionServerStep:594 : authentication failed: Failed to start SASL negotiation: -20 (SASL(-13): user not found: unable to canonify user and get auxprops) | 09:20 |
| EugenMayer | 2022-07-21 09:19:23.586+0000: 6539: error : remoteDispatchAuthSaslStep:3991 : authentication failed: authentication failed | 09:20 |
| EugenMayer | mnasiadka seems to be related to what was already required for the password merge, the new sasl password. Seems like this has been introduced but maybe the user has not been created on 'upgrade' | 09:27 |
| EugenMayer | maybe i would have needed to add something new to the globals too | 09:29 |
| EugenMayer | mnasiadka i guess i'am confronted with https://docs.openstack.org/releasenotes/kolla-ansible/xena.html#upgrade-notes | 09:36 |
| EugenMayer | while i generated the password and uses kolla upgrade, i assume (and docker ps tells the same story) that all containers have been replaced using the latest xena variant, including the libvirt one | 09:37 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: clustercheck: move from xinetd to socat https://review.opendev.org/c/openstack/kolla-ansible/+/850611 | 09:38 |
| EugenMayer | i merged the multinode inventory changes too mnasiadka, but i cannot get rid of the issue with the sasl password | 10:08 |
| mnasiadka | mgoddard: any idea what might be wrong? ^^ | 10:34 |
| EugenMayer | mnasiadka looking at the ansible playbooks, i ran saslpasswd2 -c -p -a libvirt on the nova_libvirt container with nova/the sasl password | 10:39 |
| EugenMayer | i double checked that my nova auth.conf is correct, which it is | 10:39 |
| opendevreview | Merged openstack/kolla-ansible master: Add [taskflow] section for masakari.conf.j2 https://review.opendev.org/c/openstack/kolla-ansible/+/850291 | 11:47 |
| opendevreview | Ivan Halomi proposed openstack/kolla-ansible master: Add systemd container control https://review.opendev.org/c/openstack/kolla-ansible/+/816724 | 11:57 |
| *** amoralej is now known as amoralej|lunch | 12:20 | |
| opendevreview | Michal Arbet proposed openstack/kolla master: Update ProxySQL to version 2.4 https://review.opendev.org/c/openstack/kolla/+/850636 | 12:28 |
| opendevreview | Sergei Raiskii proposed openstack/kolla-ansible master: Allow cinder-backup to be configured to use S3 backend. https://review.opendev.org/c/openstack/kolla-ansible/+/849711 | 12:38 |
| kevko | hmm, kolla build is probably broken because of some pip package incompatibiltiy :/ | 13:05 |
| hrw | ? | 13:07 |
| hrw | which build | 13:08 |
| kevko | hrw: https://review.opendev.org/c/openstack/kolla/+/850636 << if you check the logs ...there is no problem with proxysql ..but with pip dependencies | 13:11 |
| kevko | hrw: for example -> https://zuul.opendev.org/t/openstack/build/af9881ee80ad4b808626fa7d8f361841/log/kolla/build/000_FAILED_openstack-base.log <- | 13:11 |
| hrw | INFO:kolla.common.utils.openstack-base:[91mERROR: Could not install packages due to an OSError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read)) | 13:12 |
| kevko | hmm, what is that :/ | 13:12 |
| hrw | INFO:kolla.common.utils.openstack-base:1mWARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))': /pypi/simple/grpcio/ | 13:12 |
| hrw | failed to connect to pypi mirror | 13:12 |
| kevko | hmmm...it looks like intermittent error | 13:13 |
| hrw | recheck? | 13:13 |
| kevko | yeah | 13:13 |
| kevko | unfortunatelly RemoteDisconnected('Remote end closed connection without response') will be never fixed :( | 13:14 |
| opendevreview | Merged openstack/kolla-ansible master: make "external_labels" optional on prometheus.yml https://review.opendev.org/c/openstack/kolla-ansible/+/850446 | 13:26 |
| opendevreview | Verification of a change to openstack/kolla master failed: Install etcd binary from github https://review.opendev.org/c/openstack/kolla/+/847368 | 13:26 |
| *** amoralej|lunch is now known as amoralej | 13:28 | |
| kevko | hrw: still same :/ | 14:05 |
| opendevreview | Merged openstack/kolla-ansible master: init-runonce: Migrate to ECDSA keys https://review.opendev.org/c/openstack/kolla-ansible/+/847187 | 14:11 |
| opendevreview | Merged openstack/ansible-collection-kolla master: packages: add tzdata to ubuntu package list https://review.opendev.org/c/openstack/ansible-collection-kolla/+/850336 | 14:11 |
| opendevreview | Merged openstack/kolla-ansible master: Add support for Ubuntu Jammy Jellyfish (22.04 LTS) https://review.opendev.org/c/openstack/kolla-ansible/+/846070 | 14:12 |
| jingvar | I have to use custom config.json for some services , is there a way except downstream the kolla-ansible code (templates)? | 14:13 |
| opendevreview | Michal Arbet proposed openstack/kolla master: Run proxysql with proxysql user https://review.opendev.org/c/openstack/kolla/+/850656 | 14:14 |
| opendevreview | Michal Arbet proposed openstack/kolla master: Run proxysql with proxysql user https://review.opendev.org/c/openstack/kolla/+/850656 | 14:15 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database https://review.opendev.org/c/openstack/kolla-ansible/+/770215 | 14:21 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding https://review.opendev.org/c/openstack/kolla-ansible/+/770216 | 14:21 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test ProxySQL with shards in the nova cells scenario https://review.opendev.org/c/openstack/kolla-ansible/+/770621 | 14:21 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Trigger cells job https://review.opendev.org/c/openstack/kolla-ansible/+/838916 | 14:21 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Use Docker healthchecks for mariadb-server service https://review.opendev.org/c/openstack/kolla-ansible/+/805616 | 14:21 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database https://review.opendev.org/c/openstack/kolla-ansible/+/770215 | 14:29 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding https://review.opendev.org/c/openstack/kolla-ansible/+/770216 | 14:29 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test ProxySQL with shards in the nova cells scenario https://review.opendev.org/c/openstack/kolla-ansible/+/770621 | 14:29 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Trigger cells job https://review.opendev.org/c/openstack/kolla-ansible/+/838916 | 14:29 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Use Docker healthchecks for mariadb-server service https://review.opendev.org/c/openstack/kolla-ansible/+/805616 | 14:29 |
| kevko | hrw: hmm, I am able to build it normally on localhost ..what is different in CI ? it's failing every time | 15:01 |
| hrw | kevko: http://mirror.bhs1.ovh.opendev.org:8080/pypi/simple/grpcio/ - check which version failed? | 15:05 |
| hrw | hm. fetching works | 15:06 |
| kevko | hrw: it looks like it's not problem with grpcio only | 15:11 |
| kevko | https://paste.opendev.org/show/bskwQbwP8VLVJchVXknW/ | 15:11 |
| hrw | hm. asked on #opendev? | 15:12 |
| yoctozepto | hrw, kevko: #opendev is debugging the issue | 15:12 |
| hrw | yoctozepto: thanks | 15:12 |
| yoctozepto | it affects all projects | 15:12 |
| yoctozepto | something about pypi mirrors | 15:12 |
| kevko | rrr | 15:12 |
| kevko | switch to binary images :P | 15:12 |
| kevko | haha | 15:13 |
| kevko | yoctozepto: could you check quickly proxysql change i've fixed last two comments ? | 15:13 |
| kevko | yoctozepto: few lines | 15:13 |
| yoctozepto | kevko: k-a? | 15:16 |
| hrw | kevko: what are binary images? | 15:18 |
| kevko | yoctozepto: yeah | 15:19 |
| kevko | hrw: just joking ...kolla binary images .. | 15:19 |
| hrw | kevko: I tend to put sarcasm far too often than people are used to | 15:20 |
| kevko | hrw: haha, I'll try to get used to it | 15:22 |
| kevko | yoctozepto, hrw: it looks like there is more context here -> https://zuul.opendev.org/t/openstack/build/8b8b388d3e2d484b9c2215144b50d505/log/kolla/build/openstack-base.log | 15:32 |
| kevko | hmm ..no, it's the same | 15:33 |
| *** amoralej is now known as amoralej|off | 15:57 | |
| yoctozepto | kevko: why have you moved the start script to k-a? | 16:17 |
| yoctozepto | there is no templating happening | 16:17 |
| yoctozepto | do you have any plans for that? why complicate an already complicated thing? :D | 16:18 |
| kevko | yoctozepto: well, it's better to modify config.json ..not script inside ..and if i didn't use bash ..i was not able to log to file .. | 16:24 |
| kevko | yoctozepto: on the other side ..if you think it was better before ..i can revert it :P | 16:25 |
| yoctozepto | kevko: I was just curious why it's done like this; I don't see an issue with going this way but it's, so to speak, unorthodox :-) | 16:25 |
| kevko | yoctozepto: it was also frickler comment few patchset before ..so he will be happy :P | 16:26 |
| yoctozepto | kevko: was it now? | 16:26 |
| yoctozepto | let me see | 16:26 |
| yoctozepto | you have added frickler as a reviewer | 16:27 |
| yoctozepto | he has not commented on it at all | 16:27 |
| kevko | really ? i remember :D | 16:27 |
| yoctozepto | kevko: maybe you think about it so much that frickler has appeared in your dreams :-) | 16:30 |
| yoctozepto | anyway, to help you forget about proxysql, I have given you a CR+2 | 16:30 |
| yoctozepto | please use it wisely | 16:30 |
| kevko | i saw :D | 16:30 |
| kevko | i am going to reply to your comments in patchsets in stack | 16:30 |
| yoctozepto | good | 16:33 |
| yoctozepto | let's get this over with | 16:33 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding https://review.opendev.org/c/openstack/kolla-ansible/+/770216 | 17:01 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test ProxySQL with shards in the nova cells scenario https://review.opendev.org/c/openstack/kolla-ansible/+/770621 | 17:01 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Trigger cells job https://review.opendev.org/c/openstack/kolla-ansible/+/838916 | 17:01 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Use Docker healthchecks for mariadb-server service https://review.opendev.org/c/openstack/kolla-ansible/+/805616 | 17:01 |
| opendevreview | Kyle Dean proposed openstack/kayobe master: Add support for firewalld on Ubuntu https://review.opendev.org/c/openstack/kayobe/+/850390 | 17:08 |
| opendevreview | Kyle Dean proposed openstack/kayobe master: Add support for firewalld on Ubuntu https://review.opendev.org/c/openstack/kayobe/+/850390 | 17:08 |
| opendevreview | Juan Pablo Suazo proposed openstack/kolla-ansible master: Allows for multiple Ceph Conf files https://review.opendev.org/c/openstack/kolla-ansible/+/848029 | 20:12 |
| opendevreview | Juan Pablo Suazo proposed openstack/kolla-ansible master: Allows for multiple Ceph Conf files https://review.opendev.org/c/openstack/kolla-ansible/+/848029 | 20:18 |
| xiaolong | hello | 23:43 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!