| *** bengates has joined #openstack-kolla | 01:38 | |
| *** bengates has quit IRC | 01:43 | |
| *** ivan_lin has quit IRC | 01:47 | |
| *** ivan_lin has joined #openstack-kolla | 01:48 | |
| *** cosmicsound has quit IRC | 01:55 | |
| *** cosmicsound has joined #openstack-kolla | 01:55 | |
| *** oyrogerg has quit IRC | 01:57 | |
| *** oyrogerg has joined #openstack-kolla | 01:57 | |
| *** brinzhang_ is now known as brinzhang | 02:06 | |
| openstackgerrit | James Kirsch proposed openstack/kolla master: Add LetsEncrypt images for cert request/renewal https://review.opendev.org/c/openstack/kolla/+/741339 | 02:59 |
|---|---|---|
| *** vishalmanchanda has joined #openstack-kolla | 04:23 | |
| *** skramaja has joined #openstack-kolla | 04:28 | |
| *** brinzhang_ has joined #openstack-kolla | 04:45 | |
| *** brinzhang has quit IRC | 04:48 | |
| *** ivan_lin has quit IRC | 05:16 | |
| *** ivan_lin has joined #openstack-kolla | 05:17 | |
| *** ivan_lin has quit IRC | 05:25 | |
| *** ivan_lin has joined #openstack-kolla | 05:26 | |
| *** cah_link has joined #openstack-kolla | 06:55 | |
| *** bengates has joined #openstack-kolla | 07:03 | |
| *** jaicaa has quit IRC | 07:10 | |
| *** jaicaa has joined #openstack-kolla | 07:13 | |
| *** bengates has quit IRC | 07:14 | |
| *** bengates has joined #openstack-kolla | 07:15 | |
| *** bengates has quit IRC | 07:16 | |
| *** bengates has joined #openstack-kolla | 07:17 | |
| *** bengates has quit IRC | 07:18 | |
| *** bengates has joined #openstack-kolla | 07:19 | |
| *** wuchunyang has joined #openstack-kolla | 07:20 | |
| *** andrewbonney has joined #openstack-kolla | 07:24 | |
| *** luksky has joined #openstack-kolla | 07:34 | |
| yoctozepto | morning | 07:38 |
| *** rpittau|afk is now known as rpittau | 07:45 | |
| *** jovial[m] has joined #openstack-kolla | 07:58 | |
| mnasiadka | morning | 08:11 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: mariadb: Deprecate wsrep-notify.sh https://review.opendev.org/c/openstack/kolla-ansible/+/786245 | 08:12 |
| *** fuhrmannb has joined #openstack-kolla | 08:16 | |
| mnasiadka | hmm, seems the UCA Wallaby upgrade job failed because floating ip doesn't work, but Neutron claims everything is fine :) | 08:26 |
| *** luksky has quit IRC | 08:31 | |
| *** luksky has joined #openstack-kolla | 08:31 | |
| mnasiadka | although neutron-l3-agent log claims it was applied and working | 08:35 |
| mnasiadka | so I will need to reproduce it locally... | 08:35 |
| *** luksky has quit IRC | 08:42 | |
| hrw | elo | 08:48 |
| *** gfidente|afk is now known as gfidente | 08:53 | |
| *** luksky has joined #openstack-kolla | 08:55 | |
| mnasiadka | hrw: how's the Bullseye patch testing on aarch? I think yesterday you haven't pulled in k-a with the wsrep-notify patch? | 09:00 |
| hrw | mnasiadka: rabbitmq is crashing all the time | 09:02 |
| hrw | mnasiadka: so mariadb installs, several other ones too and then it reaches nova and fails due to no contact with rmq | 09:03 |
| *** gfidente has quit IRC | 09:14 | |
| *** gfidente has joined #openstack-kolla | 09:15 | |
| mnasiadka | hrw: why we are downgrading rabbitmq? | 09:21 |
| mnasiadka | hrw: 3.8.14 in Victoria and 3.8.12 in Wallaby? | 09:21 |
| mnasiadka | Starting RabbitMQ 3.8.14 on Erlang 23.3.1 vs Starting RabbitMQ 3.8.12 on Erlang 23.3.1 | 09:22 |
| *** gfidente has quit IRC | 09:24 | |
| hrw | which distro/arch? | 09:25 |
| *** vishalmanchanda has quit IRC | 09:27 | |
| mnasiadka | ubuntu/x86 | 09:27 |
| hrw | no idea | 09:28 |
| hrw | centos and debian grab 3.8.14 | 09:28 |
| *** ivve has joined #openstack-kolla | 09:29 | |
| *** k_mouza has joined #openstack-kolla | 09:29 | |
| *** kevko_ has quit IRC | 09:30 | |
| *** kevko has joined #openstack-kolla | 09:30 | |
| kevko | morning \o/ | 09:30 |
| hrw | mnasiadka: ok. it is upstream decision - they say '3.8.12' in focal Packages | 09:30 |
| hrw | https://dl.bintray.com/rabbitmq/debian/dists/focal/main/binary-amd64/:Packages | 09:31 |
| mnasiadka | hmm, rabbitmq website points to packagecloud, not to bintray | 09:33 |
| mnasiadka | and still, we have focal in Victoria, so why there's 3.8.14? :) | 09:33 |
| hrw | let me run a build | 09:37 |
| hrw | victoria uses bionic repo | 09:39 |
| hrw | during W we moved to use focal repo as someone mentioned that it exists now | 09:39 |
| mnasiadka | yeah, but it seems it gets older package | 09:40 |
| hrw | https://review.opendev.org/c/openstack/kolla/+/785114 | 09:40 |
| mnasiadka | And we could move to packagecloud, because google says bintray is getting shut down ;) | 09:40 |
| hrw | mnasiadka: how it went? ah, 'patches welcome' | 09:41 |
| hrw | ;D | 09:41 |
| mnasiadka | yeah, I'll look into it today ;) | 09:41 |
| hrw | and we need to merge it before 'switch to new way of APT key/repo' patch so it will be easy to backport | 09:41 |
| hrw | mnasiadka: if possible please avoid using apt-key. fetching gpg key file is recommended | 09:44 |
| *** vishalmanchanda has joined #openstack-kolla | 09:48 | |
| mnasiadka | hrw: what is funny we use packagecloud already for non x86 rabbitmq repos | 09:54 |
| hrw | I see only bintray | 09:55 |
| mnasiadka | for centos yes, for ubuntu we we have packagecloud in sources.list.ubuntu.aarch64 | 09:59 |
| hrw | mnasiadka: commented out | 10:02 |
| hrw | as we do not use entries from there but from repos.yaml | 10:02 |
| hrw | I have a patch which drops sources.list.* files | 10:02 |
| hrw | but it is on pile of other patches so wait for them to be merged first | 10:03 |
| mnasiadka | ah right | 10:04 |
| *** wuchunyang has quit IRC | 10:11 | |
| *** gfidente has joined #openstack-kolla | 10:14 | |
| *** e0ne has joined #openstack-kolla | 10:24 | |
| hrw | kolla-ansible-ubuntu-source-upgrade https://zuul.opendev.org/t/openstack/build/6403c9d1f7e8446bba9554a62572dd9b : SUCCESS in 58m 56s | 10:25 |
| hrw | UCA +2? | 10:25 |
| mnasiadka | incredible | 10:26 |
| hrw | mnasiadka: give it +W before it fails | 10:26 |
| mnasiadka | +w and off we go | 10:26 |
| hrw | ;D | 10:29 |
| hrw | that whole fancy zuul build interface drives me mad | 10:32 |
| *** e0ne has quit IRC | 10:37 | |
| *** gfidente has quit IRC | 10:56 | |
| *** k_mouza has quit IRC | 10:58 | |
| yoctozepto | it works for me | 10:58 |
| hrw | maybe I should restart firefox more often than once per month | 10:59 |
| *** gfidente has joined #openstack-kolla | 11:15 | |
| *** k_mouza has joined #openstack-kolla | 11:15 | |
| *** k_mouza has quit IRC | 11:15 | |
| *** k_mouza has joined #openstack-kolla | 11:16 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: rabbitmq: Move to packagecloud https://review.opendev.org/c/openstack/kolla/+/786390 | 11:17 |
| hrw | https://review.opendev.org/c/openstack/kolla-ansible/+/783887 - opinions? | 11:18 |
| mnasiadka | well, from one perspective it's nice, from another we're in feature freeze? | 11:20 |
| hrw | I meant opinion, now +w ;D | 11:20 |
| *** shyamb has joined #openstack-kolla | 11:27 | |
| *** shyam89 has joined #openstack-kolla | 11:27 | |
| *** jovial[m] has quit IRC | 11:30 | |
| *** shyam89 has quit IRC | 11:43 | |
| *** shyamb has quit IRC | 11:43 | |
| *** k_mouza has quit IRC | 11:43 | |
| *** k_mouza has joined #openstack-kolla | 11:44 | |
| *** jovial[m] has joined #openstack-kolla | 11:54 | |
| *** jovial[m] has quit IRC | 11:59 | |
| *** jovial[m] has joined #openstack-kolla | 12:01 | |
| *** e0ne has joined #openstack-kolla | 12:08 | |
| *** shyamb has joined #openstack-kolla | 12:14 | |
| mnasiadka | hrw: and gate failed | 12:19 |
| hrw | meh | 12:20 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: rabbitmq: Move to packagecloud https://review.opendev.org/c/openstack/kolla/+/786390 | 12:25 |
| *** shyamb has quit IRC | 12:25 | |
| hrw | +1 RP+1 +comment | 12:30 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: rabbitmq: Move to packagecloud https://review.opendev.org/c/openstack/kolla/+/786390 | 12:32 |
| mnasiadka | updated ;) | 12:32 |
| hrw | +2 rp+1, waiting for zuul | 12:34 |
| hrw | and anther comment | 12:37 |
| hrw | so far bullseye erlang is newer but it may change | 12:38 |
| hrw | will send comment once check | 12:38 |
| hrw | done | 12:43 |
| *** shyamb has joined #openstack-kolla | 12:50 | |
| *** ivve has quit IRC | 12:58 | |
| *** e0ne has quit IRC | 13:24 | |
| *** e0ne has joined #openstack-kolla | 13:26 | |
| *** e0ne has quit IRC | 13:26 | |
| *** samcat116 has joined #openstack-kolla | 13:27 | |
| *** e0ne has joined #openstack-kolla | 13:28 | |
| *** shyam89 has joined #openstack-kolla | 13:38 | |
| hrw | mnasiadka: will push update to packagecloud patch. jsut need to build test | 13:38 |
| *** shyam89 has quit IRC | 13:38 | |
| *** shyamb has quit IRC | 13:39 | |
| mnasiadka | hrw: sure, feel free | 13:40 |
| hrw | just informing ;D | 13:41 |
| hrw | mnasiadka: it should also fix ubuntu/aarch64 | 13:41 |
| yoctozepto | fyi, gerrit->launchpad works again (big thanks to fungi) | 13:47 |
| mnasiadka | nice | 13:49 |
| *** ivan_lin has quit IRC | 13:49 | |
| hrw | mnasiadka: hm. that erlang repo provides just a bunch of erlang packages while debian/ubuntu have more | 13:54 |
| mnasiadka | hrw: well, does rabbitmq need that more? | 13:55 |
| hrw | mnasiadka: https://www.rabbitmq.com/install-debian.html mentions using ppa for erlang. | 13:59 |
| mnasiadka | hrw: but ppa is over http - so that's why I went for the other one - I doubt it has more packages? | 13:59 |
| hrw | mnasiadka: there are two different ways of building/packging erlang for debian/ubuntu | 14:00 |
| hrw | one is 'create few packages and hope for best', second is 'just reuse debian packaging' | 14:00 |
| mnasiadka | hrw: if erlang versions fit rabbitmq requirements - we might just use what ubuntu/debian provides | 14:00 |
| mnasiadka | I just assumed we want to mimick centos - but maybe we don't | 14:01 |
| *** e0ne has quit IRC | 14:02 | |
| *** e0ne has joined #openstack-kolla | 14:03 | |
| hrw | bbl - meeting | 14:04 |
| *** k_mouza has quit IRC | 14:05 | |
| *** k_mouza_ has joined #openstack-kolla | 14:06 | |
| yoctozepto | mnasiadka: nice? it restores my hope in managing the bugs! | 14:47 |
| mnasiadka | yoctozepto: all hope is lost | 14:48 |
| yoctozepto | lol | 14:48 |
| mnasiadka | yoctozepto: mark all as invalid and wait if people will claim these are still needed :D | 14:48 |
| yoctozepto | yeah, we can probably do this for old and hard to understand ones... | 14:48 |
| mnasiadka | we also have some really old reviews, probably need some cleaning if we care | 14:49 |
| yoctozepto | side-question: did stackhpc try out cyborg? | 14:49 |
| yoctozepto | mnasiadka: yeah, we clean up with Mark from time to time | 14:49 |
| mnasiadka | I don't think so, we mainly use PCI pass through | 14:49 |
| mnasiadka | if we're talking GPU | 14:50 |
| mnasiadka | but I'm not the resident expert ;) | 14:50 |
| *** ysirndjuro has joined #openstack-kolla | 15:07 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: move to Debian 'bullseye' https://review.opendev.org/c/openstack/kolla/+/772479 | 15:09 |
| mnasiadka | hrw: now it should pass aarch64 | 15:16 |
| hrw | btw 'check arm64' is magic incantation to recheck on aarch64 only | 15:26 |
| yoctozepto | ++ | 15:28 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: move to Debian 'bullseye' https://review.opendev.org/c/openstack/kolla/+/772479 | 15:33 |
| *** bengates has quit IRC | 15:38 | |
| *** bengates has joined #openstack-kolla | 16:02 | |
| *** bengates has quit IRC | 16:06 | |
| ozzzo | working on my train->ussuri upgrade, I'm getting a rate limit error | 16:08 |
| ozzzo | Internal Server Error (\"toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit\")\\n'"} | 16:08 |
| ozzzo | what's the best way to work around that? | 16:08 |
| ozzzo | if I create an account, can I enter the account info somewhere in my config and use it for the upgrade? | 16:09 |
| *** rpittau is now known as rpittau|afk | 16:09 | |
| mnasiadka | set up your own registry and build+push images into it? | 16:09 |
| ozzzo | is there a document on that? | 16:12 |
| mnasiadka | ozzzo: https://docs.openstack.org/kolla-ansible/latest/user/multinode.html#option-1-local-registry | 16:15 |
| mnasiadka | ozzzo: of course you need to use kolla to build images and push them into that registry | 16:15 |
| ozzzo | ok ty | 16:16 |
| mnasiadka | ozzzo: https://docs.openstack.org/kolla/latest/admin/image-building.html (how to build) | 16:16 |
| mnasiadka | replace latest with your release - I guess ussuri | 16:16 |
| ozzzo | ok | 16:17 |
| hrw | ozzzo: use some tag on your images so you will know which one you deployed. handy when new nodes are added | 16:23 |
| *** samcat116 has quit IRC | 16:26 | |
| hrw | ozzzo: Linaro Developer Cloud uses ussuri-ldc-BUILDNUMBER for example | 16:26 |
| *** samcat116 has joined #openstack-kolla | 16:28 | |
| yoctozepto | ^ using different tags per build ensure you can easily rollback | 16:31 |
| yoctozepto | ensures* | 16:31 |
| yoctozepto | should you need it | 16:32 |
| hrw | yoctozepto: https://review.opendev.org/c/openstack/kolla/+/786390 | 16:36 |
| *** e0ne has quit IRC | 16:37 | |
| yoctozepto | hrw: cloudsmith? | 16:37 |
| hrw | yoctozepto: mnasiadka's patch | 16:39 |
| *** samcat116 has quit IRC | 16:42 | |
| *** k_mouza_ has quit IRC | 16:43 | |
| *** samcat116 has joined #openstack-kolla | 16:44 | |
| *** k_mouza has joined #openstack-kolla | 16:44 | |
| yoctozepto | hrw: right | 16:45 |
| yoctozepto | mnasiadka: cloudsmith? | 16:45 |
| yoctozepto | and erlang gone versionless | 16:45 |
| *** k_mouza has quit IRC | 16:51 | |
| mnasiadka | yoctozepto: cloudsmith - on rabbitmq webpage it says to use either ppa erlang repo or cloudsmith | 16:56 |
| mnasiadka | yoctozepto: ppa is http only, I tend to have http allergy | 16:56 |
| yoctozepto | we need gpg over https | 16:56 |
| yoctozepto | and then I don't really care | 16:57 |
| yoctozepto | now it's a mix | 16:57 |
| yoctozepto | anyhow | 16:57 |
| yoctozepto | I understand we need to backport this | 16:57 |
| yoctozepto | to all the branches... | 16:57 |
| yoctozepto | what about erlang pin? it gets removed | 16:57 |
| mnasiadka | yoctozepto: but hrw pointed we could use in-distro for ubu/deb | 16:57 |
| *** gfidente is now known as gfidente|afk | 16:57 | |
| *** jonaspaulo has joined #openstack-kolla | 16:57 | |
| mnasiadka | We can put it back | 16:57 |
| yoctozepto | oh my, switching back to distro-provided? | 16:58 |
| * yoctozepto 's head spins | 16:58 | |
| ozzzo | ok I'll read up on tags, ty | 16:59 |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs https://review.opendev.org/c/openstack/kolla-ansible/+/741340 | 17:01 |
| *** cah_link has quit IRC | 17:22 | |
| *** zul has joined #openstack-kolla | 17:25 | |
| *** andrewbonney has quit IRC | 17:50 | |
| *** bsanjeewa has joined #openstack-kolla | 18:09 | |
| *** jonaspaulo has quit IRC | 18:09 | |
| bsanjeewa | subscribe | 18:16 |
| * yoctozepto knows nothing about a subscription service around | 18:18 | |
| *** vishalmanchanda has quit IRC | 18:27 | |
| *** skramaja has quit IRC | 18:28 | |
| *** bsanjeewa_ has joined #openstack-kolla | 18:30 | |
| *** bsanjeewa has quit IRC | 18:33 | |
| *** e0ne has joined #openstack-kolla | 19:03 | |
| *** e0ne has quit IRC | 19:04 | |
| *** e0ne has joined #openstack-kolla | 19:05 | |
| *** e0ne has quit IRC | 19:24 | |
| *** dave-mccowan has quit IRC | 19:43 | |
| *** dave-mccowan has joined #openstack-kolla | 19:45 | |
| *** bsanjeewa_ has quit IRC | 19:50 | |
| *** macz_ has joined #openstack-kolla | 19:55 | |
| *** k_mouza has joined #openstack-kolla | 20:45 | |
| *** k_mouza has quit IRC | 20:49 | |
| ozzzo | I successfully upgraded my test kolla cluster from Train->Ussuri, but VM build fails now, with rados.Rados.connect\nrados.PermissionDeniedError | 21:01 |
| ozzzo | https://paste.ubuntu.com/p/bVNVDRhDvB/ | 21:02 |
| *** jbadiapa has quit IRC | 21:02 | |
| *** samcat116 has quit IRC | 21:28 | |
| ozzzo | looking at this document: https://docs.openstack.org/kolla-ansible/latest/reference/storage/external-ceph-guide.html | 21:35 |
| ozzzo | it looks like I should have a ceph_glance_keyring setting in globals.yml | 21:36 |
| ozzzo | but I didn't have that in Train and it was working. Maybe that is a new requirement in Ussuri? | 21:36 |
| ozzzo | I have /etc/kolla/config/glance/ceph.client.glance.keyring but no variable in globals.yml that points to it | 21:37 |
| ozzzo | er... no that's probably not it; it looks like that's the default | 21:38 |
| *** zijlboot_ has joined #openstack-kolla | 22:42 | |
| *** zijlboot has quit IRC | 22:44 | |
| *** macz_ has quit IRC | 23:16 | |
| *** macz_ has joined #openstack-kolla | 23:20 | |
| *** macz_ has quit IRC | 23:25 | |
| *** luksky has quit IRC | 23:58 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!