Wednesday, 2021-02-17

« Tuesday, 2021-02-16 Index Thursday, 2021-02-18 »
*** k_mouza has joined #openstack-kolla00:30
*** e0ne has joined #openstack-kolla00:31
*** k_mouza has quit IRC00:34
*** e0ne has quit IRC00:35
*** brinzhang has joined #openstack-kolla01:10
*** maharg101 has joined #openstack-kolla01:24
*** maharg101 has quit IRC01:29
*** ricolin has joined #openstack-kolla01:29
*** k_mouza has joined #openstack-kolla01:49
*** k_mouza has quit IRC02:00
*** ysirndjuro has left #openstack-kolla02:04
*** e0ne has joined #openstack-kolla02:32
*** e0ne has quit IRC02:37
*** maharg101 has joined #openstack-kolla03:25
*** maharg101 has quit IRC03:30
*** skramaja has joined #openstack-kolla03:51
*** k_mouza has joined #openstack-kolla04:00
*** k_mouza has quit IRC04:04
*** rohit02 has joined #openstack-kolla04:10
*** k_mouza has joined #openstack-kolla04:13
*** ricolin has quit IRC04:16
*** k_mouza has quit IRC04:18
*** e0ne has joined #openstack-kolla04:32
*** jpward has quit IRC04:33
*** vishalmanchanda has joined #openstack-kolla04:36
*** e0ne has quit IRC04:37
*** cah_link has joined #openstack-kolla05:06
*** cah_link has quit IRC05:19
*** maharg101 has joined #openstack-kolla05:26
*** maharg101 has quit IRC05:31
*** ricolin has joined #openstack-kolla06:29
*** e0ne has joined #openstack-kolla06:33
*** e0ne has quit IRC06:38
*** cz3 has quit IRC06:48
*** cz3 has joined #openstack-kolla06:49
*** cah_link has joined #openstack-kolla06:53
*** pvh_sa has joined #openstack-kolla06:55
openstackgerritPierre Riteau proposed openstack/kayobe stable/train: Add ipa_build_upper_constraints_file variable  https://review.opendev.org/c/openstack/kayobe/+/77596507:00
openstackgerritPierre Riteau proposed openstack/kayobe stable/train: Fix building CentOS 8 IPA images on stable/train  https://review.opendev.org/c/openstack/kayobe/+/77394407:02
openstackgerritPierre Riteau proposed openstack/kayobe stable/train: Test building seed deployment images in the seed job (CentOS 8 only)  https://review.opendev.org/c/openstack/kayobe/+/77385307:07
*** pvh_sa has quit IRC07:10
openstackgerritPierre Riteau proposed openstack/kayobe stable/train: Test building seed deployment images in the seed job (CentOS 8 only)  https://review.opendev.org/c/openstack/kayobe/+/77385307:26
*** maharg101 has joined #openstack-kolla07:27
*** k_mouza has joined #openstack-kolla07:29
*** maharg101 has quit IRC07:32
*** k_mouza has quit IRC07:34
*** maharg101 has joined #openstack-kolla07:38
*** heikkine has joined #openstack-kolla07:44
*** luksky has joined #openstack-kolla07:57
*** amoralej|off is now known as amoralej08:07
*** ricolin has quit IRC08:08
*** ricolin_ has joined #openstack-kolla08:12
*** rpittau|afk is now known as rpittau08:12
*** andrewbonney has joined #openstack-kolla08:13
*** bengates has joined #openstack-kolla08:15
*** ricolin_ is now known as ricolin08:17
*** bengates has quit IRC08:19
*** bengates has joined #openstack-kolla08:20
mnasiadkaMorning08:23
mnasiadka2021-02-16 20:50:20.787 8 WARNING octavia.cmd.api [-] You are running the Octavia API wsgi application using simple_server. We do not recommend this outside of simple testing. We recommend you run the Octavia API wsgi with a more full function server such as gunicorn or uWSGI.08:23
mnasiadkaHuh08:23
*** bengates has quit IRC08:23
*** bengates has joined #openstack-kolla08:25
*** e0ne has joined #openstack-kolla08:34
*** dougsz has joined #openstack-kolla08:38
*** e0ne has quit IRC08:38
*** bengates has quit IRC08:43
*** bengates has joined #openstack-kolla08:44
*** gfidente has joined #openstack-kolla08:44
mgoddardmorning08:45
*** e0ne has joined #openstack-kolla08:45
mgoddardmnasiadka: yes I noticed that a while ago. Also means no backend TLS for octavia08:45
*** jbadiapa has joined #openstack-kolla08:46
mnasiadkamgoddard: seems that Octavia deploys properly in octavia-driver-agent change, I'll add some checks for creating an LB in test-ovn.sh today (and docs of course)08:46
mgoddardmnasiadka: wonderful08:47
*** bengates_ has joined #openstack-kolla08:48
*** bengates has quit IRC08:51
openstackgerritPierre Riteau proposed openstack/kayobe stable/train: Test building seed deployment images in the seed job (CentOS 8 only)  https://review.opendev.org/c/openstack/kayobe/+/77385309:10
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/victoria: Update String type for Monasca ES template  https://review.opendev.org/c/openstack/kolla-ansible/+/77590809:18
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/ussuri: Update String type for Monasca ES template  https://review.opendev.org/c/openstack/kolla-ansible/+/77590909:18
*** jbadiapa has quit IRC09:29
openstackgerritPierre Riteau proposed openstack/kayobe stable/train: Test building seed deployment images in the seed job (CentOS 8 only)  https://review.opendev.org/c/openstack/kayobe/+/77385309:37
openstackgerritMark Goddard proposed openstack/kolla master: CI: publish images with a master-weekly tag  https://review.opendev.org/c/openstack/kolla/+/77599509:40
openstackgerritMark Goddard proposed openstack/kolla stable/train: Fix several issues (Train)  https://review.opendev.org/c/openstack/kolla/+/77460209:43
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/train: Update String type for Monasca ES template  https://review.opendev.org/c/openstack/kolla-ansible/+/77591009:44
*** rohit02 has quit IRC09:44
*** k_mouza has joined #openstack-kolla09:51
hrwelo09:51
dardeleanHi09:53
dardeleanGlance is not deployed in HA if it has a file backed storage, right?09:54
dardeleaneven if I have 3 controllers09:54
*** rohit02 has joined #openstack-kolla10:06
yoctozeptodardelean: right10:19
yoctozeptothis can be overridden but will only cause misery :P10:20
dardeleanyoctozepto thanks, what about cinder-volume? I have it with ceph also but there is only one cinder-volume service in compute service list10:22
dardeleandoes it have internal HA machanism?10:22
openstackgerritMerged openstack/kayobe master: Test building seed deployment images in the seed job  https://review.opendev.org/c/openstack/kayobe/+/77275110:45
*** e0ne has quit IRC10:53
hrwhttps://michael-prokop.at/blog/2021/02/16/how-to-properly-use-3rd-party-debian-repository-signing-keys-with-apt/ - I am considering changing how we are using apt keys11:03
*** e0ne has joined #openstack-kolla11:05
dardeleanis there any docs on update (not upgrade) procedure? for example if I am on 9.1 and want to move to 9.311:13
openstackgerritPierre Riteau proposed openstack/kayobe stable/train: Fix building CentOS 8 IPA images on stable/train  https://review.opendev.org/c/openstack/kayobe/+/77394411:19
openstackgerritPierre Riteau proposed openstack/kayobe stable/train: Test building seed deployment images in the seed job (CentOS 8 only)  https://review.opendev.org/c/openstack/kayobe/+/77385311:20
*** kevko_ has joined #openstack-kolla11:37
*** kevko__ has joined #openstack-kolla11:38
*** kevko_ has quit IRC11:40
yoctozeptodardelean: it should be as simple as re-running the 'deploy'11:57
yoctozeptojust always read11:57
yoctozepto'upgrade notes'11:57
yoctozeptoeven for minor releases11:57
dardeleanchange the tag in globals and rerun deploy, got it, thnaks11:57
yoctozeptodardelean: you should have more than one cinder-volume for HA11:57
yoctozeptodardelean: there is no tag in globals for a minor release11:58
yoctozeptooh11:58
yoctozeptoI know what you mean11:58
yoctozeptoyou should not be using these versioned images11:58
yoctozeptothey are frozen at some point11:58
dardeleanhmmm, so you advise against changing the "openstack_tag" in globals are rerun deploy?11:59
dardeleani have 9.1 atm set, would like to try 9.312:00
yoctozeptodardelean: you should first update the kolla-ansible12:01
yoctozeptoand unset the openstack_tag at all12:01
yoctozeptothat will get you the latest images with latest ansible code12:01
dardeleanyoctozepto but still on the same train release, right?12:02
dardeleanor whatever release I am on12:02
*** e0ne has quit IRC12:06
openstackgerritPierre Riteau proposed openstack/kayobe stable/train: Test building seed deployment images in the seed job (CentOS 8 only)  https://review.opendev.org/c/openstack/kayobe/+/77385312:11
*** k_mouza has quit IRC12:16
*** k_mouza has joined #openstack-kolla12:17
openstackgerritPierre Riteau proposed openstack/kayobe stable/train: Fix building CentOS 8 IPA images on stable/train  https://review.opendev.org/c/openstack/kayobe/+/77394412:18
openstackgerritPierre Riteau proposed openstack/kayobe stable/train: Test building seed deployment images in the seed job (CentOS 8 only)  https://review.opendev.org/c/openstack/kayobe/+/77385312:18
*** kevko__ has quit IRC12:26
*** kevko has joined #openstack-kolla12:26
openstackgerritEgon Rijpkema proposed openstack/kolla-ansible master: keep X-Forwarded-Proto of exterbal ssl termination.  https://review.opendev.org/c/openstack/kolla-ansible/+/75836112:31
kevkohi, yoctozepto, mgoddard will you have a time to re-review my proxysql stack ? I think it's in good shape ..so can we review it again ... what is a blocker ?12:34
*** jbadiapa has joined #openstack-kolla12:49
*** jbadiapa has quit IRC12:50
*** jbadiapa has joined #openstack-kolla12:51
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Add missing octavia-driver-agent  https://review.opendev.org/c/openstack/kolla-ansible/+/76187212:53
*** amoralej is now known as amoralej|lunch13:04
kevkoyoctozepto, mgoddard : Here ? someone ?13:05
*** rohit02 has quit IRC13:06
*** rohit02 has joined #openstack-kolla13:06
*** e0ne has joined #openstack-kolla13:09
*** skramaja has quit IRC13:34
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Remove Monasca Log Transformer  https://review.opendev.org/c/openstack/kolla-ansible/+/76990013:35
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Disable Monasca Log Metrics service by default  https://review.opendev.org/c/openstack/kolla-ansible/+/76990113:35
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Support disabling Monasca alerting pipeline  https://review.opendev.org/c/openstack/kolla-ansible/+/76990213:35
mgoddardkevko: I'll try to make time for it soon13:35
*** rohit02 has quit IRC13:55
kevkomgoddard: ok13:56
kevkomgoddard: btw, we delivered proxysql with kolla last saturday to production :) and working nice13:57
*** amoralej|lunch is now known as amoralej14:00
mgoddardkevko: nice, congrats14:10
kevkomgoddard: well, we were upgrading for 18 hours :D14:11
openstackgerritPierre Riteau proposed openstack/kayobe stable/train: Fix copying Swift ring files  https://review.opendev.org/c/openstack/kayobe/+/77390014:12
*** e0ne has quit IRC14:28
*** e0ne has joined #openstack-kolla14:29
kevkoguys, could you help me with this http://paste.openstack.org/show/802736/ I just need to create a dict from this list of dicts ..but want to have key: value   ,   so output_key : value -> this will be key   , ouput_value: value -> this will be value14:31
kevkoand I don't mind the description14:32
yoctozeptokevko: congrats, you truly are a hard worker!14:35
yoctozeptodardelean: yes, the same14:36
yoctozeptoit is designed as no-touch-unless-you-have-to-and-know-what-you-are-doing thingy14:37
*** cyberkev has joined #openstack-kolla14:38
cyberkevHello, I'm attempting a multinode deployment.   Overcome a few hurdles (mostly ssh passphrase/sudo related) and now I'm at the point of doing   kolla-ansible -i ./multinode prechecks14:40
cyberkevThe error I'm getting is:-   Hostname has to resolve uniquely to the IP address of api_interface14:41
cyberkevtried with and without it on 127.0.0.1 and its got it in hostfiles under hosts IP addresses on all 3 nodes14:41
cyberkevany pointers ?   I'm sure its something simple DNS related but cant put my finger on it14:41
kevkoyoctozepto: manual changes by client was hell ..but on the end it was successfull without downtime14:43
kevkoyoctozepto: can u advise me with above paste.openstack.org please ?14:44
*** Mareo has joined #openstack-kolla14:47
*** zzzeek has quit IRC14:49
*** zzzeek has joined #openstack-kolla14:50
mgoddardmgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak parallax Fl1nt14:50
mgoddard^ meeting in 1014:50
hrwshit. right14:52
hrwsorry, terrible day today14:52
hrwwill attend14:52
*** rafaelweingartne has joined #openstack-kolla14:54
*** brinzhang has quit IRC14:55
*** brinzhang has joined #openstack-kolla14:55
*** brinzhang has quit IRC14:57
*** brinzhang has joined #openstack-kolla14:58
mgoddard#startmeeting kolla15:00
openstackMeeting started Wed Feb 17 15:00:32 2021 UTC and is due to finish in 60 minutes.  The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: kolla)"15:00
openstackThe meeting name has been set to 'kolla'15:00
mgoddard#topic rollcall15:00
*** openstack changes topic to "rollcall (Meeting topic: kolla)"15:00
yoctozepto\o/15:00
mgoddard\o15:00
rafaelweingartne\o15:00
risson\o15:01
hrw /o\15:02
hrw /ō\ even15:03
mgoddard#topic agenda15:03
*** openstack changes topic to "agenda (Meeting topic: kolla)"15:03
mgoddard* Roll-call15:03
mgoddard* Announcements15:03
mgoddard* Review action items from the last meeting15:03
mgoddard* CI status15:03
mgoddard* Review requests15:04
mgoddard* Keystone federation & HAProxy session stickiness https://review.opendev.org/c/openstack/kolla-ansible/+/695432/56/ansible/roles/keystone/defaults/main.yml15:04
mgoddard* Dockerhub pull limits: publish weekly master images? https://review.opendev.org/c/openstack/kolla/+/77599515:04
mgoddard* Wallaby release planning15:04
mgoddard#topic announcements15:04
*** openstack changes topic to "announcements (Meeting topic: kolla)"15:04
mgoddardNone from me15:04
mgoddard#topic Review action items from the last meeting15:05
*** openstack changes topic to "Review action items from the last meeting (Meeting topic: kolla)"15:05
mgoddardmgoddard fix bifrost on Train15:06
mgoddardFixing bifrost itself proved tricky, but there is a part of the bifrost fix that we can apply via config15:06
mgoddardI added the fix to https://review.opendev.org/c/openstack/kolla/+/774602, which seems to have worked15:07
mgoddardbut now there are other issues15:07
mgoddardsomething to do with the elasticsearch 5.x repo15:07
mgoddardI'm wondering if it's a mirror sync issue15:07
yoctozeptoyeah, it fails randomly15:07
yoctozeptobut weirdly15:07
mgoddardfails every time15:08
mgoddardon ubuntu source15:08
yoctozeptohmm, but that ubuntu binary built15:08
yoctozeptosomething fishy I would say15:08
mgoddardyes15:08
mgoddardretry tomorrow15:08
yoctozeptolet's leave it be for today15:08
yoctozeptoyes15:08
mgoddard#topic CI status15:09
*** openstack changes topic to "CI status (Meeting topic: kolla)"15:09
mgoddardGenerally looks better15:10
mgoddardkolla failing in Train & earlier due to aforementioned issues15:10
mgoddard#topic Review requests15:11
*** openstack changes topic to "Review requests (Meeting topic: kolla)"15:11
mgoddardDoes anyone have a patch they would like to be reviewed?15:11
rissonYep! https://review.opendev.org/c/openstack/kolla-ansible/+/77288615:11
rissonIt has been discussed here before between you and Mr_Freezeex15:12
hrwhttps://review.opendev.org/c/openstack/kolla/+/772841 from me (centos 8 stream)15:12
kevkoyeah, https://review.opendev.org/q/hashtag:%22proxysql%22+(status:open%20OR%20status:merged)  :)15:12
kevko:D15:12
hrwkevko: could you look at https://review.opendev.org/c/openstack/kolla/+/772479 one?15:13
kevkowill15:14
mgoddardrisson: I've added review priority +1 label to the patch15:15
rissonthanks!15:15
mgoddardadded RP+1 to those15:16
mgoddardAnyone else?15:16
mgoddardI'm going to request the same as last week,15:17
mgoddardhttps://review.opendev.org/c/openstack/kolla-ansible/+/69543215:17
mgoddardkeystone federation15:17
mgoddardon that topic...15:18
mgoddard#topic Keystone federation & HAProxy session stickiness15:18
*** openstack changes topic to "Keystone federation & HAProxy session stickiness (Meeting topic: kolla)"15:18
mgoddard#link https://review.opendev.org/c/openstack/kolla-ansible/+/695432/56/ansible/roles/keystone/defaults/main.yml15:18
mgoddardrafaelweingartne: hi15:18
rafaelweingartneHello15:19
rissonWe applied that patch on our deployment and we needed the balance source option for session stickiness as explained by Pedro in his comment15:20
mgoddardWe have one main point of contention in the keystone federation patch: session stickiness15:20
mgoddardthe aim here is to talk it out15:20
*** k_mouza has quit IRC15:20
mgoddardargh, Fl1nt isn't here15:20
rafaelweingartneExactly, we explained a few times for different people, and probably when Flint asked the same, we just jumped over the question.15:20
yoctozeptoI saw the explanation, I am buying it15:21
mgoddardI would say that he's done quite a good job of explaining himself now, and I haven't seen a decent response yet, although maybe I missed it15:21
rafaelweingartneA few days ago flint explicitly showed what he wanted to address there, which is the "sticky session mode" that is being used, and not the use of the sticky session per se15:21
rissonyes, sticky sessions should be achieved based on the user's cookies, not with `balance source`15:22
mgoddardright, I think we're in agreement that stickiness is required15:22
rissonI'm not sure if HAProxy permits that though15:22
rafaelweingartneWe do not actually mind changing that, if that had been said, we would have done it.15:22
rafaelweingartnerisson: we also do not know that15:22
rafaelweingartnewe started experimenting with some options, we normally only use source, because it is easier :)15:22
rafaelweingartneto avoid more problems, what alternatives to source would you guys prefer?15:23
mgoddardFl1nt made a comment on PS57: https://review.opendev.org/c/openstack/kolla-ansible/+/695432/5615:24
rafaelweingartnecustom cookie based sticky session? Session ID? a configurable load balancing mode (least connection/round-robin)?15:24
rissonthere's an rdp-cookie option that can be passed to `balance`, not sure if it is what we're looking for15:24
rafaelweingartneyes, that seems to be the implementation Flint prefers15:25
mgoddardIt would be better to use roundrobin or leastconn with a session cookie, that would let HAProxy to appropriately let you contact the correct backend if the node you were connecting from died or if the lease of your client expired.15:25
mgoddardAdditionally, there is an optional extra option that can be used to be more deterministic on the way HAproxy is handling the backend chosen for your session which is: hash-type that can be set to many options such as consistent / map-based / sdbm, etc (see haproxy doc about that).15:25
mgoddardWe use consistent on our side but that could be something up to the operators to choose.15:25
mgoddardquoting Fl1nt there15:25
rafaelweingartneyes15:25
mgoddardTBH, balance source is what we use for horizon, so it's not going to be making things any worse15:26
rafaelweingartneactually, it does not make any difference15:26
yoctozepto^ exactly mgoddard15:26
rafaelweingartneyou know, the sticky session is only needed during the authentication phase to validate the token generated by the IdP15:26
rissonwhat was the argument against balance source again?15:26
yoctozeptoexactly, it should be either short enough to be irrelevant15:26
rafaelweingartnethat is the moment we need the sticky session, after that, it does not make much difference15:26
yoctozeptoor slow enough that it needs fixing anyhow elsewhere15:26
openstackgerritArthur Outhenin-Chalandre proposed openstack/kolla-ansible master: Add `kolla_externally_managed_cert` option  https://review.opendev.org/c/openstack/kolla-ansible/+/77288615:27
yoctozeptobut the problem is obviously that 'balance source' stays with us forever15:27
yoctozeptoin that token verifications15:27
yoctozeptostill hit it15:27
mgoddardvery old blog with info on using haproxy to insert cookies: https://www.haproxy.com/blog/load-balancing-affinity-persistence-sticky-sessions-what-you-need-to-know/#session-cookie-setup-by-the-load-balancer15:28
rissondamn, review priority has been removed from https://review.opendev.org/c/openstack/kolla-ansible/+/77288615:28
yoctozeptorisson: it's baaaack15:29
rafaelweingartneif the node that initiated the authentication dies, the user will get an error when presenting this token to other mod-OIDC instances15:29
rissonyes, but they can just try again and it'll work, right?15:30
rissonthe proper way of fixing this would not be using apache2 for authentication, but keystone doing it and storing its state in its db15:30
mgoddardhow about this for a path forward:15:31
rafaelweingartneyes,15:31
rafaelweingartnebut source-balance would do the same15:31
mgoddardkeep the current patch with balance source, enabled only with federation15:31
rafaelweingartneit validates if the node is up, before sending it to the backend15:32
mgoddardconsider switching to another method for horizon and keystone together, as a follow up15:32
rafaelweingartnethe only difference between sticky session with source-balance and the others is the "more optimal" balance of load between nodes15:33
rafaelweingartneconsidering that we could have one IP (NAT) with many different users15:33
mgoddardright15:34
mgoddardwith a central service such as keystone that is something worth considering15:34
*** shyamb has joined #openstack-kolla15:34
mgoddardthoughts on my suggestion?15:34
rissonI think that going with balance source is a good idea for now15:35
yoctozeptomgoddard: love it15:35
mgoddardwonderful15:36
*** shyam89 has joined #openstack-kolla15:36
rafaelweingartneI like your suggestion15:36
mgoddardlet's aim to get it merged before the next meeting15:36
rafaelweingartnebecause we have not extensively tested this with other balance methods15:36
risson^ this15:36
mgoddardand rafaelweingartne and Pedro can stop pulling their hair out :)15:37
rafaelweingartne:)15:37
rafaelweingartnewe do understand that the patch is huge. I also hate it15:37
rafaelweingartnebut, it was the first load of code to handle federation in Kolla-ansible15:37
mnasiadkaso next time make smaller patches :)15:37
mgoddardI've seen bigger ;)15:37
rafaelweingartnethe improvements will be much easier15:37
mnasiadkaaround haproxy balance source - that's a bit non-ideal solution, but I guess we can live with it for a while.15:38
mgoddardI think the main obstacle is the subject matter, rather than the size of the code15:38
yoctozeptoI'll re-review this week15:38
mgoddardanyways, we have some level of agreement, let's move on15:38
yoctozeptobut I expect to merge it15:38
mgoddardthanks for joining rafaelweingartne15:38
mgoddard#topic Dockerhub pull limits: publish weekly master images?15:39
*** openstack changes topic to "Dockerhub pull limits: publish weekly master images? (Meeting topic: kolla)"15:39
mgoddard#link https://review.opendev.org/c/openstack/kolla/+/77599515:39
yoctozeptoy not15:39
rafaelweingartneawesome thanks guys15:39
mgoddardpriteau and I were discussing the pull limit issue15:39
yoctozeptoit sucks15:40
mgoddardwhat if we publish master images weekly and daily?15:40
mgoddardsome projects could use the weekly images in CI15:41
mgoddarde.g. kayobe15:41
mgoddardpossibly kolla-ansible15:41
mgoddardhow often would we get hit by broken images, or blocked by images being out of date?15:42
priteauHard to say. I suppose if we get blocked we could override CI to use daily.15:42
mgoddardright15:43
yoctozeptoI think we need to add ourselves the ability to publish on demand15:43
mgoddardwell, maybe for broken images15:43
yoctozeptowe can publish on specific commits we merge15:43
yoctozeptofugly but worky15:43
mgoddardprobably not just for a feature that depends on images15:43
mgoddardor we could publish twice-weekly15:43
mgoddardthat could be a better compromise15:44
yoctozeptothat's getting overly complicated15:44
mgoddardnot really15:44
yoctozeptoSunday feels better15:44
mnasiadkaor we could build on every deployment, how long is the build?15:44
mnasiadka(on master only)15:44
mgoddardit just feels wrong15:44
yoctozeptofeels wrong15:44
mnasiadkaI think often we are dependent on something failing in the image15:44
yoctozeptobut might make CI saner15:44
mnasiadkaand then we're stuck for a week?15:44
yoctozeptowe don't build all the images15:45
yoctozeptobut indeed it might quite a bit of extra work15:45
mgoddardwell, like yoctozepto said we'd need an override15:45
yoctozeptoyeah, we can practice the override15:45
yoctozeptoempty commits with metadata are pretty cheap15:45
yoctozeptowe can publish from other pipelines than periodic15:46
yoctozeptojust not check15:46
yoctozeptoas it runs untrusted code15:46
mgoddardwhich pipeline would be appropriate?15:46
yoctozeptoon that note, remember W+1 makes the change trusted15:46
mgoddardgate?15:46
yoctozeptonope, it should be after gating15:46
yoctozeptoeither post or promote15:47
yoctozeptobut we should really keep the images built in gate15:47
yoctozeptofor publishing later15:47
yoctozeptogate is technically fine but we all know we can end up overpublishing :-)15:47
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Support bypassing Monasca Log API for control plane logs  https://review.opendev.org/c/openstack/kolla-ansible/+/77621915:48
mgoddardalternatively we have a nightly publish job that is a noop unless:15:48
mnasiadkawell, can we publish master to quay.io or github? will it work better?15:48
mgoddard* it is a one of the selected publishing days15:48
yoctozeptomnasiadka: yeah, we could test that as well15:49
yoctozeptolots of ideas; need triage :-)15:49
mgoddard* or we modify zuul config to override15:49
mnasiadkayoctozepto: I just don't like those zuul dances, because it seems like a lot of work with random success :)15:49
yoctozeptomnasiadka: i feel you15:50
hrwwhat is wrong with each-day publish? do we mirror images on CI?15:50
mgoddardhrw: new images -> invalidated registry caches -> docker pull -> pull request limit15:50
mgoddardhrw: we now do weekly publishing on stables, and it has helped a lot15:51
hrwcan we publish daily to some opendev infra registry?15:51
hrwand then use them on CI?15:51
mgoddardwe have discussed all these solutions before15:52
mgoddardthe problem is, I don't see anyone putting in time to implement them15:52
mnasiadkahrw: and that solution is nice, but requires somebody to work with infra to get this implemented15:52
mgoddardso this topic was aiming to be another stop-gap measure15:52
yoctozeptoyeah15:52
hrwk15:52
mgoddardwe can very easily reduce our publishing frequency15:53
yoctozeptoso let's do it15:53
mgoddardalthough it does come with gotchas15:53
mgoddardas discussed :)15:53
yoctozeptoand cry* when we get blocked15:53
yoctozepto* discuss15:53
yoctozeptobetter than continuous rechecks15:54
yoctozeptoand now gimme open discussion15:54
mgoddard#topic open discussion15:54
*** openstack changes topic to "open discussion (Meeting topic: kolla)"15:54
yoctozeptohrw: I like https://michael-prokop.at/blog/2021/02/16/how-to-properly-use-3rd-party-debian-repository-signing-keys-with-apt/15:54
yoctozeptoit is essentially what we have in centos15:54
yoctozeptoand I was wondering once if we could have the same for debuntu15:55
yoctozeptoso I'm all in15:55
hrwyoctozepto: I looked closer into it and can have it for Debian. Ubuntu uses 3 keys directly from keyserver so gnupg still needed15:55
yoctozeptoperhaps we can override that as well15:55
yoctozeptobut a mixed solution is fine for now15:55
yoctozeptodo it everywhere it's simple15:56
hrwyoctozepto: https://paste.centos.org/view/e526b842 is start of cleanup15:56
yoctozepto++15:56
yoctozeptolet it continue15:56
openstackgerritMark Goddard proposed openstack/kolla master: CI: publish images on a weekly basis  https://review.opendev.org/c/openstack/kolla/+/77622115:58
* hrw out16:01
mgoddardall done for this week16:01
mgoddardthanks16:01
yoctozeptothanks16:01
mgoddard#endmeeting16:01
*** openstack changes topic to "IRC meetings on Wednesdays @ 15:00 UTC - agenda @ https://goo.gl/OXB0DL | Whiteboard: https://bit.ly/2MM7mWF | IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b"16:01
openstackMeeting ended Wed Feb 17 16:01:22 2021 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-02-17-15.00.html16:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-02-17-15.00.txt16:01
openstackLog:            http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-02-17-15.00.log.html16:01
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Configure Monasca Grafana user roles  https://review.opendev.org/c/openstack/kolla-ansible/+/66767116:03
*** rafaelweingartne has quit IRC16:04
*** k_mouza has joined #openstack-kolla16:14
*** shyam89 has quit IRC16:19
*** shyamb has quit IRC16:19
*** alpha2385 has quit IRC16:24
*** amoralej is now known as amoralej|off17:17
openstackgerritMerged openstack/kolla-ansible stable/ussuri: Update String type for Monasca ES template  https://review.opendev.org/c/openstack/kolla-ansible/+/77590917:31
openstackgerritMerged openstack/kolla-ansible master: Lint and fix renos  https://review.opendev.org/c/openstack/kolla-ansible/+/75937017:31
*** dougsz has quit IRC17:32
*** bengates_ has quit IRC17:37
*** rpittau is now known as rpittau|afk17:41
*** rohit02 has joined #openstack-kolla17:52
*** gfidente is now known as gfidente|afk18:03
*** k_mouza has quit IRC18:14
*** bengates has joined #openstack-kolla18:15
*** maharg101 has quit IRC18:16
*** bengates has quit IRC18:19
*** jonaspaulo has joined #openstack-kolla18:29
*** andrewbonney has quit IRC18:33
*** rohit02 has quit IRC18:33
*** jonaspaulo has quit IRC18:37
openstackgerritAna Peric proposed openstack/kolla-ansible master: Avoid errors in apparmor chrony profile removal in Debian  https://review.opendev.org/c/openstack/kolla-ansible/+/77625218:38
openstackgerritAna Peric proposed openstack/kolla-ansible master: Avoid errors in apparmor chrony profile removal in Debian  https://review.opendev.org/c/openstack/kolla-ansible/+/77625218:42
*** e0ne has quit IRC18:46
*** jonaspaulo has joined #openstack-kolla18:51
openstackgerritAna Peric proposed openstack/kolla-ansible master: Avoid errors in apparmor chrony profile removal in Debian  https://review.opendev.org/c/openstack/kolla-ansible/+/77625218:52
openstackgerritAna Peric proposed openstack/kolla-ansible master: chronyd crash loop if Debian server is rebooted  https://review.opendev.org/c/openstack/kolla-ansible/+/77625619:11
*** e0ne has joined #openstack-kolla19:48
*** cah_link has quit IRC19:49
*** luksky has quit IRC19:51
*** luksky has joined #openstack-kolla19:52
openstackgerritMerged openstack/kolla-ansible stable/victoria: Update String type for Monasca ES template  https://review.opendev.org/c/openstack/kolla-ansible/+/77590820:04
*** maharg101 has joined #openstack-kolla20:12
*** jonaspaulo has quit IRC20:14
*** maharg101 has quit IRC20:17
*** alpha23 has joined #openstack-kolla20:28
*** kevko has quit IRC21:07
*** e0ne has quit IRC21:24
*** vishalmanchanda has quit IRC21:37
dmsimardo/ heads up: ansible==3.0.0 releasing tomorrow21:44
*** gmann is now known as gmann_afk21:51
*** alpha23 has quit IRC22:01
*** cyberkev has quit IRC23:01
*** alpha23 has joined #openstack-kolla23:22
*** gmann_afk is now known as gmann23:22
« Tuesday, 2021-02-16 Index Thursday, 2021-02-18 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!