| opendevreview | Merged openstack/kolla-ansible stable/wallaby: watcher: add missing become for copying configs https://review.opendev.org/c/openstack/kolla-ansible/+/803287 | 03:03 |
|---|---|---|
| opendevreview | Merged openstack/kolla-ansible stable/victoria: watcher: add missing become for copying configs https://review.opendev.org/c/openstack/kolla-ansible/+/803288 | 03:03 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/victoria: Add support for CentOS 8 Stream https://review.opendev.org/c/openstack/kolla/+/792763 | 04:33 |
| opendevreview | Michal Nasiadka proposed openstack/kayobe master: seed: Allow disabling of SNAT configuration https://review.opendev.org/c/openstack/kayobe/+/802684 | 04:38 |
| pvh_sa | hello there... yesterday the networking for one of my VM hosts became: the DHCP requests stopped getting an answer. eventually logging in via the console and manually setting the IP worked as a workaround. now this morning I've lost networking on 2 more hosts on the same hypervisor. these are on a different network this time (but strangely it is the same network that I moved yesterday's bad host to). this is a kolla-deployed version of queens. | 05:29 |
| pvh_sa | (network is TUN based with openvswitch) | 05:30 |
| pvh_sa | (correct that, its just 1 host that is offline) | 05:31 |
| mnasiadka | pvh_sa: kolla-ansible is just a deployment project, if you have problems with Neutron - feel free to ask on #openstack-neutron - but since you're using queens - don't expect much. | 06:02 |
| pvh_sa | thanks. haha I see that an upgrade is in my future. | 06:03 |
| pvh_sa | on the question of upgrading this all... the controller is running ubuntu 16.04 so its got python 2.7. the kolla & kolla-ansible packages are version 6.0.0. I presume that more recent versions require Python3? | 06:25 |
| mnasiadka | train is the last one to support python 2.7, and it also supports python 3 | 06:26 |
| mnasiadka | Well, in Ubuntu it might be that train is only python3, but you would need to check the docs | 06:27 |
| pvh_sa | ok... I'm navigating using the support matrix here: https://docs.openstack.org/kolla-ansible/train/user/support-matrix - looks like an upgrade to 18.04 is needed (on at least the controller) | 06:30 |
| pvh_sa | which docs should I be checking for details of e.g. python3 support? I can find generic instructions on upgrading (https://docs.openstack.org/kolla-ansible/train/user/operating-kolla.html#upgrade-procedure) but not on these specifics | 06:35 |
| mnasiadka | well, train requires python3 in Ubuntu - https://github.com/openstack/kolla-ansible/blob/804f58e063331cc4c73926e8baf71f736b650130/ansible/group_vars/all.yml#L1145 | 06:36 |
| pvh_sa | and secondly... should the upgrade procedure step through each version in turn? i.e. queens to rocky to train to ussuri? | 06:37 |
| mnasiadka | Yes, one by one | 06:37 |
| mnasiadka | rocky to stein to train to ussuri | 06:37 |
| pvh_sa | thanks, I'll plan an upgrade of the hosts to 18.04 and then get going with upgrades... | 06:38 |
| *** rpittau|afk is now known as rpittau | 07:13 | |
| hrw | but do not stop on ussuri. end in wallaby | 07:55 |
| hrw | and 20.04 | 07:55 |
| clayjar | Hi all, Greg from Pawsey in .au here, just starting out trying to offer some contributions. I've never used gerritt before, but thought I'd have a crack at some code review to see if I can help there. | 08:26 |
| clayjar | I've gone to https://review.opendev.org/admin/repos/q/filter:kolla and found https://review.opendev.org/plugins/gitiles/openstack/kolla-ansible ... but now what? I guess I'm looking for advice on a workflow, because I'm presuming it's a system designed to assist code review. We use Bitbucket and I figure there will be some parallels. | 08:29 |
| mgoddard | Hi clayjar, great to see you here | 08:29 |
| mgoddard | clayjar: I'd suggest getting familiar with the search bar syntax | 08:30 |
| mgoddard | clayjar: so, from https://review.opendev.org/dashboard/self | 08:30 |
| mgoddard | is:open AND (project:openstack/kolla OR project:openstack/kolla-ansible) | 08:31 |
| mgoddard | that's a good query to start | 08:31 |
| mgoddard | there is also the whiteboard: https://etherpad.opendev.org/p/KollaWhiteBoard | 08:31 |
| mgoddard | and the first topic is 'gerrit dashboards' | 08:32 |
| mgoddard | which include some pre-canned queries | 08:32 |
| clayjar | nice | 08:32 |
| mgoddard | if you're looking for small patches, the size column is useful | 08:33 |
| mgoddard | I think XS corresponds to size:<10 | 08:33 |
| clayjar | Okay, so looking at 803289, which looks very straightforward and sane (especially since it's a cherry pick from a merged master PR). I want to say yes this looks fine. I +1'd the Code Review. Question is, how deeply am I reviewing it? Some code changes will have far reaching implications that require knowledge of the whole lot, much won't, but how do I know which? | 08:42 |
| clayjar | Oh, and I should clarify coding conventions (am I showing my lack of practical code contribution experience?) - is it typical to push changes to master, then cherry pick backport them to various version branches? Is it near-universal? | 08:47 |
| yoctozepto | And that is a very good question and I just love the fact that you are asking yourself that. The strategy would be to focus on the parts that you understand the best. I assume you are a user of k-a already - focus on patches dealing with general stuff and projects that you deployed (especially in an applicable config). | 08:47 |
| mgoddard | yoctozepto +1 | 08:47 |
| yoctozepto | Regarding cherrypicks - in OpenStack we strive to first merge on master (unless the patch is required only on a stable branch - it happens) and then on stable branches. | 08:48 |
| clayjar | Yep, a user, working toward getting our dev cluster into prod, or rather, migrating our prod Puppet-OpenStack cluster to k-a. | 08:48 |
| clayjar | Okay good, I'm familiar with that convention. | 08:48 |
| yoctozepto | Very well, I started my contribution journey at a similar stage of deployment readiness. | 08:48 |
| yoctozepto | ... and look at me now. :-) | 08:48 |
| mgoddard | clayjar: I would probably suggest limiting yourself to changes on the master branch for now (branch:master) | 08:49 |
| clayjar | got it | 08:49 |
| yoctozepto | mgoddard +1 | 08:49 |
| mgoddard | since most backports have already been through a review process, it's mostly about applicability and fiddly details in the differences between branches | 08:50 |
| mgoddard | clayjar: I think I sent you the kolla contributor guide previously, but if not, https://docs.openstack.org/kolla/latest/contributor/index.html | 08:51 |
| clayjar | yep, that's how I ended up at an ugly version of review.opendev.org - this interface is much nicer | 08:51 |
| mgoddard | doh | 08:52 |
| mgoddard | in the review dashboard, these sections are probably best to look at: " Newer (<1wk) Open Patches (limit 100) ", " Older (>1wk) Open Patches Passing Zuul Tests (limit 100)" | 08:53 |
| clayjar | Is that some syntax you just used there? '-age:1w' works, but not sure about ^ | 08:58 |
| yoctozepto | Mark means the pre-made dashboards from the whiteboard links | 08:59 |
| yoctozepto | https://etherpad.opendev.org/p/KollaWhiteBoard | 09:00 |
| yoctozepto | in "Gerrit dashboards" | 09:00 |
| clayjar | Colour me confused still sorry - that heading only has 3 tiny.cc links, to k k-a and kayobe dashboards | 09:01 |
| hrw | clayjar: which give you very nice gerrit dashboard | 09:03 |
| hrw | with several lists on one screen. try | 09:03 |
| clayjar | Okay, I'm catching up now | 09:03 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: Upgrade prometheus elasticsearch exporter to 1.2.1 https://review.opendev.org/c/openstack/kolla/+/800499 | 09:05 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible stable/wallaby: Do not try to get container facts on deployment host https://review.opendev.org/c/openstack/kolla-ansible/+/797584 | 09:07 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible stable/wallaby: Do not run timesync checks on deployment host https://review.opendev.org/c/openstack/kolla-ansible/+/797584 | 09:12 |
| opendevreview | Mark Goddard proposed openstack/kolla stable/victoria: CentOS Stream: update kolla-ansible Zuul jobs https://review.opendev.org/c/openstack/kolla/+/798892 | 09:15 |
| yoctozepto | clayjar: as you are from AU, I'm guessing it will be uncomfortable for you to join our team meeting at 3 PM UTC? | 09:18 |
| yoctozepto | (today and every Wednesday) | 09:18 |
| clayjar | Yep :/ | 09:18 |
| yoctozepto | note we might consider rescheduling as the current time is pretty arbitrary and we generally have more flexibility | 09:20 |
| clayjar | Understood. At some point I may try to form a habit of napping during a Wednesday and staying up for such things, because the joy of the WFH gear I now have is balanced by travel budgets goin to $0 for the foreseeable future, and .eu and .us is where all the action is. | 09:20 |
| yoctozepto | I see | 09:21 |
| clayjar | Meanwhile, COB now, and I've followed through on setting an hour aside for this a total of twice now (: See you next week. | 09:21 |
| yoctozepto | see you! | 09:22 |
| hrw | keeping up with AU/NZ while being in EU was always hard | 09:28 |
| opendevreview | Ilya Popov proposed openstack/kolla-ansible master: Extra var ironic_enable_keystone_integration added. https://review.opendev.org/c/openstack/kolla-ansible/+/793664 | 09:39 |
| yoctozepto | mgoddard, mnasiadka: I just realised there is an old but gold change of mine that lies with no merge conflict and no reviews https://review.opendev.org/c/openstack/kolla-ansible/+/773246 - please have a look and comment (or just merge away) | 09:42 |
| hrw | https://review.opendev.org/c/openstack/kolla/+/784923 (apt keys) and https://review.opendev.org/c/openstack/kolla/+/785569 (drop rhel) soon will need 'feature freeze, moved to Yeti' | 09:49 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: Remove references to resolv.conf workaround https://review.opendev.org/c/openstack/kayobe/+/803459 | 10:35 |
| kevko | hrw: when is feature freeze ? | 11:11 |
| kevko | btw, anyone for second +2 for this trivial fix ? https://review.opendev.org/c/openstack/kolla-ansible/+/803031 | 11:12 |
| hrw | kevko: +2+w | 11:13 |
| kevko | hrw: thank you .. | 11:13 |
| hrw | kevko: ff is still in future. | 11:13 |
| hrw | kevko: both those patches gets old and were done during wallaby cycle | 11:13 |
| kevko | thanks god | 11:14 |
| kevko | hrw: yeah i know, i remember it very well | 11:14 |
| kevko | hrw: unfortunatelly i have +1 only and you have it already from me for long time:) | 11:15 |
| yoctozepto | kevko, hrw: fwiw, the apt keys change has 6x +2 from me so I guess it could have been merged at least 3 times already :-) | 11:17 |
| yoctozepto | sorry, the RHEL one | 11:18 |
| hrw | ;) | 11:20 |
| kevko | and what about second +2 for this https://review.opendev.org/c/openstack/kolla-ansible/+/797646 ? :P | 11:21 |
| * kevko feel a chance | 11:21 | |
| yoctozepto | approved | 11:22 |
| kevko | thank you :) | 11:23 |
| wuchunyang | hi，a small ps need review too. need second +2 | 11:49 |
| wuchunyang | https://review.opendev.org/c/openstack/kolla-ansible/+/799857 | 11:49 |
| wuchunyang | https://review.opendev.org/c/openstack/kolla-ansible/+/801625 | 11:49 |
| hrw | wuchunyang: I think that it should be rewritten in a way to have enable_rabbitmq_external | 11:59 |
| wuchunyang | Add a new variable enable_rabbitmq_external ? | 12:02 |
| hrw | wuchunyang: I wrote comment in review. hard to tell for me which way to sort it | 12:03 |
| wuchunyang | hrw thanks , we don't have enable_rabbitmq_external so far. | 12:14 |
| hrw | I know | 12:15 |
| opendevreview | Merged openstack/kolla-ansible master: Fix freezed spice console in horizon https://review.opendev.org/c/openstack/kolla-ansible/+/803031 | 12:21 |
| opendevreview | Merged openstack/kolla-ansible stable/victoria: Trivial fix horizon's healthcheck when SSL turned on https://review.opendev.org/c/openstack/kolla-ansible/+/803286 | 12:40 |
| opendevreview | Merged openstack/kolla-ansible stable/wallaby: Trivial fix horizon's healthcheck when SSL turned on https://review.opendev.org/c/openstack/kolla-ansible/+/803282 | 12:40 |
| opendevreview | Merged openstack/kolla-ansible master: Do not run timesync checks on deployment host https://review.opendev.org/c/openstack/kolla-ansible/+/797646 | 12:40 |
| opendevreview | Merged openstack/kolla-ansible master: Fix deployment failure when kolla_dev_mod is enabled https://review.opendev.org/c/openstack/kolla-ansible/+/801625 | 13:00 |
| yoctozepto | hrw: reviewed apt patch thoroughly; I like the approach; wdyt about streamlining the keys for our purposes and keeping deprecated backward-compat for users? | 13:08 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible stable/wallaby: Fix freezed spice console in horizon https://review.opendev.org/c/openstack/kolla-ansible/+/803440 | 13:08 |
| hrw | yoctozepto: adapt or die? | 13:09 |
| hrw | yoctozepto: those who mirror all used distros probably also overwrite sources.list to ignore repos.yaml | 13:10 |
| opendevreview | Merged openstack/kolla-ansible master: Fix nova deployment failure when rabbitmq is disabled https://review.opendev.org/c/openstack/kolla-ansible/+/799857 | 13:12 |
| yoctozepto | hrw: ok, then just merge the two lists into one, under a new name and avoid all misunderstandings; legacy gone | 13:13 |
| yoctozepto | fwiw, I believe users find it much easier to adapt to kolla changes than to k-a changes anyway | 13:14 |
| hrw | yoctozepto: base_apt_keys and base_remote_apt_keys you mean? | 13:15 |
| hrw | yoctozepto: with names reuse if someone override them in template-overrides then build will fail so they notice format change. otherwise keys will be ignored | 13:16 |
| yoctozepto | hrw: just all base_repo_keys? | 13:16 |
| yoctozepto | hrw: well, ignored would trigger an issue later :-) | 13:16 |
| yoctozepto | I mean, since we are meddling with backward compat, then let's properly put all keys in one array | 13:16 |
| yoctozepto | this could also be a dict, rather than an array | 13:17 |
| yoctozepto | food for thought | 13:17 |
| hrw | yoctozepto: note that both lists have different ways of being downloaded. one list would complicate loop | 13:17 |
| yoctozepto | hrw: you are speaking as if you did not know how to use an 'if' statement :-) | 13:18 |
| hrw | yoctozepto: I know how. just do not see a sense in creating one list | 13:18 |
| yoctozepto | ok, I will not be pushing that | 13:18 |
| yoctozepto | let's see what other cores think | 13:19 |
| hrw | and I hate dealing with %- -% | 13:19 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: Debian/Ubuntu: handle APT keys in proper way https://review.opendev.org/c/openstack/kolla/+/784923 | 13:29 |
| hrw | yoctozepto: fixed naming etc. no code changes | 13:30 |
| opendevreview | Merged openstack/kolla master: drop leftovers of RHEL support https://review.opendev.org/c/openstack/kolla/+/785569 | 13:53 |
| yoctozepto | yay | 14:27 |
| yoctozepto | hrw: need to fix merge conflict now https://review.opendev.org/c/openstack/kolla/+/784923 | 14:29 |
| yoctozepto | I am open to approving if others prefer the current code | 14:29 |
| hrw | ok | 14:40 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: Debian/Ubuntu: handle APT keys in proper way https://review.opendev.org/c/openstack/kolla/+/784923 | 14:41 |
| hrw | easy it was | 14:41 |
| hrw | btw - I skip meeting today. have other one at same time | 14:45 |
| yoctozepto | will you rebase https://review.opendev.org/c/openstack/kolla/+/786938/14 too? | 14:45 |
| yoctozepto | ok, have fun | 14:46 |
| hrw | to tbh I am still thinking should this one go or be forgotten | 14:47 |
| hrw | read my comment there | 14:47 |
| yoctozepto | I read; I think we still leave a way to customise; a more flexible one | 14:48 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla master: base: do not overwrite APT sources.list https://review.opendev.org/c/openstack/kolla/+/786938 | 14:49 |
| hrw | done | 14:49 |
| hrw | hm. updated uca to xena... | 14:49 |
| yoctozepto | :D | 14:49 |
| hrw | and for 'apt keys' as one list you are right - get this IN and then we can change to one list if want | 14:50 |
| yoctozepto | yeah | 14:51 |
| jpic_ | hi all! do you think we want to add support for airgapped kolla-ansible deployments? it's not so hard really, basically on the ansible host you'd run a docker registry which you would pre-provision with a script, same goes for a few python packages such as the python-docker, let the user figure how they want to do their OS package mirror given that usually it's made available on airgapped networks | 14:51 |
| hrw | jpic_: idea is good | 14:53 |
| hrw | jpic_: docker registry part is easy as many people build their own images into local registry and then deploy from it | 14:53 |
| yoctozepto | teach ppl to deploy harbor and do it correctly ;-) | 14:54 |
| jpic_ | well it's not that simple to build image airgapped, as it relies on a bunch of third party package repositories | 14:54 |
| yoctozepto | I suggest to run builds in dmz | 14:55 |
| jpic_ | this harbor ? https://github.com/portdirect/harbor | 14:55 |
| yoctozepto | but yeah, k-a should be made to easily allow airgapped deployments | 14:55 |
| yoctozepto | wow, the internets have everything - not this harbor! | 14:56 |
| yoctozepto | this one https://goharbor.io/ | 14:56 |
| hrw | mgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak parallax Fl1nt: meeting in 2 | 14:58 |
| mgoddard | #startmeeting kolla | 15:00 |
| opendevmeet | Meeting started Wed Aug 4 15:00:06 2021 UTC and is due to finish in 60 minutes. The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| opendevmeet | The meeting name has been set to 'kolla' | 15:00 |
| mgoddard | #topic rollcall | 15:00 |
| mgoddard | \o | 15:00 |
| yoctozepto | o/ | 15:00 |
| priteau | \o/ | 15:03 |
| mgoddard | #topic agenda | 15:04 |
| mgoddard | * Roll-call | 15:04 |
| mgoddard | * Agenda | 15:04 |
| mgoddard | * Announcements | 15:04 |
| mgoddard | * Review action items from the last meeting | 15:04 |
| mgoddard | * CI status | 15:04 |
| mgoddard | * Release tasks | 15:04 |
| mgoddard | ** Policy on dead projects (vide Freezer) https://bugs.launchpad.net/kolla-ansible/+bug/1901698 | 15:04 |
| mgoddard | * Xena cycle planning | 15:04 |
| mgoddard | ** Xena feature prioritisation https://docs.google.com/spreadsheets/d/1BuVMwP8eLnOVJDX8f3Nb6hCrNcNpRQl57T2ENU9Xao8/edit?usp=sharing | 15:04 |
| mgoddard | * Kolla operator pain points https://etherpad.opendev.org/p/pain-point-elimination | 15:04 |
| mgoddard | * Open discussion | 15:04 |
| mgoddard | #topic announcements | 15:04 |
| mgoddard | none here | 15:05 |
| mgoddard | anyone else? | 15:05 |
| mgoddard | #topic Review action items from the last meeting | 15:05 |
| mgoddard | mgoddard email kolla-klubbers & openstack-discuss about pain points | 15:05 |
| mgoddard | done | 15:06 |
| mgoddard | no responses :) | 15:06 |
| yoctozepto | ;-( | 15:06 |
| mgoddard | clearly our users are pain free | 15:06 |
| yoctozepto | painKillers | 15:06 |
| mgoddard | #topic CI status | 15:07 |
| mgoddard | I noticed that the debian upgrade job is borked on master | 15:07 |
| mgoddard | #link https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b1e/793664/7/check/kolla-ansible-debian-source-upgrade/b1e0b6b/primary/logs/ansible/chrony-install | 15:08 |
| yoctozepto | ah, yes, looks like ansible broke with python autodetection; come on, ansible :P | 15:09 |
| mgoddard | last passed 2021-07-01... | 15:10 |
| yoctozepto | the wonder of non-voting jobs | 15:10 |
| mgoddard | oh, is that what it is? makes sense now | 15:11 |
| mgoddard | https://opendev.org/openstack/kolla-ansible/src/branch/master/tests/upgrade.sh#L27 | 15:12 |
| mgoddard | any other CI breakage to speak of? | 15:14 |
| yoctozepto | not seen any ohter | 15:15 |
| yoctozepto | nor other | 15:15 |
| yoctozepto | nor otter | 15:15 |
| yoctozepto | nor oh-errr | 15:15 |
| mgoddard | good | 15:15 |
| mgoddard | #topic Release tasks | 15:15 |
| mgoddard | R-9 | 15:16 |
| mgoddard | #link https://docs.openstack.org/kolla/latest/contributor/release-management.html | 15:18 |
| mgoddard | so next week we Switch binary images to current release | 15:18 |
| mgoddard | otherwise I think we're ok | 15:18 |
| mgoddard | #topic Policy on dead projects (vide Freezer) https://bugs.launchpad.net/kolla-ansible/+bug/1901698 | 15:19 |
| mgoddard | I smell a yoctozepto | 15:19 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add mariadb arbitrator to mariadb role https://review.opendev.org/c/openstack/kolla-ansible/+/780811 | 15:19 |
| yoctozepto | a wild yoctozepto appeared | 15:19 |
| yoctozepto | Discuss / Change core / Items / Run | 15:20 |
| yoctozepto | and back to the topic - yes | 15:20 |
| yoctozepto | and I see yesterday we also got asked on the channel about freezer specifically | 15:20 |
| yoctozepto | anyhow | 15:21 |
| mgoddard | it's certainly in my list of 'would not recommend' projects | 15:21 |
| yoctozepto | the topic is general - how should we approach projects that seem dead and unusable | 15:21 |
| yoctozepto | because | 15:21 |
| mgoddard | but how do we decide? | 15:21 |
| yoctozepto | ( mgoddard: mine too ) | 15:21 |
| yoctozepto | mgoddard: precisely | 15:21 |
| yoctozepto | but let me continue | 15:21 |
| yoctozepto | because | 15:21 |
| yoctozepto | the issue is we (as kolla) endorse the projects we support | 15:21 |
| yoctozepto | it is just how the worlds seems to view that | 15:22 |
| yoctozepto | and that is problematic because it just grows the frustration in users as there are parts that are simply known-not-to-work | 15:22 |
| yoctozepto | as opposed to merely not-known-whether-it-works | 15:22 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test Mariadb-Arbitrator with shards in the nova cells scenario https://review.opendev.org/c/openstack/kolla-ansible/+/780970 | 15:23 |
| yoctozepto | I think we kind of need a policy/doc thing on that | 15:23 |
| yoctozepto | with extra bells and whistles around tries to use such projects | 15:23 |
| yoctozepto | though I, personally, would just drop them not to create illusions of workability | 15:23 |
| yoctozepto | what are your thoughts? | 15:24 |
| yoctozepto | I know it's mostly only mgoddard today | 15:24 |
| priteau | New variable? enable_experimental_services: True | 15:24 |
| yoctozepto | but perhaps priteau can say a thing or two | 15:24 |
| yoctozepto | oh, he did | 15:24 |
| mgoddard | it's a tricky one | 15:24 |
| priteau | And without it you can't deploy them? | 15:24 |
| yoctozepto | priteau: that's a part of those bells and whistles, yeah, good idea, thanks! | 15:25 |
| mgoddard | on the one hand I don't like that we have unstable/unusable projects that add to our maintenance burden | 15:25 |
| priteau | Doc is important too of course, but we know it isn't read by everyone | 15:25 |
| mgoddard | on the other hand, I quite like that a project can go quiet but be rekindled by an interested party | 15:26 |
| yoctozepto | well, perhaps the freezer example is just radical and we should just drop freezer and be happy with all the others | 15:27 |
| mgoddard | ansible-collection-kolla-moribund-projects | 15:27 |
| yoctozepto | xD | 15:27 |
| yoctozepto | what other projects come to your mind, btw? | 15:27 |
| yoctozepto | be honest | 15:27 |
| mgoddard | it could quickly go stale though | 15:27 |
| mgoddard | unless we maintained it | 15:28 |
| mgoddard | freezer seems pretty dead, if it doesn't support py3.8 | 15:28 |
| mgoddard | a targeted approach might be best | 15:29 |
| mgoddard | for others, maybe a long period of deprecation? | 15:29 |
| yoctozepto | ok, so for freezer a mail to the mailing list that it looks bad | 15:29 |
| yoctozepto | perhaps some TC look at it too | 15:29 |
| mgoddard | https://www.stackalytics.io/?module=freezer-group | 15:29 |
| yoctozepto | and what "others" do you have in mind? | 15:29 |
| mgoddard | there have been some commits | 15:29 |
| mnasiadka | we could just add metadata to a role if it's tested and supported, or deprecated (known not to work, no data, no known users, etc) - and build some support matrix based on that? I'm not saying we should remove those deprecated, but just deprecate them? | 15:30 |
| yoctozepto | yeah, there are commits but nobody looks at bug reports | 15:30 |
| priteau | Isn't this the fix for your bug? https://review.opendev.org/c/openstack/freezer/+/795715 | 15:32 |
| yoctozepto | priteau: looks like it | 15:33 |
| priteau | Although, source & master images should have it? | 15:33 |
| yoctozepto | the bug is not mine though :-) | 15:33 |
| priteau | Oh, bug comments are from 2020 | 15:33 |
| yoctozepto | yeah, but people are running stable | 15:33 |
| yoctozepto | yeah | 15:33 |
| hrw | Sooner or later we will reverse support. Provide a list of known-to-work images and rest will generate 'UNSUPPORTED, YOU HAVE BEEN WARNED' message on build and deploy. plus 1 minute sleep per image | 15:34 |
| yoctozepto | priteau: you could comment with that in the bug report | 15:34 |
| priteau | Already did | 15:34 |
| yoctozepto | hrw: I suggest one hour, make them learn :D | 15:34 |
| yoctozepto | priteau: great! | 15:35 |
| hrw | yoctozepto: 1 minute is annoying. 1 hour is 'where it is, I'll kill it' | 15:35 |
| mgoddard | would love to get rid of these: | 15:35 |
| mgoddard | aodh | 15:35 |
| mgoddard | murano | 15:35 |
| mgoddard | solum | 15:35 |
| mgoddard | trove | 15:35 |
| mgoddard | gnocchi | 15:35 |
| mgoddard | qdrouterd | 15:35 |
| mgoddard | storm | 15:35 |
| mgoddard | vitrage | 15:35 |
| mgoddard | mistral | 15:35 |
| mgoddard | ovs-dpdk | 15:35 |
| mgoddard | vmtp | 15:35 |
| mgoddard | tacker | 15:35 |
| mgoddard | watcher | 15:35 |
| mgoddard | collectd | 15:36 |
| mgoddard | sahara | 15:36 |
| mgoddard | telegraf | 15:36 |
| mgoddard | ceilometer | 15:36 |
| mgoddard | freezer | 15:36 |
| mgoddard | senlin | 15:36 |
| mgoddard | skydive | 15:36 |
| hrw | base | 15:36 |
| yoctozepto | for the love of CI, that's one long list :-) | 15:36 |
| mgoddard | I doubt it would help our CI - few are tested! | 15:36 |
| priteau | I suppose there are still many deployments using ceilometer + gnocchi + aodh | 15:36 |
| mnasiadka | vitrage's HA is broken, so nothing new | 15:36 |
| priteau | collectd, wasn't flint adding new things recently? | 15:37 |
| mgoddard | priteau: true, that's just my preference :) | 15:37 |
| yoctozepto | I hope he was not | 15:37 |
| yoctozepto | :D | 15:37 |
| mgoddard | so where are we now? | 15:37 |
| mgoddard | freezer fixed? | 15:37 |
| yoctozepto | perhaps? at least that error is hidden now | 15:38 |
| mgoddard | it stumbles on | 15:38 |
| mgoddard | taking us back to the general question of how proactive we should be | 15:39 |
| mgoddard | I'd quite like to see a page in docs for each service | 15:39 |
| yoctozepto | that priteau-proposed flag sounds cool | 15:39 |
| yoctozepto | mgoddard: yeah, and warnings on bad services | 15:40 |
| mgoddard | as I think a lot of users at least get that far - search for 'kolla freezer' | 15:40 |
| mgoddard | the flag sounds interesting | 15:40 |
| mgoddard | but what are the criteria? | 15:40 |
| mgoddard | having a banner in docs that says 'we do not test this in CI' makes sense to me | 15:41 |
| priteau | We have a support matrix right? | 15:41 |
| mgoddard | true | 15:41 |
| yoctozepto | projects with long history of weird issues and no help coming is yet another stage | 15:41 |
| priteau | Maybe we need another level in the matrix in addition to "C - Community maintained" | 15:42 |
| mgoddard | D - despised | 15:42 |
| mgoddard | U - unloved? | 15:42 |
| yoctozepto | P - promoted | 15:43 |
| yoctozepto | U - underrated | 15:43 |
| priteau | U - unsupported | 15:43 |
| johnthetubaguy[m] | FWIW, I like what cinder did around their drivers with requiring testing, and eventually removing things with no tests. Maybe you have to set: allow_crazy_untested_services: True or the config doesn't validate? | 15:43 |
| johnthetubaguy[m] | in Nova we found no one bothered to read the support matrix, some people noticed having to enable extra config flags to use dodgy stuff | 15:45 |
| mgoddard | is there anyone who would like to work on this? | 15:46 |
| yoctozepto | I can | 15:46 |
| yoctozepto | unless priteau wants to | 15:46 |
| priteau | E - experimental (another suggestion for the matrix) | 15:46 |
| yoctozepto | anyhow, I think we agree on two things | 15:47 |
| yoctozepto | or three | 15:47 |
| priteau | Unfortunately no time at the moment | 15:47 |
| hrw | now we have Terrible, Common issues, Not even try levels in support matrix | 15:47 |
| yoctozepto | 1) add the extra flag (starting Xena, with reno and general docs) | 15:47 |
| yoctozepto | 2) extend the support matrix to mark "worse" services (this needs discussing how to choose and call these really) | 15:47 |
| yoctozepto | 3) add docs per service with warning banner for services not tested in CI and especially for those that we know are ugly | 15:48 |
| priteau | 4) drop the really bad ones that won't be fixed? | 15:48 |
| yoctozepto | ad 4) that is long term I guess | 15:49 |
| yoctozepto | anyhow, I can drive this | 15:49 |
| mgoddard | just to add another take on it - H - needs help. That was an original goal of the support matrix, to highlight where we lack skills/interest, and attract help | 15:49 |
| yoctozepto | oh, good one | 15:49 |
| priteau | Nice one | 15:50 |
| mgoddard | although that assumes the issues are on our side | 15:50 |
| mgoddard | sor tof | 15:50 |
| yoctozepto | so rtof | 15:50 |
| yoctozepto | read the online form | 15:50 |
| mgoddard | anyway, I think your list makes sense yoctozepto, especially if you're picking it up :) | 15:50 |
| yoctozepto | mgoddard: thanks | 15:50 |
| yoctozepto | I just want to focus on these and restore https://review.opendev.org/c/openstack/kolla-ansible/+/773246 and its stack | 15:50 |
| yoctozepto | shameless plug but please consider reviewing and pushing that | 15:50 |
| yoctozepto | as it's rotting for no good reason | 15:51 |
| mgoddard | yeah, let's do that | 15:51 |
| mgoddard | ok, 10m left | 15:51 |
| mgoddard | #topic Xena feature prioritisation https://docs.google.com/spreadsheets/d/1BuVMwP8eLnOVJDX8f3Nb6hCrNcNpRQl57T2ENU9Xao8/edit?usp=sharing | 15:51 |
| mgoddard | We covered this last time | 15:52 |
| yoctozepto | I gave it some thought and using the launchpad blueprints for Masakari now | 15:52 |
| mgoddard | How did that project-config patch go? | 15:52 |
| yoctozepto | perhaps we could use it for Kolla projects too | 15:52 |
| yoctozepto | got stuck | 15:52 |
| mgoddard | could do | 15:53 |
| priteau | mgoddard: the description for "H" could explain that it needs help in integration / testing in Kolla Ansible and/or fixing issues upstream | 15:53 |
| mgoddard | priteau: true | 15:53 |
| mgoddard | re blueprints, it could work | 15:53 |
| mgoddard | we don't really use them much today | 15:53 |
| yoctozepto | yup | 15:53 |
| yoctozepto | we could start | 15:53 |
| mgoddard | there are many stale ones | 15:54 |
| yoctozepto | yeah, needs some form of cleanup | 15:54 |
| yoctozepto | read my post on masakari for how I've done that | 15:54 |
| yoctozepto | closed the obviously implemented ones | 15:54 |
| yoctozepto | and reset priorities for other oldies | 15:54 |
| yoctozepto | then we set priorities for our real goals | 15:54 |
| mgoddard | I think I have done that mostly | 15:54 |
| yoctozepto | and handle it all from there | 15:55 |
| yoctozepto | plus the gerrit tags | 15:55 |
| yoctozepto | I will ping infra | 15:55 |
| yoctozepto | done | 15:56 |
| yoctozepto | both channels | 15:56 |
| mgoddard | so no hashtags for masakari? | 15:56 |
| yoctozepto | nah, we don't have that much of change traffic | 15:57 |
| yoctozepto | and only really one core that does most of the reviews | 15:57 |
| mgoddard | doh | 15:58 |
| yoctozepto | yup | 15:58 |
| mgoddard | #topic Kolla operator pain points https://etherpad.opendev.org/p/pain-point-elimination | 15:58 |
| mgoddard | I think I misspoke earlier about this | 15:59 |
| mgoddard | while I received no direct responses, we do have 4 pain points | 15:59 |
| mgoddard | or 3, if you can count | 15:59 |
| mgoddard | (clarkb) Stopping docker and its containers to perform rolling reboots for operating system patches can result in bad rabbitmq state preventing services like cinder-scheduler from reconnecting successfully afterwards. This was addressed by completely rebuilding the rabbitmq cluster with a blank state. Not sure if there is a better rolling reboot process or maybe improvements can be made | 15:59 |
| mgoddard | to rabbitmq in kolla to make it more resilient? This was with a victoria kolla deployment. | 15:59 |
| mgoddard | (Fl1nt): Kolla-ansible lack a bit of an offline mode for companies that are using it in a really source restricted environment that doesn't get access to the internet at all but get access to internal mirrors and cloned repositories. I'm already working on this issue right now and few patches have already been proposed and merged back, few remaining patches are coming. | 15:59 |
| mgoddard | Kolla-ansible lack some self-recovery mechanisms for offline environments. For example, mariadb cannot recover automatically after the cluster is restarted, and human intervention is highly likely to be required. It is hoped that some self-recovery solutions can be introduced to improve the recoverability of the cluster. | 15:59 |
| yoctozepto | :-( | 16:00 |
| yoctozepto | sad stories | 16:00 |
| mgoddard | some food for though | 16:00 |
| mgoddard | time's up y'all | 16:01 |
| mgoddard | thanks | 16:01 |
| mgoddard | #endmeeting | 16:01 |
| opendevmeet | Meeting ended Wed Aug 4 16:01:07 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:01 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/kolla/2021/kolla.2021-08-04-15.00.html | 16:01 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/kolla/2021/kolla.2021-08-04-15.00.txt | 16:01 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/kolla/2021/kolla.2021-08-04-15.00.log.html | 16:01 |
| yoctozepto | thanks mgoddard | 16:02 |
| hrw | thx | 16:03 |
| hrw | tried to read while attending other call | 16:03 |
| *** rpittau is now known as rpittau|afk | 16:14 | |
| yoctozepto | mgoddard, mnasiadka, hrw: hashtags work now 8-) | 17:06 |
| mnasiadka | so we can now be millenials? | 17:07 |
| yoctozepto | mnasiadka: on irc we all are #openstack-kolla 4evah | 17:13 |
| hrw | ;) | 17:15 |
| *** ricolin_ is now known as ricolin | 18:02 | |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible stable/wallaby: Do not run timesync checks on deployment host https://review.opendev.org/c/openstack/kolla-ansible/+/797584 | 19:36 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible stable/wallaby: Do not run timesync checks on deployment host https://review.opendev.org/c/openstack/kolla-ansible/+/797584 | 19:55 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible stable/wallaby: Do not run timesync checks on deployment host https://review.opendev.org/c/openstack/kolla-ansible/+/797584 | 19:57 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!