| *** k_mouza has quit IRC | 00:14 | |
| *** Dubla has quit IRC | 01:26 | |
| *** whoami-rajat has joined #openstack-kolla | 02:26 | |
| *** happyhemant has quit IRC | 02:41 | |
| *** skramaja has joined #openstack-kolla | 02:49 | |
| *** BjoernT has joined #openstack-kolla | 03:37 | |
| *** BjoernT_ has joined #openstack-kolla | 03:40 | |
| *** BjoernT has quit IRC | 03:42 | |
| *** schwicht has joined #openstack-kolla | 03:55 | |
| *** shyamb has joined #openstack-kolla | 04:52 | |
| *** BjoernT_ has quit IRC | 04:53 | |
| *** shyamb has quit IRC | 04:57 | |
| *** shyamb has joined #openstack-kolla | 05:04 | |
| openstackgerrit | gugug proposed openstack/kolla-ansible master: Fix the genconfig subcommand failed when enable_ceph https://review.opendev.org/668773 | 05:26 |
|---|---|---|
| *** Luzi has joined #openstack-kolla | 05:27 | |
| openstackgerrit | gugug proposed openstack/kolla-ansible master: Fix the genconfig subcommand failed when enable_ceph https://review.opendev.org/668773 | 05:29 |
| openstackgerrit | gugug proposed openstack/kolla-ansible master: Fix the genconfig subcommand failed when enable_ceph https://review.opendev.org/668773 | 05:31 |
| *** cah_link has joined #openstack-kolla | 05:37 | |
| *** shyamb has quit IRC | 05:49 | |
| *** unicell has joined #openstack-kolla | 05:55 | |
| *** luksky11 has joined #openstack-kolla | 06:06 | |
| yoctozepto | morning | 06:16 |
| *** shyamb has joined #openstack-kolla | 06:24 | |
| *** ivve has joined #openstack-kolla | 06:31 | |
| *** cgrosjean has joined #openstack-kolla | 06:51 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Add mon address to ceph release version check https://review.opendev.org/669269 | 06:53 |
| *** dking_desktop has quit IRC | 07:02 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Add mon address to ceph release version check https://review.opendev.org/669269 | 07:04 |
| *** ivve has quit IRC | 07:06 | |
| *** rpittau|afk is now known as rpittau | 07:06 | |
| mnasiadka | mgoddard: so how do we fix mariadb to really fail and not go forward? :) | 07:07 |
| yoctozepto | mnasiadka: retry the early fail patch https://review.opendev.org/666876 | 07:10 |
| yoctozepto | though mgoddard said it did not help at the time due to ansible being quirky | 07:10 |
| mnasiadka | well, any_errors_fatal won't work for blocks | 07:12 |
| mnasiadka | there we need to fix it in some other way | 07:13 |
| mnasiadka | https://github.com/ansible/ansible/issues/49041 | 07:15 |
| mnasiadka | if that's the issue :) | 07:15 |
| yoctozepto | mnasiadka: I believe mgoddard said something about handlers | 07:18 |
| yoctozepto | but let him speak for himself, he should be around soon | 07:19 |
| *** shyamb has quit IRC | 07:23 | |
| *** cgrosjean has quit IRC | 07:28 | |
| *** dking_desktop has joined #openstack-kolla | 07:29 | |
| mgoddard | mnasiadka, yoctozepto: I didn't fully investigate the issues, but blocks and handlers came up in my googling and handlers definitely seemed to be a problem | 07:33 |
| mgoddard | need to try a few different cases with different versions of ansible | 07:34 |
| *** rgogunskiy has joined #openstack-kolla | 07:37 | |
| mgoddard | yoctozepto: you have some new options in gerrit | 07:40 |
| mgoddard | congrats :) | 07:40 |
| mgoddard | use them wisely :p | 07:40 |
| mnasiadka | mgoddard: well, in master we use ansible 2.8.2 | 07:42 |
| mgoddard | mnasiadka: true, but was testing locally | 07:43 |
| mnasiadka | mgoddard: ah | 07:43 |
| yoctozepto | mgoddard: thanks :-) will do | 07:46 |
| *** unicell has quit IRC | 07:51 | |
| *** Wasaac has joined #openstack-kolla | 07:52 | |
| *** Wasaac has quit IRC | 08:01 | |
| *** Wasaac_ has joined #openstack-kolla | 08:01 | |
| *** gkadam has joined #openstack-kolla | 08:02 | |
| *** dking_desktop has quit IRC | 08:04 | |
| *** ivve has joined #openstack-kolla | 08:06 | |
| yoctozepto | mgoddard: not wanting to sound greedy but what about: https://launchpad.net/~kolla-drivers :-) | 08:07 |
| mgoddard | yoctozepto: done | 08:08 |
| yoctozepto | mgoddard: thanks, finally can manage affected releases | 08:09 |
| yoctozepto | mnasiadka: https://review.opendev.org/668623 | 08:12 |
| *** k_mouza has joined #openstack-kolla | 08:12 | |
| *** k_mouza has quit IRC | 08:12 | |
| *** k_mouza has joined #openstack-kolla | 08:13 | |
| *** Wasaac_ has quit IRC | 08:19 | |
| *** Wasaac has joined #openstack-kolla | 08:20 | |
| *** shyamb has joined #openstack-kolla | 08:35 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Fixes for MariaDB recovery https://review.opendev.org/667904 | 08:39 |
| *** k_mouza has quit IRC | 08:45 | |
| *** k_mouza has joined #openstack-kolla | 08:45 | |
| mnasiadka | yoctozepto: done | 08:53 |
| yoctozepto | mnasiadka: thanks | 08:54 |
| yoctozepto | then backport to stein, then mariadb fixes and stein is pretty done and path to train is well-paved | 08:54 |
| mnasiadka | yoctozepto: and then we can fix bugs that will show up :D | 08:58 |
| mnasiadka | let me look into that ceph-nfs on ubuntu, missing ceph fsals I see | 08:59 |
| yoctozepto | mnasiadka: yeah, worth fixing after struggle with getting ganesha in centos, and now having it not deployable anyway in ubuntu (was not deployable in centos but I fixed that case) | 09:00 |
| mnasiadka | yoctozepto: yeah, I would say it would be nice to drop root for ceph, but then we want to move to ceph-ansible :) | 09:01 |
| yoctozepto | mnasiadka: yeah, better leave it for now | 09:02 |
| yoctozepto | mnasiadka: had more thoughts about ceph deprecation in stein and removal in train? | 09:02 |
| mnasiadka | yoctozepto: we hate when RDO does something last minute, do you want to be in the hate department for some users? :) | 09:03 |
| mgoddard | mnasiadka: that is true | 09:03 |
| yoctozepto | mnasiadka: good point | 09:04 |
| mnasiadka | and then removal == migration path, which will be painful - I prefer us to work on that really good, instead of in a hurry | 09:04 |
| mnasiadka | and train is like what - November? | 09:04 |
| yoctozepto | mnasiadka: but still best deprecate (with support promise) in stein | 09:04 |
| yoctozepto | and remove then in U | 09:04 |
| yoctozepto | letting users know now | 09:04 |
| yoctozepto | as kplant suggested | 09:05 |
| yoctozepto | (weighty opinion) | 09:05 |
| yoctozepto | mnasiadka: yeah, or even oct? | 09:05 |
| yoctozepto | like stein was ready in april | 09:05 |
| yoctozepto | we are a bit laggy | 09:05 |
| yoctozepto | ;p | 09:05 |
| mnasiadka | well, it's a significant feature, so we need at least two deprecation cycles | 09:06 |
| mnasiadka | so the approach to deprecate in S, and remove in U (or later) sounds good to me | 09:06 |
| yoctozepto | +1 | 09:07 |
| yoctozepto | mgoddard: ^ | 09:07 |
| *** jistr has quit IRC | 09:07 | |
| yoctozepto | deprecate with support promise, no features being planned (or accepted) | 09:07 |
| yoctozepto | for removal in U | 09:07 |
| yoctozepto | hopefully guys leave nautilus in train and we are pretty much done | 09:08 |
| yoctozepto | though ceph is stabler now than it was before anyways | 09:08 |
| mgoddard | sounds reasonable. We can't remove until we have a deprecation + migration path, but we can at least notify people early | 09:08 |
| mnasiadka | mgoddard: that sounds like standard deprecation policy approach - https://governance.openstack.org/tc/reference/tags/assert_follows-standard-deprecation.html | 09:09 |
| yoctozepto | mgoddard, mnasiadka: always fun to reinvent the wheel | 09:09 |
| mgoddard | we should probably be a bit more strict about not accepting feature changes for ceph | 09:10 |
| mgoddard | only bug fixes | 09:10 |
| yoctozepto | mnasiadka, mgoddard: we can try that operator list as recommended | 09:12 |
| mnasiadka | mgoddard: is there a reason you did -w on https://review.opendev.org/#/c/669104, or you are just waiting for second +2? :) | 09:12 |
| mgoddard | mnasiadka: yeah, clicked by accident | 09:13 |
| yoctozepto | No such list openstack-operators | 09:13 |
| yoctozepto | uh-huh | 09:13 |
| mgoddard | yoctozepto: I requested feedback on openstack-discuss (new all in one list) | 09:13 |
| mgoddard | yoctozepto: http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006895.html | 09:14 |
| mgoddard | no response :) | 09:14 |
| yoctozepto | mgoddard: yeah, docs need to be updated | 09:14 |
| *** jistr has joined #openstack-kolla | 09:14 | |
| mgoddard | sometimes feels like ML goes to /dev/null | 09:14 |
| yoctozepto | unfortunately :-( | 09:15 |
| yoctozepto | but seriously mixing dev and user space might have annoyed some subscribers | 09:15 |
| openstackgerrit | Merged openstack/kolla-ansible master: Add parameters to configure number of processes and threads of horizon https://review.opendev.org/669193 | 09:17 |
| openstackgerrit | Merged openstack/kolla-ansible master: Add upgrade-bifrost command https://review.opendev.org/667665 | 09:17 |
| openstackgerrit | Merged openstack/kolla master: Fix ovs-appctl by adding --pidfile to ovsdb command. https://review.opendev.org/668630 | 09:17 |
| openstackgerrit | Merged openstack/kolla master: Change port of docker registry https://review.opendev.org/586392 | 09:17 |
| yoctozepto | mgoddard: may you make that one last touch to mariadb fixes commit message and me and mnasiadka make it go into master? | 09:19 |
| yoctozepto | I think with this and nova we get rid of many seemingly random failures | 09:20 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Fixes for MariaDB bootstrap and recovery https://review.opendev.org/667904 | 09:20 |
| mgoddard | yoctozepto: done | 09:20 |
| mgoddard | yoctozepto: +1. Adding retries everywhere might squash some more | 09:21 |
| mgoddard | but that would be a bigger change | 09:21 |
| mgoddard | I would quite like to combine it with making those tasks more DRY, I had some PoC patches for it | 09:21 |
| mgoddard | just a lot of boilerplate | 09:22 |
| mgoddard | also need to look into performance overhead of that approach | 09:22 |
| yoctozepto | mgoddard: sure, but I think everywhere is a bit of a crazy approach | 09:22 |
| yoctozepto | but in image pulling for sure | 09:22 |
| yoctozepto | everywhere we depend on external factors | 09:22 |
| yoctozepto | like registry | 09:22 |
| mgoddard | also keystone and mariadb interaction | 09:23 |
| mgoddard | e.g. DB deadlock issues | 09:24 |
| mgoddard | yoctozepto: https://review.opendev.org/#/c/587590/ and https://review.opendev.org/#/c/587591/ if you're interested | 09:25 |
| mgoddard | combine those with https://review.opendev.org/665922 | 09:25 |
| yoctozepto | mnasiadka: https://review.opendev.org/667904 | 09:26 |
| yoctozepto | mgoddard: I haven't grasped the idea behind those deadlocks yet | 09:27 |
| mgoddard | I think it's just a thing you have to cope with - oslo.db retries | 09:27 |
| yoctozepto | giving those changes starts for now | 09:27 |
| yoctozepto | mgoddard: I mean as if we control the process and do it in order where is the deadlock coming from | 09:28 |
| yoctozepto | unless we don't wait somewhere? | 09:29 |
| mgoddard | yoctozepto: not certain, but could be during an upgrade, existing db? | 09:29 |
| yoctozepto | no idea, but these are rarer anyways | 09:30 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/stein: Add upgrade-bifrost command https://review.opendev.org/667630 | 09:30 |
| yoctozepto | now we probably end up with the most common fails to be related to infra mirrors having their tea breaks | 09:30 |
| yoctozepto | also: https://review.opendev.org/665643 | 09:32 |
| yoctozepto | mgoddard, ^ how likely is that? | 09:32 |
| yoctozepto | not encountered in fact I believe | 09:32 |
| mgoddard | not encountered yet | 09:34 |
| yoctozepto | future-proofing? | 09:34 |
| mgoddard | yeah | 09:34 |
| mgoddard | play by the rules or it'll come back to bite :) | 09:34 |
| yoctozepto | right approach | 09:35 |
| mnasiadka | hmm, ubuntu uses ceph mimic, right? | 09:35 |
| mnasiadka | can we just, not support ceph-nfs on ubuntu? | 09:38 |
| mnasiadka | other option is some work to bump up ubuntu to nautilus as well | 09:38 |
| mnasiadka | (but then we'll use ceph upstream repos for that) | 09:39 |
| yoctozepto | mnasiadka: giving up already? :D mimic is not going to work at all? | 09:41 |
| mgoddard | mnasiadka: has it ever worked on ubuntu? | 09:41 |
| mgoddard | is there a bug report for it? | 09:42 |
| mgoddard | just wondering if there's an audience for it | 09:42 |
| mnasiadka | yoctozepto: well, given the alternative of debugging ceph fsal and nfs-ganesha on an OS I haven't used for some time... with backlevel version of ceph, I would prefer to spend that time to first get the versions in sync before and end the craziness :) | 09:42 |
| mnasiadka | mgoddard: probably there is not, ceph-nfs was unusable before yoctozepto fixed the USER definition in Dockerfile :) | 09:42 |
| mgoddard | maybe time would be better spent on ceph-ansible migration? | 09:44 |
| mgoddard | which hopefully has ceph-nfs support :) | 09:44 |
| mgoddard | would be interesting to know if there are any feature gaps | 09:45 |
| yoctozepto | mgoddard: wonder how ceph-nfs is popular in general | 09:45 |
| mgoddard | not sure | 09:46 |
| yoctozepto | there is no bug report | 09:46 |
| yoctozepto | CI revealed the issue | 09:46 |
| mgoddard | not even sure what it does :) | 09:46 |
| yoctozepto | it was down for any deployment | 09:46 |
| yoctozepto | so it could not be used by our users ;-) | 09:46 |
| yoctozepto | mgoddard: exports ceph storage via nfs protocol | 09:46 |
| yoctozepto | potent feature when cephfs was unstable | 09:47 |
| yoctozepto | but nowadays... | 09:47 |
| yoctozepto | well, maybe if you can't have cephfs client | 09:49 |
| yoctozepto | but it's so irrelevant to openstack :D | 09:49 |
| yoctozepto | manila can do cephfs | 09:49 |
| mnasiadka | ok, let me try to use the CI to enable debug :) | 09:50 |
| yoctozepto | mgoddard, mnasiadka: we should revive the vagrant thing to allow sensible local testing... | 09:51 |
| mgoddard | yoctozepto: that's true, it currently depends on kolla-cli which we are removing | 09:51 |
| yoctozepto | anyways, mgoddard, one thing pre-stein - deploy guide - what are we doing with it? in my eyes the whole docs is deploy guide | 09:51 |
| yoctozepto | now we have to either modify the jobs to really test deploy guide changes | 09:52 |
| mnasiadka | mgoddard: well, Borne Mace fixed the functional tests, maybe let's not drop it :D | 09:52 |
| yoctozepto | or just merge the two and be happy | 09:52 |
| mgoddard | I don't think we decided what to do with it | 09:52 |
| yoctozepto | mnasiadka: you will support it? | 09:52 |
| yoctozepto | mgoddard: yeah, we didn't | 09:52 |
| mgoddard | some work was put in to make it build again | 09:52 |
| yoctozepto | and then it broke | 09:52 |
| yoctozepto | and I fixed it again | 09:52 |
| yoctozepto | but we don't test it | 09:53 |
| yoctozepto | so it's dumb | 09:53 |
| yoctozepto | I would merge the two because they are really the same thing | 09:53 |
| mgoddard | I'm all up for making it a link to our docs but chason seemed resistent | 09:53 |
| mnasiadka | yoctozepto: I was rather saying that it might be a shame to loose this code, question what is the workload behind replacing kolla-ansible with kolla-cli | 09:53 |
| yoctozepto | mnasiadka: first I'd have to learn what kolla-cli does | 09:54 |
| yoctozepto | ;D | 09:54 |
| mgoddard | mnasiadka: it's always a shame to drop code, but we got no feedback from anyone using it, and the maintainers left, and it was never released | 09:54 |
| mnasiadka | yoctozepto: last time I tried, it did not build due to some tox tests failing, maybe we need to do another approach :) | 09:54 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: DNM: Troubleshoot ceph-nfs on ubuntu https://review.opendev.org/669315 | 09:56 |
| yoctozepto | Kolla-Ansible requires that hosts, groups, and services are specified in an inventory file. With Kolla-CLI, you can add/remove hosts, change group associations, etc from the CLI or API. Kolla-Ansible also maintains passwords and various configuration variables in a variety of global, group and host files. With Kolla-CLI, you can now view and change these from the CLI/API. | 09:56 |
| yoctozepto | so it essentially manages the environment for kolla-ansible | 09:57 |
| mnasiadka | yeah, I remember you can use multiple inventories and so on | 09:57 |
| yoctozepto | like virtualenv | 09:57 |
| yoctozepto | for k-a | 09:58 |
| yoctozepto | mgoddard: what's the main reasoning behind removing kolla-cli? | 09:58 |
| mgoddard | yoctozepto: it's stale, (was) broken, unreleased, and AFAIT, unused. It would require overhead to maintain | 10:00 |
| mgoddard | at the virtual PTG we were discussing ways we could trim down the scope of the project to reduce maintenance overhead, improve sustainability etc | 10:01 |
| mgoddard | it doesn't cost us anything to not maintain it so I don't mind pausing on retiring it if we're not sure | 10:02 |
| yoctozepto | ok, thanks for insights | 10:02 |
| yoctozepto | is there some interface for repo statistics | 10:02 |
| yoctozepto | or do I have to clone it | 10:02 |
| yoctozepto | ;D | 10:02 |
| mgoddard | stackalytics? | 10:03 |
| yoctozepto | mgoddard: k, forgot, used once | 10:04 |
| yoctozepto | thanks | 10:04 |
| *** rgogunskiy has quit IRC | 10:05 | |
| *** cgrosjean has joined #openstack-kolla | 10:05 | |
| yoctozepto | so it's 99% oracle juice | 10:05 |
| yoctozepto | shaken, not mixed | 10:06 |
| mgoddard | indeed | 10:06 |
| mgoddard | kolla team is now 0% oracle | 10:06 |
| yoctozepto | Lines of code shows current state or all git commits? | 10:06 |
| *** VickyG has joined #openstack-kolla | 10:06 | |
| *** Wasaac has quit IRC | 10:10 | |
| *** Wasaac has joined #openstack-kolla | 10:11 | |
| *** shyamb has quit IRC | 10:15 | |
| yoctozepto | mnasiadka: remove non-ceph jobs to receive results faster | 10:16 |
| mgoddard | not sure about LoC | 10:16 |
| *** Wasaac has quit IRC | 10:27 | |
| *** Wasaac_ has joined #openstack-kolla | 10:27 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Wait for all compute services before cell discovery https://review.opendev.org/668623 | 10:31 |
| openstackgerrit | Merged openstack/kolla stable/stein: Fixes ceph-osd deploys with loop devices. https://review.opendev.org/668291 | 10:31 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible stable/stein: Wait for all compute services before cell discovery https://review.opendev.org/669321 | 10:32 |
| *** luksky11 has quit IRC | 10:32 | |
| yoctozepto | ^ rocky needs manual merging, I'll do it later, there's no hurry in fact ;-) | 10:33 |
| *** cgrosjean has quit IRC | 10:36 | |
| openstackgerrit | Merged openstack/kolla stable/stein: CI: set the same gate queue for kolla and kolla-ansible https://review.opendev.org/669103 | 10:40 |
| openstackgerrit | Merged openstack/kolla stable/rocky: CI: set the same gate queue for kolla and kolla-ansible https://review.opendev.org/669104 | 10:40 |
| mgoddard | yoctozepto, mnasiadka: what are we going to do re minimum version of ansible in stein? | 10:43 |
| mgoddard | https://review.opendev.org/#/c/668153/ broke 2.4 support - we either need to bump to 2.5 or revert that patch | 10:44 |
| mgoddard | here is the patch to bump in master: https://review.opendev.org/#/c/668387/ | 10:44 |
| yoctozepto | mgoddard: bump to 2.5 is more future-proof | 10:44 |
| mgoddard | yoctozepto: it is, but after mnasiadka's comments about no last minute changes | 10:45 |
| mgoddard | which I do agree with - we are way over time on stein | 10:45 |
| yoctozepto | exactly, and possible breaking 2.9 is around the corner | 10:45 |
| yoctozepto | I saw testing patch as well | 10:46 |
| yoctozepto | which tests ubuntu with the latest | 10:46 |
| yoctozepto | and centos with minimaum | 10:46 |
| mgoddard | yes | 10:47 |
| mnasiadka | mgoddard: if you ask me, we should require at least 2.7, Stein's lifetime is at least end of next year right? | 10:48 |
| mgoddard | 18 months until EM | 10:49 |
| yoctozepto | seems mnasiadka is not always against last moment changes :D | 10:49 |
| mgoddard | lol | 10:51 |
| mnasiadka | yoctozepto: CI runs 2.8, I guess in all branches - so I think we should be fine | 10:51 |
| *** shyamb has joined #openstack-kolla | 10:51 | |
| mgoddard | CI runs latest | 10:51 |
| mgoddard | let's go for 2.5 minimum, since our code can run on that. Ideally people will use later | 10:52 |
| mgoddard | if not that's their problem | 10:52 |
| yoctozepto | is there anything valuable in 2.6 or 2.7 that we may miss soon mnasiadka mgoddard | 10:53 |
| mgoddard | we should only really bump the minimum if we depend on features, or there are serious bugs | 10:53 |
| yoctozepto | mgoddard: my point | 10:53 |
| yoctozepto | but we might not know it YET | 10:53 |
| mnasiadka | yoctozepto: support, if we go 2.6 :) | 10:54 |
| mnasiadka | yoctozepto: 2.9 will be out end of this year I think, and then 2.6 is eol | 10:55 |
| mgoddard | this is just a minimum - we don't have a maxium | 10:55 |
| mgoddard | (maybe we should) | 10:56 |
| yoctozepto | another random issue http://logs.openstack.org/04/667904/7/check/kolla-ansible-ubuntu-source-upgrade-ceph/0913848/primary/logs/ansible/test-openstack | 10:56 |
| yoctozepto | mgoddard: we should protect against 2.10 which will break our inventories by default | 10:56 |
| yoctozepto | due to '-' | 10:56 |
| mgoddard | -? | 10:57 |
| yoctozepto | mhm | 10:57 |
| yoctozepto | it's illegal char to name a group | 10:57 |
| yoctozepto | like glance-api | 10:57 |
| mgoddard | what! | 10:57 |
| mgoddard | that's ridiculous | 10:57 |
| mgoddard | that will break the entire world | 10:57 |
| mgoddard | yoctozepto: same job: http://logs.openstack.org/04/667904/7/check/kolla-ansible-ubuntu-source-upgrade-ceph/0913848/primary/logs/kolla/keystone/keystone.txt.gz#_2019-07-05_10_20_00_259 | 10:59 |
| yoctozepto | mgoddard: not me to judge | 11:02 |
| yoctozepto | they offer some option to change the behavior | 11:02 |
| *** luksky11 has joined #openstack-kolla | 11:02 | |
| yoctozepto | re: the issue - wonder why previous api calls suceeded | 11:02 |
| yoctozepto | as if it broke in the meantime | 11:03 |
| yoctozepto | as in it upgraded everything fine and dandy | 11:03 |
| yoctozepto | tests come | 11:03 |
| yoctozepto | succeed, succeed | 11:03 |
| yoctozepto | and FAIL | 11:03 |
| yoctozepto | :D | 11:03 |
| *** whoami-rajat has quit IRC | 11:06 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/stein: Specify endpoint when creating monasca user https://review.opendev.org/669327 | 11:09 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/rocky: Specify endpoint when creating monasca user https://review.opendev.org/669328 | 11:09 |
| *** shyamb has quit IRC | 11:14 | |
| *** Wasaac_ has quit IRC | 11:14 | |
| ohwhyosa[m] | Hey evryone, the images on the docker-hub are under the username kolla, right? | 11:29 |
| ohwhyosa[m] | https://hub.docker.com/u/kolla/ | 11:29 |
| ohwhyosa[m] | ^ That one? | 11:29 |
| mnasiadka | mgoddard: regarding "WSREP has not yet prepared node for application use" - isn't that the case when some mariadb node has been upgraded and the node is in some weird state? | 11:29 |
| mgoddard | ohwhyosa[m]: yes | 11:29 |
| mgoddard | mnasiadka: it sounds like something like that, but I'm not sure | 11:30 |
| mnasiadka | mgoddard: maybe our haproxy mariadb setup is not optimal? | 11:30 |
| mnasiadka | mgoddard: we shouldn't be directing requests to node's that are not ready :) | 11:30 |
| mgoddard | mnasiadka: what haproxy setup? :) | 11:31 |
| ohwhyosa[m] | It is down | 11:31 |
| mgoddard | bizarrely, in multinode jobs we disable haproxy, but enable it for single node jobs | 11:31 |
| mnasiadka | mgoddard: what? :) | 11:31 |
| ohwhyosa[m] | mgoddard: 404 | 11:31 |
| mgoddard | ohwhyosa[m]: yeah, someone saw this recently. The images are still available | 11:32 |
| mgoddard | mnasiadka: yeah | 11:32 |
| mnasiadka | mgoddard: why do we disable it? :) | 11:32 |
| mgoddard | mnasiadka: it's because the VIP traffic would be blocked by secgroups | 11:32 |
| mgoddard | or IP filtering | 11:32 |
| ohwhyosa[m] | mgoddard: Oh, good! I'm getting a 404 with the images, went to check the name and though it might be that | 11:32 |
| mnasiadka | mgoddard: well... then we need to talk with infra how to get proper secgroups/ip filtering implemented | 11:33 |
| mgoddard | mnasiadka: so all mariadb traffic goes to primary node | 11:33 |
| mnasiadka | mgoddard: it's not very good case that we disable haproxy for multinode :D | 11:33 |
| mgoddard | yeah | 11:33 |
| mgoddard | I think the workaround is to configure an overlay - there is a zuul role for it | 11:33 |
| mgoddard | multi-node-firewall | 11:34 |
| mgoddard | or multi-node-bridge | 11:35 |
| mgoddard | it's been like this for a while, we could try again | 11:36 |
| *** shyamb has joined #openstack-kolla | 11:41 | |
| mnasiadka | mgoddard: well, firewall is one thing - how do we get an additional ip? :) | 11:45 |
| mgoddard | mnasiadka: that would be multi-node-bridge, to setup a VXLAN overlay | 11:45 |
| mgoddard | it uses OVS though, which might conflict with our own OVS | 11:45 |
| mnasiadka | yeah, but let's try - another thing on my list :) | 11:46 |
| mgoddard | perhaps we could use NAT, or no VIP with different ports (using new listen_port config) | 11:48 |
| mgoddard | i.e. disable keepalived | 11:48 |
| mgoddard | don't necessarily need to test haproxy failover | 11:48 |
| mgoddard | although that would be nice | 11:48 |
| yoctozepto | mgoddard, mnasiadka: is anyone writing down these ideas so we can discuss/assign/implement them later? | 11:51 |
| mgoddard | yoctozepto: I doubt it :) | 11:52 |
| yoctozepto | irc is good but scrolling back (and remembering to do it) is not | 11:52 |
| yoctozepto | ;-) | 11:52 |
| yoctozepto | mgoddard: and what's the best place? launchpad bugs, launchpad blueprints? or maybe storyboard? | 11:55 |
| yoctozepto | whiteboard is too shabby for this stuff | 11:56 |
| mgoddard | CI improvements etherpad? | 11:56 |
| mgoddard | maybe with a blueprint linking to it? | 11:56 |
| yoctozepto | sorry, I meant etherpad being too shabby | 11:56 |
| mgoddard | ah | 11:57 |
| yoctozepto | it's fine for loose notes like CI issues or whiteboard | 11:57 |
| yoctozepto | but not this kind of stuff | 11:57 |
| yoctozepto | imho | 11:57 |
| kplant | is the issue you guys are discussing that traffic from hapxy -> galera is blocked when sourced from the vip by security groups? | 11:58 |
| kplant | in multinode ci jobs | 11:58 |
| yoctozepto | kplant: vip traffic is blocked because cloud provider does not know about it | 12:00 |
| yoctozepto | about the virtual ip address | 12:01 |
| kplant | ah, ok | 12:02 |
| *** openstackgerrit has quit IRC | 12:04 | |
| *** openstackgerrit has joined #openstack-kolla | 12:05 | |
| openstackgerrit | Mark Goddard proposed x/kayobe master: Publish docs and release notes on openstack.o.o https://review.opendev.org/669342 | 12:05 |
| mgoddard | w00p ^ first kayobe patch notification | 12:05 |
| yoctozepto | mgoddard: https://storyboard.openstack.org/#!/project/x/kayobe <- I see you are using storyboard | 12:05 |
| yoctozepto | (and also kolla-kubernetes) | 12:05 |
| mgoddard | yoctozepto: yeah, just to confuse things | 12:05 |
| *** cgrosjean has joined #openstack-kolla | 12:05 | |
| mgoddard | kolla-k8s is retired | 12:06 |
| yoctozepto | mgoddard: well, we should decide on whether we start using storyboard or keep to launchpad | 12:07 |
| yoctozepto | I prefer launchpad | 12:07 |
| mgoddard | yoctozepto: we did | 12:07 |
| mgoddard | and agreed to stick to LP until fingers are prised off :p | 12:08 |
| mgoddard | mostly just people not wanting to switch rather than a particular dislike for storyboard | 12:09 |
| *** pescobar has joined #openstack-kolla | 12:09 | |
| mnasiadka | and the con of managing releases still in LP :) | 12:09 |
| mgoddard | (or overhead of having to manage releases, depending on how you look at it :) ) | 12:10 |
| openstackgerrit | Merged openstack/kolla-ansible stable/stein: Enable deflate when using tls for horizon https://review.opendev.org/668290 | 12:12 |
| yoctozepto | LP feels somehow better atm to me | 12:12 |
| *** henriqueof has joined #openstack-kolla | 12:14 | |
| mgoddard | heading off now. Please try to help shepherd these last stein patches through | 12:15 |
| mgoddard | including the ansible 2.5 minimum (needs master merge & backport) | 12:15 |
| mgoddard | have a nice weekend all, with any luck we'll release Stein next week! | 12:15 |
| yoctozepto | mgoddard: buh-bye! | 12:17 |
| yoctozepto | mnasiadka: https://review.opendev.org/667363 | 12:19 |
| *** whoami-rajat has joined #openstack-kolla | 12:19 | |
| *** goldyfruit has quit IRC | 12:21 | |
| *** Wasaac has joined #openstack-kolla | 12:22 | |
| mnasiadka | looks good, let's see :) | 12:23 |
| openstackgerrit | Merged openstack/kolla master: CI: Clean up Zuul repo mirrors for CentOS https://review.opendev.org/664151 | 12:28 |
| *** skramaja_ has joined #openstack-kolla | 12:39 | |
| *** skramaja has quit IRC | 12:40 | |
| *** skramaja_ has quit IRC | 12:42 | |
| *** shyamb has quit IRC | 12:55 | |
| *** shyamb has joined #openstack-kolla | 12:55 | |
| *** rgogunskiy has joined #openstack-kolla | 13:13 | |
| *** cgrosjean has quit IRC | 13:19 | |
| *** cgrosjean has joined #openstack-kolla | 13:19 | |
| *** shyamb has quit IRC | 13:25 | |
| *** cah_link has quit IRC | 13:28 | |
| *** goldyfruit has joined #openstack-kolla | 13:29 | |
| openstackgerrit | Merged openstack/kolla-ansible stable/stein: Wait for mariadb to stop after shutdown https://review.opendev.org/667363 | 13:33 |
| *** gkadam_ has joined #openstack-kolla | 13:50 | |
| *** Luzi has quit IRC | 13:50 | |
| *** gkadam has quit IRC | 13:52 | |
| *** cah_link has joined #openstack-kolla | 13:53 | |
| *** cah_link has quit IRC | 14:00 | |
| *** Wasaac has quit IRC | 14:03 | |
| *** Wasaac has joined #openstack-kolla | 14:04 | |
| *** fxpester has joined #openstack-kolla | 14:23 | |
| *** jaicaa_ has joined #openstack-kolla | 14:27 | |
| openstackgerrit | Bharat Kunwar proposed x/kayobe-config master: Make source kayobe-env zsh compatible https://review.opendev.org/669364 | 14:27 |
| *** jaicaa has quit IRC | 14:30 | |
| *** fxpester has quit IRC | 14:38 | |
| *** Wasaac has quit IRC | 14:45 | |
| *** Wasaac has joined #openstack-kolla | 14:46 | |
| *** Wasaac has quit IRC | 14:47 | |
| *** Wasaac has joined #openstack-kolla | 14:48 | |
| *** goldyfruit has quit IRC | 14:54 | |
| *** rpittau is now known as rpittau|afk | 15:12 | |
| ohwhyosa[m] | Hello again! | 15:38 |
| *** luksky11 has quit IRC | 15:38 | |
| ohwhyosa[m] | I pretty much managed to deploy the first multinode (took me a while :P) | 15:38 |
| ohwhyosa[m] | Though when using ovs_dpdk it throws "dict object has no value ipv4" | 15:38 |
| ohwhyosa[m] | and breaks | 15:39 |
| ohwhyosa[m] | And it is when it breaks like this/at this point that destroy hangs and doesn't work | 15:39 |
| ohwhyosa[m] | I'm using the config example from the docs | 15:39 |
| openstackgerrit | Merged openstack/kolla-ansible master: CI - remove unused setup scripts https://review.opendev.org/666791 | 15:42 |
| openstackgerrit | Corey Bryant proposed openstack/kolla master: Add Python 3 Train unit tests https://review.opendev.org/669375 | 15:43 |
| openstackgerrit | Corey Bryant proposed openstack/kolla-ansible master: Add Python 3 Train unit tests https://review.opendev.org/669376 | 15:44 |
| *** Wasaac has quit IRC | 15:52 | |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible stable/stein: CI - remove unused setup scripts https://review.opendev.org/669384 | 15:54 |
| *** goldyfruit has joined #openstack-kolla | 15:58 | |
| *** Sravan has joined #openstack-kolla | 16:00 | |
| *** Sravan has quit IRC | 16:04 | |
| *** Sravan has joined #openstack-kolla | 16:10 | |
| ohwhyosa[m] | Nah, I removed that now and it doesn't create the bridges and stuff, it seems | 16:12 |
| *** rgogunskiy has quit IRC | 16:15 | |
| *** rgogunskiy has joined #openstack-kolla | 16:16 | |
| *** rgogunskiy has quit IRC | 16:19 | |
| *** henriqueof has quit IRC | 16:21 | |
| *** cgrosjean has quit IRC | 16:22 | |
| *** gkadam_ has quit IRC | 16:37 | |
| *** k_mouza has quit IRC | 16:39 | |
| *** Wasaac has joined #openstack-kolla | 16:42 | |
| *** cah_link has joined #openstack-kolla | 16:46 | |
| *** Wasaac has quit IRC | 16:47 | |
| *** schwicht has quit IRC | 16:50 | |
| *** unicell has joined #openstack-kolla | 16:50 | |
| *** k_mouza has joined #openstack-kolla | 16:56 | |
| *** k_mouza has quit IRC | 17:00 | |
| *** factor has joined #openstack-kolla | 17:03 | |
| *** whoami-rajat has quit IRC | 17:06 | |
| kplant | have any of you guys done k2k federation with two clouds deployed with kolla-ansible? | 17:07 |
| *** factor has quit IRC | 17:08 | |
| *** luksky11 has joined #openstack-kolla | 17:14 | |
| *** cah_link has quit IRC | 17:23 | |
| *** unicell has quit IRC | 17:30 | |
| *** mloza has quit IRC | 17:35 | |
| goldyfruit | Guys, could you please review the Masakari reviews ? | 17:41 |
| *** cah_link has joined #openstack-kolla | 17:42 | |
| *** henriqueof has joined #openstack-kolla | 17:44 | |
| *** factor has joined #openstack-kolla | 17:48 | |
| *** cah_link has quit IRC | 18:18 | |
| *** factor has quit IRC | 18:20 | |
| *** factor has joined #openstack-kolla | 18:34 | |
| *** factor has quit IRC | 18:40 | |
| *** factor has joined #openstack-kolla | 18:40 | |
| *** factor has quit IRC | 18:47 | |
| *** factor has joined #openstack-kolla | 19:16 | |
| openstackgerrit | Rafael Folco proposed openstack/kolla master: DNM: Add rhel8 support for early CentOS8 testing https://review.opendev.org/669471 | 19:19 |
| *** Sravan has quit IRC | 19:33 | |
| *** Sravan has joined #openstack-kolla | 19:33 | |
| openstackgerrit | Rafael Folco proposed openstack/kolla master: DNM: Add rhel8 support for early CentOS8 testing https://review.opendev.org/669471 | 19:55 |
| *** goldyfruit has quit IRC | 20:09 | |
| *** Sravan has quit IRC | 20:15 | |
| *** dciabrin has quit IRC | 20:27 | |
| *** Sravan has joined #openstack-kolla | 20:40 | |
| *** unicell has joined #openstack-kolla | 21:04 | |
| *** Sravan has quit IRC | 21:04 | |
| *** Sravan has joined #openstack-kolla | 21:06 | |
| *** Sravan has quit IRC | 21:10 | |
| *** Sravan has joined #openstack-kolla | 21:24 | |
| *** ivve has quit IRC | 21:25 | |
| *** Sravan has quit IRC | 21:46 | |
| *** dciabrin has joined #openstack-kolla | 21:54 | |
| *** henriqueof has quit IRC | 22:08 | |
| *** jistr has quit IRC | 22:24 | |
| *** jistr has joined #openstack-kolla | 22:24 | |
| *** luksky11 has quit IRC | 22:33 | |
| *** unicell has left #openstack-kolla | 23:52 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!