| *** k_mouza has quit IRC | 00:04 | |
| *** k_mouza has joined #openstack-kolla | 00:22 | |
| *** k_mouza has quit IRC | 00:26 | |
| *** k_mouza has joined #openstack-kolla | 00:32 | |
| *** maharg101 has joined #openstack-kolla | 00:33 | |
| *** k_mouza has quit IRC | 00:36 | |
| *** maharg101 has quit IRC | 00:38 | |
| *** LinPeiWen has joined #openstack-kolla | 00:40 | |
| *** JamesBenson has quit IRC | 00:42 | |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Add support for encrypting backend Neutron API Server https://review.opendev.org/756367 | 00:42 |
|---|---|---|
| *** JamesBenson has joined #openstack-kolla | 00:43 | |
| *** brinzhang has joined #openstack-kolla | 00:44 | |
| *** JamesBenson has quit IRC | 00:47 | |
| *** wuchunyang has joined #openstack-kolla | 00:52 | |
| *** yankcrime has quit IRC | 00:53 | |
| *** zzzeek has quit IRC | 01:10 | |
| *** zzzeek has joined #openstack-kolla | 01:11 | |
| *** k_mouza has joined #openstack-kolla | 01:13 | |
| *** k_mouza has quit IRC | 01:17 | |
| *** JamesBenson has joined #openstack-kolla | 01:20 | |
| *** JamesBenson has quit IRC | 01:24 | |
| *** k_mouza has joined #openstack-kolla | 01:26 | |
| *** k_mouza has quit IRC | 01:30 | |
| *** LinPeiWen has quit IRC | 01:46 | |
| *** k_mouza has joined #openstack-kolla | 01:57 | |
| *** dciabrin has quit IRC | 01:57 | |
| *** k_mouza has quit IRC | 02:01 | |
| *** dciabrin has joined #openstack-kolla | 02:07 | |
| *** k_mouza has joined #openstack-kolla | 02:23 | |
| *** k_mouza has quit IRC | 02:27 | |
| *** JamesBenson has joined #openstack-kolla | 02:30 | |
| *** JamesBenson has quit IRC | 02:34 | |
| *** k_mouza has joined #openstack-kolla | 02:51 | |
| *** k_mouza has quit IRC | 02:55 | |
| *** LinPeiWen has joined #openstack-kolla | 02:59 | |
| *** JamesBenson has joined #openstack-kolla | 03:05 | |
| *** maharg101 has joined #openstack-kolla | 03:06 | |
| *** JamesBenson has quit IRC | 03:10 | |
| *** maharg101 has quit IRC | 03:11 | |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Add support for encrypting backend Neutron API Server https://review.opendev.org/756367 | 03:26 |
| *** ricolin_ has joined #openstack-kolla | 04:01 | |
| *** bengates has joined #openstack-kolla | 04:01 | |
| *** wuchunyang has quit IRC | 04:01 | |
| *** bengates has quit IRC | 04:05 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #openstack-kolla | 04:33 | |
| *** vishalmanchanda has joined #openstack-kolla | 04:35 | |
| *** cah_link has joined #openstack-kolla | 04:45 | |
| *** cah_link has quit IRC | 05:00 | |
| *** zzzeek has quit IRC | 05:02 | |
| *** zzzeek has joined #openstack-kolla | 05:05 | |
| *** maharg101 has joined #openstack-kolla | 05:07 | |
| *** maharg101 has quit IRC | 05:12 | |
| *** wuchunyang has joined #openstack-kolla | 05:31 | |
| *** skramaja has joined #openstack-kolla | 05:31 | |
| *** zzzeek has quit IRC | 05:42 | |
| *** zzzeek has joined #openstack-kolla | 05:43 | |
| *** rpittau|afk is now known as rpittau | 05:43 | |
| *** brinzhang_ has joined #openstack-kolla | 05:52 | |
| *** cah_link has joined #openstack-kolla | 05:55 | |
| *** brinzhang has quit IRC | 05:55 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Add log for cron script fernet-rotate.sh https://review.opendev.org/756083 | 06:09 |
| *** LinPeiWen has quit IRC | 06:11 | |
| *** zzzeek has quit IRC | 06:11 | |
| *** zzzeek has joined #openstack-kolla | 06:12 | |
| *** LinPeiWen has joined #openstack-kolla | 06:15 | |
| *** also_stingrayza has joined #openstack-kolla | 06:16 | |
| *** stingrayza has quit IRC | 06:17 | |
| *** LinPeiWen has quit IRC | 06:23 | |
| *** jobewan has quit IRC | 06:46 | |
| yoctozepto | morning | 06:49 |
| *** jobewan has joined #openstack-kolla | 06:52 | |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible stable/ussuri: Fix haproxy bundle generation https://review.opendev.org/757007 | 07:01 |
| wuchunyang | hello what time will kolla be released? | 07:07 |
| *** maharg101 has joined #openstack-kolla | 07:08 | |
| *** nikparasyr has joined #openstack-kolla | 07:09 | |
| *** maharg101 has quit IRC | 07:12 | |
| yoctozepto | wuchunyang: hoping for the week after ptg I guess @ mgoddard | 07:13 |
| wuchunyang | ok, thanks | 07:15 |
| *** bengates has joined #openstack-kolla | 07:26 | |
| *** e0ne has joined #openstack-kolla | 07:27 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Add log for cron script fernet-rotate.sh https://review.opendev.org/756083 | 07:31 |
| *** maharg101 has joined #openstack-kolla | 07:33 | |
| *** kevko has joined #openstack-kolla | 07:42 | |
| mgoddard | morning | 07:45 |
| mgoddard | yoctozepto: let's see. Some things are out of our hands as usual :) | 07:45 |
| yoctozepto | mgoddard: yeah, that's where the "hope" kicks in :-) | 07:45 |
| yoctozepto | mgoddard: anyhow, good news is octavia is passing | 07:45 |
| yoctozepto | mgoddard: did you manage to keep the workaround in a single task? | 07:46 |
| *** gfidente has joined #openstack-kolla | 07:51 | |
| wuchunyang | yoctozepto mgoddard hello. i want to know why we defer this ps https://review.opendev.org/#/c/755589/ for 'w' cycle ? | 07:55 |
| patchbot | patch 755589 - kolla-ansible - octavia: support tenant management network - 11 patch sets | 07:55 |
| yoctozepto | wuchunyang: because it does magic not expected from kolla-ansible - we want to have as much octavia support as we can in V but we need to have a longer think cycle on this one | 07:56 |
| *** e0ne has quit IRC | 07:56 | |
| wuchunyang | but without this ps , the octavia still not work out of the box. could we add this ps as a preview in 'v' cycle ? | 08:01 |
| mnasiadka | host networking was never configured by kolla-ansible, so I think operators can survive to do it manually for one more cycle | 08:04 |
| mnasiadka | I believe this functionality is more fit into kayobe... | 08:09 |
| mgoddard | yoctozepto: re single task, I had handler inversion. Basically I was about 1 hour behind you :) | 08:10 |
| *** gfidente has quit IRC | 08:10 | |
| *** gfidente has joined #openstack-kolla | 08:14 | |
| hrw | morning | 08:17 |
| hrw | https://review.opendev.org/754355 anyone? | 08:17 |
| patchbot | patch 754355 - kolla - convert STATUS_* consts into Enum - 3 patch sets | 08:17 |
| *** k_mouza has joined #openstack-kolla | 08:20 | |
| wuchunyang | yoctozepto mnasiadka ok, thanks, This is not very friendly to testers， anyhow tenant network is the simplest way to run a octavia service. | 08:24 |
| *** k_mouza_ has joined #openstack-kolla | 08:28 | |
| *** k_mouza has quit IRC | 08:31 | |
| *** LinPeiWen has joined #openstack-kolla | 08:36 | |
| openstackgerrit | Merged openstack/kolla-ansible master: octavia: generate certificates automatically https://review.opendev.org/754280 | 08:41 |
| yoctozepto | mgoddard: no problem | 08:45 |
| yoctozepto | mgoddard: maybe bcoca fixes that evaluation order and it's possible | 08:45 |
| yoctozepto | but a single non-looped task per host is quite nice now | 08:46 |
| yoctozepto | mgoddard, mnasiadka, wuchunyang: well, we could hide this indeed behind a flag explicitly called EXPERIMENTAL_blah_blah and write in big letters it's only meant for testers and PoCers atm and should never ever be used in production as it may change with no upgradability warranty | 08:48 |
| mgoddard | that would at least allow us to merge the test :) | 08:48 |
| yoctozepto | mgoddard: indeed, that was the hidden assumption | 08:49 |
| yoctozepto | it's already much much nicer | 08:49 |
| yoctozepto | octavia-certificates is semi-production (safe to use most of the time but not completely production quality - for that one should use a proper CA) | 08:50 |
| yoctozepto | and we coordinate config | 08:50 |
| yoctozepto | mgoddard, wuchunyang: please follow up on https://review.opendev.org/745997 (I failed to save my comments from an open tab lol) | 08:53 |
| patchbot | patch 745997 - kolla-ansible - add octavia openrc file - 12 patch sets | 08:53 |
| wuchunyang | yeah， i agree with this. i often need to poc octavia, so it would nice to support a simply way for octavia. | 08:53 |
| wuchunyang | 16:48 (mgoddard) that would at least allow us to merge the test :) and CI will work too | 08:54 |
| wuchunyang | yoctozepto will update soon | 08:57 |
| *** jan00 has joined #openstack-kolla | 08:57 | |
| yoctozepto | wuchunyang: please submit another patch (this one is close to merge, let's not break it now) | 08:57 |
| * yoctozepto back to local hacking | 08:58 | |
| wuchunyang | ok , thanks will update soon | 08:59 |
| jan00 | hey, we just tripped over https://bugs.launchpad.net/keystone/+bug/1895723 and realised that our tokens were quite old. I believe the cronjob in the 'keystone_fernet' container is never run on ubuntu because the 'cron_path' in 'ansible/roles/keystone/templates/keystone-fernet.json.j2' should be '/var/spool/cron/crontabs/root' rather than | 09:02 |
| openstack | Launchpad bug 1895723 in kolla-ansible ussuri "Keystone is restarting due to stale primary key" [High,Fix committed] - Assigned to Radosław Piliszek (yoctozepto) | 09:02 |
| jan00 | '/var/spool/cron/crontabs/root/fernet-cron' | 09:02 |
| yoctozepto | mnasiadka: ^ | 09:03 |
| *** ricolin_ has quit IRC | 09:04 | |
| *** LinPeiWen has quit IRC | 09:05 | |
| openstackgerrit | Merged openstack/kolla-ansible master: add octavia openrc file https://review.opendev.org/745997 | 09:10 |
| mgoddard | mnasiadka, yoctozepto: could we have an ultra fast rotation in CI that triggers within a job? | 09:11 |
| mgoddard | ideally initial tokens would become invalid before the end of the job | 09:11 |
| mgoddard | rotate once per second? | 09:11 |
| *** kevko has quit IRC | 09:24 | |
| *** kevko has joined #openstack-kolla | 09:24 | |
| openstackgerrit | wu.chunyang proposed openstack/kolla-ansible master: remove redundant line and replace octavia user https://review.opendev.org/757039 | 09:28 |
| *** LinPeiWen has joined #openstack-kolla | 09:28 | |
| wuchunyang | yoctozepto help review again, thanks. trivial fix | 09:29 |
| openstackgerrit | wu.chunyang proposed openstack/kolla-ansible master: DNM[test] https://review.opendev.org/757041 | 09:30 |
| openstackgerrit | wu.chunyang proposed openstack/kolla-ansible master: [test] https://review.opendev.org/757042 | 09:32 |
| *** yankcrime has joined #openstack-kolla | 09:32 | |
| mnasiadka | mgoddard: we could but then fernet tokens would be invalid after 3 seconds :) | 09:38 |
| mgoddard | mnasiadka: that was a joke | 09:38 |
| mgoddard | 10-15m could work though | 09:39 |
| mnasiadka | maybe, if we merge the logging and backport it as part of a bug troubleshooting, at least we could understand what is happening | 09:39 |
| mnasiadka | but I really hate that crond only logs to syslog | 09:39 |
| mgoddard | jan00's assessment sounds plausible | 09:41 |
| mnasiadka | I would prefer to get it verified, which shouldn't be too complicated | 09:44 |
| openstackgerrit | Pierre Riteau proposed openstack/kayobe stable/train: Document migration of seed VM to CentOS 8 https://review.opendev.org/757056 | 09:45 |
| openstackgerrit | Pierre Riteau proposed openstack/kayobe stable/train: Document migration of seed VM to CentOS 8 https://review.opendev.org/757056 | 09:47 |
| yoctozepto | wuchunyang: thx, done | 09:50 |
| yoctozepto | mgoddard, mnasiadka, jan00: yeah, we've definitely had to have different paths for the two distro families | 09:51 |
| *** wuchunyang has quit IRC | 10:07 | |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: docs: Add skeleton for scenario docs https://review.opendev.org/748582 | 10:15 |
| mnasiadka | yoctozepto: we could put cron files into /etc/cron.d, it seems both debuntu and centos reads those in the same manner | 10:17 |
| mnasiadka | but we would need to change the format of the file (include user name) | 10:17 |
| mnasiadka | although ubuntu manual claims system administrator should use /etc/crontab, not /etc/cron.d :D | 10:18 |
| mnasiadka | ok, for sure the path on Ubuntu is at least wrong | 10:23 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: Fix fernet cron path on Ubuntu/Debian https://review.opendev.org/757076 | 10:30 |
| mnasiadka | 1..2..3.. testing ^^ | 10:30 |
| jan00 | I've just did `rm -rf /var/spool/cron/crontabs/root` inside the keystone-fernet container and then `crontab -e` and it was placed at `/var/spool/cron/crontabs/root`, but `/etc/crontab` or `/etc/cron.d` should work aswell | 10:33 |
| *** ewimmer__ has quit IRC | 10:38 | |
| mnasiadka | jan00: to use /etc/crontab we would need to copy contents from the original and add our line, there's no convenient way to add a line to it on startup - I just changed the path in change 757076 and decreased fernet cron timers so we can see if it's rotating in the CI change. | 10:39 |
| mnasiadka | I wonder if that path ever worked on Ubuntu... | 10:39 |
| mnasiadka | mgoddard, yoctozepto: what's the plan for this change https://review.opendev.org/#/c/746384/17? I'm having a slight of a problem approving it without a CI, but maybe we should push it as ,,tech preview''? | 10:42 |
| patchbot | patch 746384 - kolla-ansible - Add support for GlusterFS NFS Manila backend - 17 patch sets | 10:42 |
| mnasiadka | or it's too late and we push it to W? | 10:42 |
| mgoddard | mnasiadka: I haven't really looked at that one | 10:43 |
| mgoddard | haven't heard an FFE request | 10:43 |
| mnasiadka | so let's mark it as -2 priority, I can work on adding a CI job for Manila-glusterfs in W, if that feature makes sense | 10:44 |
| mnasiadka | mgoddard: https://review.opendev.org/#/c/756083/ - passes now | 10:44 |
| patchbot | patch 756083 - kolla-ansible - Add log for cron script fernet-rotate.sh - 11 patch sets | 10:44 |
| mgoddard | mnasiadka: we don't have tests for any of the other manila backends do we? | 10:44 |
| mgoddard | ceph would be nice | 10:45 |
| mnasiadka | mgoddard: no, we don't - I have ceph on my lengthy roadmap :) | 10:45 |
| *** priteau has joined #openstack-kolla | 10:49 | |
| *** zzzeek has quit IRC | 10:52 | |
| *** e0ne has joined #openstack-kolla | 10:52 | |
| *** zzzeek has joined #openstack-kolla | 10:54 | |
| *** jan00 has quit IRC | 11:07 | |
| *** zzzeek has quit IRC | 11:17 | |
| *** jan00 has joined #openstack-kolla | 11:17 | |
| kevko | if you want to help with ceph CI ..I can give a hand ..we have automatic ceph deploy in our CI inside company | 11:17 |
| *** zzzeek has joined #openstack-kolla | 11:18 | |
| mgoddard | kevko: thanks. We use ceph-ansible in CI currently, only we don't test manila | 11:23 |
| *** dciabrin_ has joined #openstack-kolla | 11:33 | |
| *** skramaja has quit IRC | 11:34 | |
| *** dciabrin has quit IRC | 11:34 | |
| *** hrw has quit IRC | 11:34 | |
| *** dking has quit IRC | 11:34 | |
| *** rphillips has quit IRC | 11:34 | |
| *** rockey has quit IRC | 11:34 | |
| *** hrww has joined #openstack-kolla | 11:34 | |
| *** hrww is now known as hrw | 11:34 | |
| *** rphillips has joined #openstack-kolla | 11:34 | |
| kevko | mgoddard: ah, ok, same on my side :) | 11:35 |
| hrw | maybe you will know... any hints how to satisfy ansible yaml parser? fails on: cmd: "make install DESTDIR=/somewhere/" | 11:37 |
| *** skramaja has joined #openstack-kolla | 11:40 | |
| *** dking has joined #openstack-kolla | 11:40 | |
| *** rockey has joined #openstack-kolla | 11:40 | |
| *** jbadiapa has joined #openstack-kolla | 11:45 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Add log for cron script fernet-rotate.sh https://review.opendev.org/756083 | 12:04 |
| *** Fl1nt has joined #openstack-kolla | 12:04 | |
| Fl1nt | Hi everyone! | 12:04 |
| Fl1nt | mgoddard, I found out a bug on the haproxy role I think. It's related to the precheck task. | 12:05 |
| mnasiadka | hrw: what is the error from ansible yaml parser? | 12:05 |
| Fl1nt | if you call kolla-ansible prechecks without any limit, the set facts about whether we can run haproxy/keepalived prechecks. | 12:06 |
| Fl1nt | on an existing deployment | 12:06 |
| Fl1nt | will skip them | 12:06 |
| Fl1nt | whereas if precheck task is implicitely called from an upgrade task for instance | 12:07 |
| Fl1nt | it won't skip them | 12:07 |
| Fl1nt | additionally | 12:07 |
| Fl1nt | when we copy the HAProxy certs, we use the src: {{ kolla_fqdn_external/internal_cert }} and dst: {{ node_config }}/haproxy/{{ item }} | 12:09 |
| Fl1nt | which is then working because we take from /etc/kolla/config or the CICD/deployment job workspace vars that the user set and push it onto controllers. | 12:10 |
| Fl1nt | but later on | 12:10 |
| hrw | mnasiadka: sorted out. | 12:10 |
| Fl1nt | within the precheck task | 12:10 |
| hrw | mnasiadka: forgot 'command:' above | 12:10 |
| mnasiadka | hrw: ah | 12:10 |
| Fl1nt | it will never work for users that will have {{ kolla_fqdn_external_cert }} variable with a value different than: {{ node_config_directory }}/haproxy/haproxy.pem | 12:11 |
| Fl1nt | because of the local_run from the controller itself that doesn't contain this /etc/kolla/config/ or any customised directory. | 12:12 |
| Fl1nt | THIS: will always be false if you don't put your certificate variable {{ kolla_fqdn_external_cert }} under: /etc/kolla/haproxy/haproxy.pem path | 12:14 |
| Fl1nt | consequently, if you've got a platform, that use node_custom_config within kolla_external_fqdn and that you deploy it, it will work. | 12:15 |
| Fl1nt | but once you want to upgrade it | 12:15 |
| Fl1nt | it won't work anymore. | 12:15 |
| Fl1nt | Do we consider this as an issue? | 12:16 |
| Fl1nt | From my point of view, that is two issues, one about the fact that precheck step called standalone wont actually correctly check anything for this role as it will skip the steps. | 12:17 |
| Fl1nt | and the second is about the path management. | 12:17 |
| mnasiadka | Fl1nt: that's an excellent write up, have you thought about raising a bug and even fixing it? :) | 12:18 |
| Fl1nt | yep | 12:18 |
| Fl1nt | but I wanted to get your insight about that first | 12:18 |
| Fl1nt | as I may well have missed a critical var/process somewhere ^^ | 12:18 |
| Fl1nt | and when I'm saying 'your' insight I mean, the whole team willing to validate my investigations ^^ | 12:20 |
| mgoddard | Fl1nt: don't really have time to grok it right now | 12:22 |
| *** kevko has quit IRC | 12:22 | |
| Fl1nt | sure no problems ^^ | 12:29 |
| mgoddard | Fl1nt: I tried to read it again, still not sure I understand the problem | 12:29 |
| mgoddard | simple example? | 12:29 |
| Fl1nt | let say you have a platform based on Train but using CentOS7 as container base image, and you want to use your newly built CentOS8 images. | 12:30 |
| Fl1nt | let say within that platform, you're using ElasticSearch (5.X deployed as you use CentOS7). | 12:30 |
| Fl1nt | in order for you to deploy that new platform, you'll need to use both upgrade task action in order to trigger the 5.x to 6.x migration process of the ElasticSearch role | 12:31 |
| Fl1nt | and | 12:31 |
| Fl1nt | deploy task | 12:31 |
| Fl1nt | in order to switch you others services containers to CentOS 8 based images. | 12:31 |
| mgoddard | ah | 12:32 |
| mgoddard | this is why we support 6.x on CentOS 7 | 12:32 |
| mgoddard | maybe this needs documenting | 12:32 |
| mgoddard | I'm not sure what this has to do with haproxy though | 12:33 |
| Fl1nt | Well, One need to switch to CentOS 8 at some point so, even with that in mind we still get an issue ^^ | 12:33 |
| Fl1nt | so | 12:33 |
| Fl1nt | with that in mind, this platform is also relying on TLS Haproxy (Ext/Int) | 12:33 |
| Fl1nt | so | 12:33 |
| Fl1nt | here we have two distinct bugs: | 12:34 |
| Fl1nt | the first one came from the fact that when you invoke the precheck task directly from the kolla-ansible prechecks action | 12:35 |
| Fl1nt | the HAProxy checks tasks are skipped | 12:36 |
| Fl1nt | because of this: https://opendev.org/openstack/kolla-ansible/src/tag/9.2.0/ansible/roles/haproxy/tasks/precheck.yml#L29 and this: https://opendev.org/openstack/kolla-ansible/src/tag/9.2.0/ansible/roles/haproxy/tasks/precheck.yml#L32 | 12:37 |
| Fl1nt | which is kind of funny because this bug actually let us deploy correctly our platform and especially the TLS part of it. | 12:38 |
| mgoddard | because you are using --limit | 12:38 |
| Fl1nt | nope sir ^^ | 12:38 |
| Fl1nt | hence why there is a bug ^^ | 12:39 |
| mgoddard | oh, because haproxy is running | 12:39 |
| Fl1nt | yes ^^ | 12:39 |
| mgoddard | we don't need to check for the VIP if haproxy is already running | 12:39 |
| mgoddard | or rather, we can't | 12:39 |
| yoctozepto | yeah, we kinda can't easily/reliably | 12:40 |
| Fl1nt | the issue here is that fact actually inhibit the run of the certs path checks etc too | 12:41 |
| mgoddard | ok, I see | 12:42 |
| Fl1nt | as we always preform a standalone call for precheck before any deployment, that is kind of dangerous as it skip a test that can actually prevent for the platform to be reached ^^ | 12:42 |
| mgoddard | Fl1nt: https://opendev.org/openstack/kolla-ansible/src/tag/9.2.0/ansible/roles/haproxy/tasks/precheck.yml#L34 | 12:43 |
| mgoddard | it doesn't reference haproxy_vip_prechecks | 12:44 |
| Fl1nt | yep, and that's what let me know there is something more | 12:45 |
| mgoddard | I still don't understand the problem | 12:46 |
| Fl1nt | the problem is, those check tasks are all skipped when running a standalone kolla-ansible prechecks action. | 12:46 |
| mgoddard | if haproxy is running | 12:48 |
| mgoddard | the certificate check still runs | 12:48 |
| mgoddard | only the port checks are skipped | 12:48 |
| Fl1nt | ok, I've collected some logs | 12:53 |
| Fl1nt | here is the standalone precheck action relevant part: | 12:54 |
| Fl1nt | http://paste.openstack.org/show/ruLTIF1AQ7MbW2tfojKb/ | 12:54 |
| Fl1nt | and the command used to launch it: `kolla-ansible -i $CONFIG_DIR/multinode --configdir $CONFIG_DIR --passwords $CONFIG_DIR/passwords.yml prechecks` | 12:54 |
| mgoddard | TASK [haproxy : Checking if external haproxy certificate exists] *************** | 12:55 |
| mgoddard | ok: [emea-cp01 -> localhost] | 12:55 |
| mgoddard | TASK [haproxy : Fail if external haproxy certificate is absent] **************** | 12:55 |
| mgoddard | skipping: [emea-cp01] | 12:55 |
| mgoddard | first task ok | 12:55 |
| Fl1nt | hold on | 12:55 |
| mgoddard | fail task skipped | 12:55 |
| Fl1nt | yeah wait a minute ^^ | 12:55 |
| Fl1nt | now | 12:55 |
| Fl1nt | if I do: `kolla-ansible -i $CONFIG_DIR/multinode --configdir $CONFIG_DIR --passwords $CONFIG_DIR/passwords.yml upgrade` | 12:55 |
| Fl1nt | the upgrade action within HAProxy | 12:56 |
| Fl1nt | call the config.yml | 12:56 |
| Fl1nt | and trigger a check step | 12:57 |
| * Fl1nt goes collect the logs | 12:57 | |
| Fl1nt | so, when using precheck as a standalone, it's OK, it find the certs | 13:01 |
| Fl1nt | but once you call upgrade | 13:01 |
| Fl1nt | http://paste.openstack.org/show/IjggZr2HMFg70a0zRYfc/ | 13:01 |
| Fl1nt | it fail but it's OK | 13:01 |
| Fl1nt | because it is the intended behavior | 13:01 |
| Fl1nt | as | 13:01 |
| Fl1nt | the used check var | 13:02 |
| Fl1nt | can't be found on remote host | 13:02 |
| *** rpittau is now known as rpittau|afk | 13:03 | |
| Fl1nt | so, the first task: https://opendev.org/openstack/kolla-ansible/src/tag/9.2.0/ansible/roles/haproxy/tasks/precheck.yml#L34 is always true | 13:03 |
| Fl1nt | because it register within haproxy_cert_file the return of the check | 13:04 |
| Fl1nt | and the second | 13:04 |
| Fl1nt | is either valid when the stat.exist and kolla_enable_tls is true | 13:05 |
| mgoddard | where is the check step called from config.yml? | 13:05 |
| Fl1nt | hum... yeah, you're right it's weird, it's not | 13:07 |
| mgoddard | :) | 13:08 |
| Fl1nt | hold on checking something... | 13:08 |
| Fl1nt | hum, find out what's going on | 13:10 |
| Fl1nt | the CICD pipeline call prechecks before upgrade too as inherit from the parent pipeline that explicitly call for it. | 13:10 |
| Fl1nt | so, the first bug isn't a bug ^^ | 13:11 |
| Fl1nt | I'll dig deeper ^^ | 13:11 |
| Fl1nt | but that doesn't explain why to identical command comes with two different behaviors :( | 13:13 |
| Fl1nt | re-running it ^^ | 13:13 |
| Fl1nt | to see if it was something temporary or a constant. | 13:13 |
| mgoddard | Fl1nt: maybe your pipeline is actually lacking the cert and the precheck is doing its job? :D | 13:14 |
| Fl1nt | it's checked out from the same git repository, it can't be there at deploy run and not at upgrade ^^ But I'm checking, maybe there was a network hiccups or something. | 13:16 |
| Fl1nt | all right, found out... | 13:20 |
| Fl1nt | I'll have to slap someone with a big trout ^^ | 13:20 |
| openstackgerrit | Pierre Riteau proposed openstack/kayobe stable/train: Document migration of seed VM to CentOS 8 https://review.opendev.org/757056 | 13:21 |
| *** wuchunyang has joined #openstack-kolla | 13:23 | |
| *** TrevorV has joined #openstack-kolla | 13:28 | |
| Fl1nt | mgoddard, one last thing that bug me up, how is the precheck task: "Checking if external haproxy certificate exists" able to found out a valid cert on our remote server when it check for {{ kolla_external_cert_fqdn }} which is | 13:29 |
| Fl1nt | equal to | 13:29 |
| mgoddard | Fl1nt: delegate_to: localhost | 13:29 |
| Fl1nt | wait | 13:29 |
| Fl1nt | Am I missing something here? https://opendev.org/openstack/kolla-ansible/src/tag/9.2.0/ansible/roles/haproxy/tasks/precheck.yml#L34 | 13:29 |
| mgoddard | Fl1nt: local_action == delegate_to: localhost | 13:30 |
| Fl1nt | aaaaaaaah ok, I though local_action was actually doing something else. | 13:30 |
| Fl1nt | ok, got it, sorry for all that confusion... | 13:32 |
| mgoddard | np | 13:32 |
| Fl1nt | going back to my cloudkitty funny party ^^ | 13:32 |
| Fl1nt | btw hrw here is your worst nightmare: https://imgur.com/a/YSzBTeL | 13:37 |
| hrw | Fl1nt: trust me, you do not want to work with my worst nightmare hw | 13:42 |
| Fl1nt | Too late I'm already ^^ | 13:42 |
| Fl1nt | currently installing them using fedora. | 13:43 |
| hrw | Fl1nt: rpi4 is just yet another sbc. category of devices where I usually do not care | 13:44 |
| hrw | Fl1nt: edb9301 was painful. early 2.6 on zaurus was painful. nhk15 kernel trees merge was painful. | 13:45 |
| Fl1nt | Is that because of the BCM ? | 13:47 |
| Fl1nt | because it seems like the VideoCore VI was open sourced isn't ? | 13:48 |
| hrw | Fl1nt: I do aarch64 servers. you know, such multicore computers which comes in 19" cases to put in racks. | 13:59 |
| hrw | Fl1nt: with proper storage, sane memory sizes etc | 13:59 |
| hrw | Fl1nt: hardware which lacks nvme, lacks sata, has <32GB ram is not interesting | 14:00 |
| hrw | Fl1nt: simple enough? | 14:00 |
| Fl1nt | ok | 14:04 |
| hrw | good | 14:07 |
| hrw | I am aware that people use SBCs for things normal users get servers. | 14:07 |
| hrw | or to say it nicer: I am aware that people use SBCs for things normally users get servers. | 14:08 |
| Fl1nt | wasn chating about that to pissed you off. | 14:08 |
| hrw | I am pissed off. | 14:10 |
| hrw | I am not pissed off. | 14:10 |
| hrw | ;d | 14:10 |
| Fl1nt | it's just that as you told it, people don't buy expensive ARM Servers to get some labs or recreation/side projects and tests about distributivity of something. | 14:10 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Add log for cron script fernet-rotate.sh https://review.opendev.org/756083 | 14:11 |
| hrw | Fl1nt: rpi4 is terrible choice. vendor does not give a shit about upstreaming their stuff or doing QA | 14:11 |
| hrw | Fl1nt: you spend few hundred $€ on rpi instead of going to something saner. your choice. | 14:11 |
| mnasiadka | hrw: what is saner? ;) | 14:15 |
| Fl1nt | It's supported on Linux, there is a distribution, with a package manager and packages that I'm interested in, I don't care about using 1 server, I need to test relationship between things to a scale that using VM isn't possible at a consumer grade level. | 14:15 |
| Fl1nt | And if it was for an enterprise grade budget, I'wd have gone with an AMD/Intel x86 as it's still more powerfull and standard over ARM. | 14:17 |
| hrw | Fl1nt: try using aarch64 server one day. they are highly standardized | 14:18 |
| hrw | we made them boring | 14:18 |
| hrw | Fl1nt: rpi4 is not supported in fedora 33 | 14:18 |
| hrw | it may be supported in 34 | 14:19 |
| hrw | it is 'somekind of working' state so not listed as supported device | 14:19 |
| Fl1nt | sure, we've got few of them in test thanks to your last recommandation, but they're still away from x86 in terms of performance and GPU support. | 14:20 |
| Fl1nt | and on that chatting, the critical part was "distribution". If you want to do it on a relatively controlled budget, you're let with few options and RPi is the most balanced one. wasn't pinching you or something, was just chatting. | 14:21 |
| hrw | mkey | 14:21 |
| * hrw -> other container builds | 14:21 | |
| *** wuchunyang has quit IRC | 14:23 | |
| *** k_mouza_ has quit IRC | 14:25 | |
| *** k_mouza has joined #openstack-kolla | 14:26 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: Switch to Victoria stable branches https://review.opendev.org/755339 | 14:34 |
| *** e0ne_ has joined #openstack-kolla | 14:39 | |
| *** e0ne has quit IRC | 14:43 | |
| *** k_mouza has quit IRC | 14:52 | |
| *** bsanjeewa has joined #openstack-kolla | 15:01 | |
| openstackgerrit | Merged openstack/kayobe master: Synchronise global extra variables files with defaults https://review.opendev.org/756586 | 15:01 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/ussuri: Performance: use import_tasks for register and bootstrap https://review.opendev.org/757124 | 15:02 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: Performance: use import_tasks for register and bootstrap https://review.opendev.org/757125 | 15:03 |
| *** k_mouza has joined #openstack-kolla | 15:04 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/stein: Performance: use import_tasks for register and bootstrap https://review.opendev.org/757126 | 15:04 |
| *** skramaja has quit IRC | 15:10 | |
| *** nikparasyr has left #openstack-kolla | 15:18 | |
| *** k_mouza has quit IRC | 15:28 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: docs: Add information on migrating to CentOS 8 https://review.opendev.org/757133 | 15:30 |
| mgoddard | Fl1nt: your ES issue reminded me I needed to document it | 15:30 |
| mgoddard | ^ | 15:30 |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: Finish off removing Congress https://review.opendev.org/757136 | 15:33 |
| *** k_mouza has joined #openstack-kolla | 15:35 | |
| *** bengates has quit IRC | 15:35 | |
| *** bengates has joined #openstack-kolla | 15:36 | |
| *** k_mouza has quit IRC | 15:39 | |
| *** bengates has quit IRC | 15:41 | |
| *** k_mouza has joined #openstack-kolla | 15:41 | |
| *** maharg101 has quit IRC | 15:50 | |
| openstackgerrit | Pierre Riteau proposed openstack/kolla-ansible stable/train: docs: Add information on migrating to CentOS 8 https://review.opendev.org/757133 | 15:54 |
| *** k_mouza has quit IRC | 15:55 | |
| *** k_mouza has joined #openstack-kolla | 15:57 | |
| openstackgerrit | Pierre Riteau proposed openstack/kolla-ansible stable/train: docs: Add information on migrating to CentOS 8 https://review.opendev.org/757133 | 15:57 |
| openstackgerrit | Pierre Riteau proposed openstack/kayobe stable/ussuri: Configure bifrost to use firewalld trusted zone https://review.opendev.org/757143 | 16:01 |
| Fl1nt | ah ah ah glad it helped ^^ | 16:05 |
| Fl1nt | so, I did the upgrade, how am I supposed to force k-a to use ES6 now? with elasticsearch_use_v6 to true ? | 16:05 |
| *** zijlboot has quit IRC | 16:05 | |
| *** priteau has quit IRC | 16:06 | |
| *** k_mouza has quit IRC | 16:06 | |
| *** e0ne_ has quit IRC | 16:07 | |
| mgoddard | Fl1nt: if you are already on CentOS 8 it uses v6 always | 16:10 |
| mgoddard | on C7 yes, you can set that flag (and the one for kibana) | 16:10 |
| Fl1nt | Cos8 on host or containers? | 16:10 |
| mgoddard | both. host & container should match | 16:11 |
| Fl1nt | I've got host on C7 and containers on C8 for now | 16:11 |
| mgoddard | but the flag uses the host to set the default | 16:11 |
| mgoddard | we don't test/support that | 16:11 |
| Fl1nt | let's test it live :p | 16:11 |
| mgoddard | Fl1nt: https://opendev.org/openstack/kolla-ansible/src/branch/master/specs/centos8-migration.rst | 16:12 |
| Fl1nt | it works \o/ | 16:13 |
| Fl1nt | tempest validated ^^ and functional test validated :D | 16:13 |
| Fl1nt | cloudkitty installed and using ES6 index :D Perfect! | 16:13 |
| mgoddard | there was a bug raised recently about it | 16:13 |
| mgoddard | haproxy | 16:14 |
| Fl1nt | I'll have to update the hosts soon but I need to be careful with this one as I've got beta workload on it ^^ | 16:14 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: docs: Add information on migrating to CentOS 8 https://review.opendev.org/757133 | 16:18 |
| *** kevko has joined #openstack-kolla | 16:34 | |
| *** maharg101 has joined #openstack-kolla | 16:38 | |
| *** gfidente is now known as gfidente|afk | 16:40 | |
| *** priteau has joined #openstack-kolla | 16:41 | |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Add support for encrypting backend Neutron API Server https://review.opendev.org/756367 | 16:43 |
| *** maharg101 has quit IRC | 16:45 | |
| openstackgerrit | Pierre Riteau proposed openstack/kolla-ansible master: Apply bool filter to all enable_prometheus_* variables https://review.opendev.org/757154 | 16:51 |
| *** gfidente|afk has quit IRC | 16:52 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: docs: Add information on migrating to CentOS 8 https://review.opendev.org/757133 | 16:53 |
| *** Fl1nt has quit IRC | 17:28 | |
| *** jbadiapa has quit IRC | 17:56 | |
| *** priteau has quit IRC | 17:58 | |
| *** priteau has joined #openstack-kolla | 18:03 | |
| *** cah_link has quit IRC | 18:05 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: Switch to Victoria stable branches https://review.opendev.org/755339 | 18:06 |
| *** priteau has quit IRC | 18:08 | |
| *** vishalmanchanda has quit IRC | 18:15 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: docs: Add information on migrating to CentOS 8 https://review.opendev.org/757133 | 18:28 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: docs: Add information on migrating to CentOS 8 https://review.opendev.org/757133 | 18:31 |
| *** k_mouza has joined #openstack-kolla | 18:32 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: docs: Add information on migrating to CentOS 8 https://review.opendev.org/757133 | 18:32 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: docs: Add information on migrating to CentOS 8 https://review.opendev.org/757133 | 18:35 |
| *** priteau has joined #openstack-kolla | 18:42 | |
| *** maharg101 has joined #openstack-kolla | 18:43 | |
| *** maharg101 has quit IRC | 18:47 | |
| *** priteau has quit IRC | 18:48 | |
| *** bsanjeewa_ has joined #openstack-kolla | 18:51 | |
| *** bsanjeewa has quit IRC | 18:54 | |
| *** priteau has joined #openstack-kolla | 19:14 | |
| *** k_mouza has quit IRC | 19:20 | |
| *** priteau has quit IRC | 19:27 | |
| *** cah_link has joined #openstack-kolla | 19:34 | |
| *** TrevorV has quit IRC | 19:42 | |
| *** zzzeek has quit IRC | 19:49 | |
| *** zzzeek has joined #openstack-kolla | 19:51 | |
| *** zijlboot has joined #openstack-kolla | 20:36 | |
| *** maharg101 has joined #openstack-kolla | 20:43 | |
| *** maharg101 has quit IRC | 20:48 | |
| *** zijlboot has quit IRC | 21:55 | |
| *** zijlboot has joined #openstack-kolla | 21:56 | |
| *** zijlboot has quit IRC | 22:00 | |
| *** JustAFlerkin has joined #openstack-kolla | 22:18 | |
| JustAFlerkin | hi, having an issue with the kolla/ubuntu-source-bootstrap_gnocchi:master image. it complains about not having a rados library. 8-/ Using a ceph backend. | 22:19 |
| JustAFlerkin | did some googling and didn't find any viable workarounds. | 22:19 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!