| *** gregoryo has joined #openstack-kolla | 01:13 | |
| openstackgerrit | Chason Chan proposed openstack/kolla-ansible master: ubuntu: update configuration Stein UCA https://review.opendev.org/674241 | 02:33 |
|---|---|---|
| openstackgerrit | Chason Chan proposed openstack/kolla-ansible master: ubuntu: update configuration Stein UCA https://review.opendev.org/674241 | 03:54 |
| openstackgerrit | Merged openstack/kolla master: Upgrade Monasca to InfluxDB 1.7 https://review.opendev.org/658367 | 04:57 |
| openstackgerrit | wu.chunyang proposed openstack/kolla-cli master: Do not always copy files https://review.opendev.org/675813 | 05:16 |
| *** pcaruana has joined #openstack-kolla | 05:29 | |
| *** pcaruana has quit IRC | 05:37 | |
| *** cah_link has joined #openstack-kolla | 05:45 | |
| *** janki has joined #openstack-kolla | 05:48 | |
| *** pcaruana has joined #openstack-kolla | 05:50 | |
| haria | hrw: Thanks! that worked | 05:54 |
| haria | I got the all-in-one installation completed, but the dashboard is not accessible. I can see traffic coming in to the horizon container using tcpdump, but nothing going out... | 05:55 |
| haria | (horizon)[root@os37 apache2]# tcpdump dst port 443 | 05:58 |
| haria | tcpdump: verbose output suppressed, use -v or -vv for full protocol decode | 05:58 |
| haria | listening on enp4s0f0, link-type EN10MB (Ethernet), capture size 262144 bytes | 05:58 |
| haria | 01:55:32.783995 IP 10.1.4.240.56714 > os37.mydomain.https: Flags [S], seq 3352333923, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0 | 05:58 |
| haria | 01:55:32.784220 IP 10.1.4.240.56715 > os37.mydomain.https: Flags [S], seq 717297450, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0 | 05:58 |
| haria | 01:55:33.023504 IP 10.1.4.240.56716 > os37.mydomain.https: Flags [S], seq 147624411, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0 | 05:58 |
| haria | 01:55:33.282042 IP 10.1.4.240.56715 > os37.mydomain.https: Flags [S], seq 717297450, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0 | 05:58 |
| haria | 01:55:33.290826 IP 10.1.4.240.56714 > os37.mydomain.https: Flags [S], seq 3352333923, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0 | 05:58 |
| haria | 01:55:33.522300 IP 10.1.4.240.56716 > os37.mydomain.https: Flags [S], seq 147624411, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0 | 05:58 |
| haria | 01:55:33.782986 IP 10.1.4.240.56715 > os37.mydomain.https: Flags [S], seq 717297450, win 8192, options [mss 1460,nop,nop,sackOK], length 0 | 05:58 |
| haria | 01:55:33.791816 IP 10.1.4.240.56714 > os37.mydomain.https: Flags [S], seq 3352333923, win 8192, options [mss 1460,nop,nop,sackOK], length 0 | 05:58 |
| haria | 01:55:34.024272 IP 10.1.4.240.56716 > os37.mydomain.https: Flags [S], seq 147624411, win 8192, options [mss 1460,nop,nop,sackOK], length 0 | 05:58 |
| haria | Please ignore, I was trying https, its working with http!! | 06:05 |
| *** dpawlik has joined #openstack-kolla | 06:08 | |
| hrw | morning | 06:15 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: Set 'distro_python_version' variable https://review.opendev.org/675581 | 06:23 |
| mnasiadka | morning | 06:47 |
| openstackgerrit | Chason Chan proposed openstack/kolla-ansible master: Update python_path for "source" install type https://review.opendev.org/675822 | 06:51 |
| openstackgerrit | Chason Chan proposed openstack/kolla-ansible master: update horizon configuration for python3 migrating https://review.opendev.org/674241 | 06:53 |
| openstackgerrit | Chason Chan proposed openstack/kolla-ansible master: update horizon configuration for python3 migrating https://review.opendev.org/674241 | 06:56 |
| *** ivve has joined #openstack-kolla | 07:05 | |
| *** luksky has joined #openstack-kolla | 07:09 | |
| openstackgerrit | Chason Chan proposed openstack/kolla-ansible master: Update python_path for "source" install type https://review.opendev.org/675822 | 07:17 |
| hrw | https://review.opendev.org/#/c/675581/ folks | 07:18 |
| mnasiadka | geez, fluentd vs td-agent can't be more complex? | 07:35 |
| hrw | ;D | 07:39 |
| hrw | mnasiadka: send patch which will kill td-agent? | 07:39 |
| mnasiadka | hrw: worse, I need to move from fluentd to td-agent on CentOS, and all the paths are crazy... | 07:39 |
| *** zbr is now known as zbr|flu | 07:47 | |
| *** rpittau|afk is now known as rpittau | 07:54 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Change fluentd to td-agent on CentOS https://review.opendev.org/675614 | 07:56 |
| hrw | mnasiadka: sorry ale pewnie gdzieś brakuje !x86 ;( | 07:58 |
| mnasiadka | hrw: alez jestem pewien :) | 07:58 |
| mnasiadka | hrw: a tak serio to zaraz dodam zeby !x86 CentOS uzywal CentOS opstools... muzeum ale coz. | 07:59 |
| hrw | cóż | 08:01 |
| hrw | arm64 node in linaro-london infra got bumped to 16 vcpu. hope it will be enough for ci | 08:03 |
| hrw | INFO:kolla.common.utils.fluentd:No package td-agent available. | 08:08 |
| hrw | INFO:kolla.common.utils.fluentd:Error: Not tolerating missing names on install, stopping. | 08:08 |
| hrw | like expected | 08:08 |
| yoctozepto | morning | 08:10 |
| stingrayza | morning folks o/ | 08:12 |
| stingrayza | happy monday | 08:13 |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: DNM: Use python 3.6 in CentOS source images via IUS https://review.opendev.org/675367 | 08:15 |
| *** janki has quit IRC | 08:16 | |
| *** janki has joined #openstack-kolla | 08:17 | |
| *** jchhatbar has joined #openstack-kolla | 08:18 | |
| *** janki has quit IRC | 08:22 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: DNM: Use python 3.6 in CentOS source images via IUS https://review.opendev.org/675367 | 08:27 |
| *** gregoryo has quit IRC | 08:40 | |
| *** mgkwill has quit IRC | 08:42 | |
| *** jschluet has quit IRC | 08:42 | |
| *** jchhatbar has quit IRC | 08:44 | |
| *** openstackgerrit has quit IRC | 08:45 | |
| *** mgkwill has joined #openstack-kolla | 08:47 | |
| *** jschluet has joined #openstack-kolla | 08:47 | |
| *** irclogbot_0 has quit IRC | 08:49 | |
| *** irclogbot_2 has joined #openstack-kolla | 08:52 | |
| *** gfidente has joined #openstack-kolla | 08:54 | |
| mgoddard | morning | 09:03 |
| mnasiadka | morning mgoddard | 09:03 |
| mnasiadka | hrw: now it should be a bit better I guess | 09:05 |
| *** rgogunskiy has joined #openstack-kolla | 09:16 | |
| *** Blinkiz has joined #openstack-kolla | 09:17 | |
| hrw | mnasiadka: +1 and comments | 09:17 |
| Blinkiz | Hello. Am having a problem with task Baremetal: Install docker SDK for python. The error is Cannot uninstall 'requests'. https://paste.ubuntu.com/p/v6sSmrg2c8/ | 09:18 |
| Blinkiz | How can I solve this? | 09:19 |
| hrw | Blinkiz: how they got installed? | 09:21 |
| mgoddard | Blinkiz: this normally means you need to use a virtualenv | 09:22 |
| mgoddard | Blinkiz: https://docs.openstack.org/kolla-ansible/latest/user/virtual-environments.html#target-hosts | 09:22 |
| Blinkiz | I do think kolla has installed this. But am not sure. | 09:23 |
| Blinkiz | The control host (the one am deploying from) is running kolla in a virtual environment. Maybe I need to update something... | 09:24 |
| hrw | Blinkiz: login to target machine and uninstall requests | 09:24 |
| mgoddard | Blinkiz: use of a virtualenv on the control host is separate from the target hosts | 09:24 |
| Blinkiz | It seems like on my centos machines the yum package "python-requests" is installed with version 2.6.0. It is the package "cloud-init" that is requiring python-requests | 09:27 |
| yoctozepto | Blinkiz: hmm, you are deploying cloud in a cloud? ;D | 09:28 |
| Blinkiz | package cloud-init is installed by Ubuntu MAAS deployment tool | 09:28 |
| yoctozepto | ah, that explains it | 09:28 |
| Blinkiz | No, am installing on baremetal machines | 09:28 |
| mnasiadka | hrw: your comments are my thoughts just after sending the new patchset, but got distracted with something else :) | 09:28 |
| yoctozepto | the cleanest approach is to use virtualenv on target hosts as suggested by others | 09:28 |
| mnasiadka | yoctozepto: what's wrong with cloud in a cloud? :) | 09:29 |
| yoctozepto | https://docs.openstack.org/kolla-ansible/latest/user/virtual-environments.html#target-hosts | 09:29 |
| Blinkiz | yoctozepto, thank you. I do think that is the best way to avoid problems in the future | 09:29 |
| hrw | Blinkiz: to make it easier: login to target, uninstall python* :D | 09:29 |
| yoctozepto | hrw: I think MAAS might not like cloud-init being removed | 09:29 |
| Blinkiz | hrw, yes, I can uninstall cloud-init, I do not need it now after the machines has been deployed. | 09:30 |
| hrw | my deployment machines were fresh installed to make sure nothing interfere | 09:30 |
| yoctozepto | mnasiadka: nothing, was jutst curious | 09:30 |
| hrw | yoctozepto: I do not care will anyone at canonical cry due to it | 09:30 |
| yoctozepto | hrw: oh, they cry for every replacement of ubuntu with debian :-) | 09:30 |
| hrw | oh ah | 09:31 |
| yoctozepto | as I'm CentOS guy, Foreman runs installation orchestration here ;D I found pxelinux/kickstart templating capabilities very attractive | 09:32 |
| hrw | preseed/kickstarts are great | 09:34 |
| yoctozepto | MAAS thinks otherwise afair | 09:35 |
| yoctozepto | liked the graphics, disliked caps | 09:35 |
| hrw | MAAS - mumbling as a service | 09:36 |
| yoctozepto | massacre* as a service | 09:37 |
| yoctozepto | fixed typo, no problem | 09:38 |
| hrw | yoctozepto: that's US | 09:38 |
| *** bodgix_ has quit IRC | 09:51 | |
| *** bodgix has joined #openstack-kolla | 10:31 | |
| *** openstackgerrit has joined #openstack-kolla | 10:38 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Change fluentd to td-agent on CentOS https://review.opendev.org/675614 | 10:38 |
| *** pcaruana has quit IRC | 10:43 | |
| *** pcaruana has joined #openstack-kolla | 10:43 | |
| yoctozepto | mgoddard: re: Update minimum version of Ansible to 2.6 - why not just bump to 2.8 - this is no show stopper for users imho (this is train) | 10:47 |
| yoctozepto | hrw, mnasiadka: ^ | 10:47 |
| hrw | yoctozepto: distros have 2.7 | 10:48 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Change fluentd to td-agent on CentOS https://review.opendev.org/675614 | 10:48 |
| yoctozepto | hrw: then 2.7 | 10:48 |
| mnasiadka | yoctozepto: well, in master we should always bump up Ansible to latest, because if we don't - then we get in deep shit again :) | 10:50 |
| mnasiadka | yoctozepto: both in kolla-toolbox and outside | 10:50 |
| yoctozepto | mnasiadka: yeah, kolla-toolbox is 100% | 10:53 |
| yoctozepto | if they release 2.9 before train we should try in there too ;D | 10:53 |
| yoctozepto | anyways, for host hrw has a point in that some may want to use distro-provided ansible | 10:54 |
| yoctozepto | on the other hand, it is SO easy to install it from pip that it begs for independence | 10:54 |
| hrw | yoctozepto: then it would probably need to be in requirements.txt | 10:55 |
| yoctozepto | hrw: well, even in docs we recommend pip install ansible | 10:57 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Change fluentd to td-agent on CentOS https://review.opendev.org/675614 | 10:57 |
| mnasiadka | yoctozepto: well, I say we should test latest what distro provides + latest latest Ansible :) | 10:57 |
| hrw | mnasiadka: have you read my 'set fluent_packages' comment? | 10:58 |
| Blinkiz | As a user of kolla-ansible I do prefer that kolla is using preferred ansible version in the virtualenv. It is also in the docs so the impact should be mild. | 10:58 |
| mnasiadka | hrw: yeah, doing three things at the same time - but getting there :) | 10:59 |
| hrw | mnasiadka: also consider setting fluend_user = 'fluentd' at beginning so you then only need to overwrite it in 2 places | 10:59 |
| mnasiadka | hrw: good point | 11:00 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Change fluentd to td-agent on CentOS https://review.opendev.org/675614 | 11:05 |
| *** janki has joined #openstack-kolla | 11:08 | |
| *** whoami-rajat has joined #openstack-kolla | 11:08 | |
| hrw | mnasiadka: +1 and comments. | 11:08 |
| hrw | mnasiadka: looks like +2 next time | 11:09 |
| *** yankcrime has quit IRC | 11:11 | |
| yoctozepto | mnasiadka: more and more comments from me | 11:12 |
| mnasiadka | yeah | 11:12 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Change fluentd to td-agent on CentOS https://review.opendev.org/675614 | 11:12 |
| mnasiadka | yoctozepto: around extend_start script changes... I have a problem that we should deprecate if we remove something? :D | 11:12 |
| yoctozepto | mnasiadka: what are you removing? | 11:13 |
| yoctozepto | the image either works or not, dummy or wrong commands can freely go away | 11:14 |
| mnasiadka | yoctozepto: so, /var/lib/${FLUENTD} exists because packages that we install create it (CentOS opstools, Ubuntu, whatever), but the treasuredata package for Redhat derivatives does not use that at all | 11:14 |
| mnasiadka | yoctozepto: so either I add check if it exists and then change rights, or remove the rights check at all, or move them to k-a :) | 11:15 |
| *** kplant has joined #openstack-kolla | 11:15 | |
| mnasiadka | yoctozepto: s/use/create/ | 11:16 |
| *** yankcrime has joined #openstack-kolla | 11:17 | |
| yoctozepto | mnasiadka: rights should be proper anyway, they are more relevant to upgrades though, which you should probably think about since you are so eager :D | 11:17 |
| hrw | yoctozepto: nice catch on those conditions | 11:17 |
| yoctozepto | hrw: thx | 11:18 |
| yoctozepto | mnasiadka: I seem to have left 3 comments on 3 different versions of change, please read them all :-) | 11:19 |
| *** yankcrime has quit IRC | 11:29 | |
| mnasiadka | yoctozepto: will do, now I’m having breakfast :) | 11:41 |
| yoctozepto | mnasiadka: bon appetit, a bit late ;-) | 11:42 |
| *** henriqueof has joined #openstack-kolla | 11:42 | |
| *** yankcrime has joined #openstack-kolla | 11:45 | |
| *** janki has quit IRC | 11:51 | |
| *** janki has joined #openstack-kolla | 11:53 | |
| mnasiadka | yoctozepto: this is how it ends when you work from home :) | 11:56 |
| hrw | mnasiadka: have you dressed at least? | 11:58 |
| yoctozepto | hrw: never programmed in pyjamas? ;D | 11:59 |
| hrw | yoctozepto: better not ask. | 11:59 |
| hrw | yoctozepto: during 12 years of working at home I probably worked in any set of clothes | 12:00 |
| mnasiadka | hrw: yeah, I have to drive my kid to kindergarten every morning, so I have to :) | 12:00 |
| hrw | :D | 12:00 |
| *** hjensas has joined #openstack-kolla | 12:05 | |
| *** priteau has joined #openstack-kolla | 12:20 | |
| goldyfruit_ | o/ | 12:22 |
| goldyfruit_ | Guys, could you please have a look to https://review.opendev.org/#/c/615715 ? | 12:25 |
| goldyfruit_ | I really would like to see this merge ^^ | 12:25 |
| kplant | ooo masakari | 12:25 |
| mnasiadka | goldyfruit_: basically we have two options, either you add a CI job with this, or we need to test it manually :) | 12:26 |
| mnasiadka | goldyfruit_: guess which option makes the merge faster :) | 12:26 |
| goldyfruit_ | mnasiadka, added already https://review.opendev.org/#/c/616050/ | 12:27 |
| *** edleafe has joined #openstack-kolla | 12:29 | |
| mnasiadka | goldyfruit_: ran a recheck on the CI masakari jobs, the ansible role change looks good - but let's see the final CI results | 12:34 |
| goldyfruit_ | mnasiadka, (fingers crossed) | 12:36 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Change fluentd to td-agent on CentOS https://review.opendev.org/675614 | 12:39 |
| *** jchhatbar has joined #openstack-kolla | 12:47 | |
| *** jchhatbar has quit IRC | 12:48 | |
| *** jchhatbar has joined #openstack-kolla | 12:48 | |
| *** janki has quit IRC | 12:50 | |
| *** _Cyclone_ has joined #openstack-kolla | 12:52 | |
| *** jchhatbar has quit IRC | 12:52 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: DNM: Use python 3.6 in CentOS source images via IUS https://review.opendev.org/675367 | 12:57 |
| hrw | uf. centos/aarch64 ci job runs now on 16 vcpus. let's see will it fit in 3h now | 12:59 |
| kplant | i would offer a few 8180s but those are x86 :p | 13:01 |
| *** jroll has quit IRC | 13:01 | |
| *** jroll has joined #openstack-kolla | 13:02 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: DNM: Test ironic https://review.opendev.org/675893 | 13:06 |
| *** luksky has quit IRC | 13:08 | |
| *** janki has joined #openstack-kolla | 13:09 | |
| mgoddard | hrw, mnasiadka, yoctozepto: on Ansible versions (again) :) | 13:10 |
| mgoddard | kolla-toolbox, we decide what goes in and should use latest | 13:10 |
| hrw | mgoddard: k-t already is at 2.8 | 13:11 |
| mnasiadka | We already do in master | 13:11 |
| mnasiadka | Need to fix docs and then backport | 13:11 |
| mgoddard | for control host, we should be fairly open IMO unless there is a good reason not to be | 13:12 |
| mgoddard | until we have a good reason to bump the minimum, we should leave it | 13:12 |
| mgoddard | a good reason being we depend on a feature in some version | 13:12 |
| mgoddard | or some feature we depend on is broken until some version | 13:13 |
| hrw | I use 2.7.10 to deploy | 13:13 |
| mgoddard | a fine choice | 13:14 |
| mgoddard | if we bump the minimum to 2.8 you would have to change, which is not nice for you | 13:14 |
| hrw | it's from pip anyway | 13:14 |
| hrw | but if we go for 'really need to be 2.8+' then imho it should go into requirements.txt | 13:15 |
| mgoddard | right | 13:15 |
| hrw | ok, deploy started with ansible 2.8.3 | 13:16 |
| mgoddard | I'm not aware of many bugs due to ansible versions | 13:16 |
| mgoddard | that's my opinion anyway | 13:17 |
| mnasiadka | There were some nasty before 2.3, but none really since that | 13:18 |
| mnasiadka | We should just support what is not eol | 13:18 |
| *** priteau has quit IRC | 13:18 | |
| hrw | 2.8 adds some warnings | 13:19 |
| mnasiadka | I’ll create a change to get rid of them | 13:19 |
| mnasiadka | :) | 13:19 |
| mnasiadka | Mainly the one with hostgroup naming | 13:19 |
| *** priteau has joined #openstack-kolla | 13:19 | |
| mnasiadka | So it doesn’t bite us in 2.10 | 13:19 |
| hrw | mnasiadka: "when: docker_storage_driver" as well. and complaints that we just use /usr/bin/python thing | 13:20 |
| *** rabel has joined #openstack-kolla | 13:21 | |
| mgoddard | the hostgroup change is insane | 13:23 |
| rabel | hi. when trying to deploy openstack with kolla-ansible on centos7 following the quickstart guide, i get an error in task "placement : Running placement bootstrap container". it says "manifest for kolla/centos-binary-placement-api:rocky not found: manifest unknown: manifest unknown". is this a known issue? | 13:23 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: Set 'distro_python_version' variable https://review.opendev.org/675581 | 13:23 |
| hrw | https://review.opendev.org/#/c/672700/ anyone? | 13:24 |
| hrw | rabel: placement started with queens | 13:24 |
| hrw | rabel: go stein btw. | 13:24 |
| hrw | meh. ignore first comment | 13:24 |
| *** mchlumsky has joined #openstack-kolla | 13:25 | |
| rabel | hrw, thx, ill try stein | 13:25 |
| hrw | https://hub.docker.com/r/kolla/centos-binary-placement-api/tags shows stein and master only | 13:25 |
| yoctozepto | rabel: make sure you don't mix kolla-ansible stein with rocky images | 13:28 |
| yoctozepto | as in configured for rocky | 13:28 |
| yoctozepto | but downloaded k-a for stein | 13:28 |
| yoctozepto | mgoddard: being realist, I think it's best to grab the latest stable so that if there is something nice to backport and it depends on a missing feature we are not forced to rewrite it | 13:29 |
| yoctozepto | and it's super easy to upgrade ansible in virtualenv which we should advocate anyway | 13:29 |
| mgoddard | yoctozepto: that will always be a problem if we keep bumping the minimum | 13:30 |
| mgoddard | keeping it more open will make it more likely to backport | 13:30 |
| mgoddard | maybe | 13:30 |
| mgoddard | yoctozepto: good point about upgrading ansible - that should be part of the upgrade procedure | 13:31 |
| * hrw off | 13:33 | |
| henriqueof | I think koll-ansible is not deploying VPNaaS correctly, my VPN are stuck on PENDING_CREATE state and in the docks it says that the neutron_vpnaas_agent container should be runnning but i don't see it on the network node. | 13:49 |
| *** spiette has quit IRC | 13:51 | |
| *** mchlumsky has quit IRC | 13:52 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: DNM: Use python 3.6 in CentOS source images via IUS https://review.opendev.org/675367 | 13:52 |
| *** spiette has joined #openstack-kolla | 13:53 | |
| *** mchlumsky has joined #openstack-kolla | 13:53 | |
| *** KeithMnemonic has joined #openstack-kolla | 13:54 | |
| *** janki has quit IRC | 13:57 | |
| *** rgogunskiy has quit IRC | 13:57 | |
| *** BjoernT has joined #openstack-kolla | 13:59 | |
| *** atmark has joined #openstack-kolla | 14:00 | |
| yoctozepto | henriqueof: afair vpnaas extends l3-agent | 14:02 |
| *** BjoernT_ has joined #openstack-kolla | 14:04 | |
| *** BjoernT has quit IRC | 14:05 | |
| *** luksky has joined #openstack-kolla | 14:06 | |
| openstackgerrit | Merged x/kayobe master: Move to openstackdocstheme for docs & release notes https://review.opendev.org/674468 | 14:11 |
| *** dpawlik has quit IRC | 14:17 | |
| henriqueof | yoctozepto: Thanks, will check l3 agent logs. | 14:18 |
| mnasiadka | ok, only crappy extend_start needs to be fixed | 14:18 |
| mnasiadka | mgoddard: are you fine with moving stuff like this to k-a? https://opendev.org/openstack/kolla/src/branch/master/docker/fluentd/extend_start.sh#L26-L28 | 14:19 |
| mnasiadka | mgoddard: treasuredata rpms don't have this directory as part of the collection and it just fails :-) | 14:19 |
| yoctozepto | mnasiadka: imho it's to be dropped anyway | 14:20 |
| mgoddard | mnasiadka: or mkdir -p ? | 14:20 |
| yoctozepto | no need to move | 14:20 |
| yoctozepto | permissions are not going anywhere :D | 14:20 |
| mnasiadka | mgoddard: yoctozepto complained about mkdir -p :D | 14:20 |
| mgoddard | lol | 14:20 |
| yoctozepto | mgoddard: why create unused directories | 14:20 |
| yoctozepto | hides bugs behind them | 14:20 |
| mgoddard | if its unused why do we need to chown it? | 14:21 |
| yoctozepto | that's my point | 14:21 |
| mnasiadka | well, are we sure it's not used when not using treasuredata packages? | 14:21 |
| mgoddard | so rather than move to k-a you would remove entirely? | 14:21 |
| yoctozepto | I would | 14:21 |
| yoctozepto | mnasiadka: packages should install proper perms anyway, we don't meddle with it | 14:22 |
| mgoddard | pretty sure I remember seeing /var/lib/fluentd mentioned in a bug | 14:22 |
| yoctozepto | yes, it was fixed | 14:22 |
| yoctozepto | fluentd has some tech debt in it | 14:22 |
| yoctozepto | td-agent migration is a nice time to repay it ;p | 14:22 |
| mnasiadka | yoctozepto: on arm there is no td-agent migration :) | 14:23 |
| openstackgerrit | Gaëtan Trellu proposed openstack/kolla-ansible master: Add Masakari Ansible role https://review.opendev.org/615715 | 14:23 |
| mnasiadka | yoctozepto: although I'm making stuff better for CentOS/x86_64, on arm it's still old fluentd package from the museum | 14:25 |
| yoctozepto | mnasiadka: fully aware | 14:25 |
| mnasiadka | there is this (of course raised by hrw) - https://github.com/treasure-data/omnibus-td-agent/issues/131 | 14:25 |
| yoctozepto | but I mean there is a person (you) who is to repay | 14:26 |
| yoctozepto | geez, they are bit late ;p | 14:26 |
| mnasiadka | a bit? | 14:27 |
| mnasiadka | best thing is, treasure data is owned by Arm :D | 14:27 |
| openstackgerrit | Isaac Prior proposed openstack/kolla-ansible master: Fixes Monasca log transformer UTC offset exception https://review.opendev.org/675660 | 14:28 |
| henriqueof | yoctozepto: I enabled debug on l3 agent and when a new vpn is created it does not trigger any action, maybe it is a neutron related problem. | 14:29 |
| mnasiadka | mgoddard: so, the perms fix is from this change: https://opendev.org/openstack/kolla/commit/00032822535e0aca208263aa2bd84be83132d60a | 14:29 |
| mnasiadka | mgoddard: I guess the author meant to fix perms during upgrade... | 14:30 |
| mgoddard | that makes sense | 14:31 |
| yoctozepto | but this is no longer the case though | 14:31 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Change fluentd to td-agent on CentOS https://review.opendev.org/675614 | 14:31 |
| yoctozepto | and see | 14:32 |
| mgoddard | does td-agent on ubuntu use /var/lib/td-agent? | 14:32 |
| yoctozepto | there is the fluentd-user | 14:32 |
| yoctozepto | we are not using it anymore afaik | 14:32 |
| mnasiadka | well, I don't have any running setup on Ubuntu to check, if there is anything in this directory | 14:32 |
| mnasiadka | but on a freshly built image it's empty | 14:32 |
| yoctozepto | mgoddard: I think the bug was about exactly that | 14:32 |
| mgoddard | often /var/lib/ is used at runtime | 14:33 |
| yoctozepto | there is some /var/run mention too | 14:33 |
| yoctozepto | it just needs cleaning :-) | 14:33 |
| mnasiadka | I prefer going the safe way :) | 14:34 |
| mnasiadka | fluentd gives us enough problems anyway :) | 14:34 |
| Blinkiz | According to kolla-ansible documentation, ceph should be deployed external before running kolla-ansible. Okay. | 14:36 |
| Blinkiz | In kolla-ansible documentation there is text about ceph.client.nova.keyring. Problem is that this file is not created by default by ceph-ansible. And it does not exist on https://docs.ceph.com/docs/master/rbd/rbd-openstack/ | 14:37 |
| Blinkiz | After reading text, am guessing nova keyring is cinder keyring? | 14:37 |
| Blinkiz | https://docs.openstack.org/kolla-ansible/stein/reference/storage/external-ceph-guide.html have the text about transferring ceph.client.nova.keyring into /etc/kolla/config/nova | 14:39 |
| Blinkiz | If I use ceph-ansible with the flag openstack_config: True, there is no nova client when running "ceph auth ls" | 14:39 |
| kplant | you're really asking a ceph-ansible question here | 14:40 |
| kplant | but! you justneed to add it to openstack_keys: | 14:40 |
| kplant | somethng like { name: client.nova, caps: { mon: "profile rbd", osd: "profile rbd pool={{ openstack_nova_pool.name }}"}, mode: "0600" } | 14:41 |
| Blinkiz | kplant, okay. But why? Am asking from a user friendly perspective. If ceph-ansible is not creating nova keyring, do openstack really need it? ceph-ansible has openstack_keys and am guessing they know what to create? | 14:42 |
| kplant | it's better to be specific than just have one keyring that has access to everything | 14:44 |
| Blinkiz | Again, I do think that ceph.client.nova.keyring is not needed. Insted ceph.client.cinder.keyring should be used and rbd_user should be cinder. | 14:44 |
| kplant | no, cinder is not nova | 14:45 |
| kplant | there are nova volumes and there are cinder volumes - they're different | 14:45 |
| Blinkiz | okay, thanks | 14:45 |
| yoctozepto | mnasiadka: /var/lib is supposed to hold plugins | 14:52 |
| yoctozepto | it's not runtime thingy | 14:52 |
| yoctozepto | let's not keep this legacy around :-) | 14:52 |
| yoctozepto | also please see sudoers file | 14:52 |
| yoctozepto | %kolla ALL=(root) NOPASSWD: /bin/chmod 2775 /var/lib/fluentd, /usr/bin/chmod 2775 /var/lib/fluentd | 14:52 |
| yoctozepto | looks unused | 14:52 |
| yoctozepto | and other lines look wrong now that fluentd/td-agent depends on multiple factors | 14:53 |
| yoctozepto | clean, clean, cleany clean | 14:54 |
| hrw | mnasiadka: thanks for reminding. pinged issue | 14:55 |
| mnasiadka | yoctozepto: so, if somebody comes with a bug - I assign it to you? :) | 14:56 |
| yoctozepto | mnasiadka: fair enough, we can share roles, I like infra stuff, let fluentd and friends be my problem, this could be part of this support status matrix we are making | 14:57 |
| yoctozepto | the areas of expertise or something | 14:57 |
| yoctozepto | primary SPOC, secondary SPOC | 14:58 |
| mnasiadka | I still have a problem understanding why fluentd is in common role :) | 14:58 |
| yoctozepto | it's so common :D | 14:58 |
| yoctozepto | the dep on fluentd is strong because haproxy cannot log into file (lol!) and needs a lightweight syslog server to intercept and store logs | 14:59 |
| yoctozepto | patch haproxy and we move fluentd out of common | 14:59 |
| *** ktibi has joined #openstack-kolla | 14:59 | |
| yoctozepto | guys, please review an oldie of mine before I forget it: https://review.opendev.org/668445 | 15:06 |
| yoctozepto | egonzalez made some comments but they seemed to be rushed and he never replied back :-( | 15:06 |
| yoctozepto | this is related to hrw's work but predates it, I needed it to debug behavior with upgrade | 15:09 |
| yoctozepto | and added as a goodie | 15:09 |
| kplant | +1 | 15:12 |
| yoctozepto | thx kplant | 15:14 |
| *** ktibi_ has joined #openstack-kolla | 15:19 | |
| *** dave-mccowan has joined #openstack-kolla | 15:21 | |
| *** ktibi__ has joined #openstack-kolla | 15:22 | |
| *** ktibi has quit IRC | 15:22 | |
| *** ktibi_ has quit IRC | 15:26 | |
| *** luksky has quit IRC | 15:26 | |
| *** priteau has quit IRC | 15:45 | |
| *** factor has joined #openstack-kolla | 15:48 | |
| openstackgerrit | Merged x/kayobe master: Publish docs and release notes on openstack.o.o https://review.opendev.org/669342 | 15:52 |
| *** dave-mccowan has quit IRC | 15:56 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Change fluentd to td-agent on CentOS https://review.opendev.org/675614 | 15:56 |
| hrw | mnasiadka: nice. commented | 16:03 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Change fluentd to td-agent on CentOS https://review.opendev.org/675614 | 16:04 |
| mgoddard | hurrah! https://docs.openstack.org/kayobe/latest/ | 16:06 |
| hrw | fuck. | 16:06 |
| hrw | 16 vcpu did not helped | 16:06 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: DNM: Test ironic https://review.opendev.org/675893 | 16:08 |
| hrw | ~curse aarch64 | 16:08 |
| *** ktibi__ has quit IRC | 16:11 | |
| *** rpittau is now known as rpittau|afk | 16:15 | |
| yoctozepto | mgoddard: <fanfares> | 16:16 |
| openstackgerrit | Isaac Prior proposed openstack/kolla-ansible master: Fixes Monasca log transformer UTC offset exception https://review.opendev.org/675660 | 16:18 |
| yoctozepto | mnasiadka: posted comment with reminder | 16:22 |
| yoctozepto | (kind one for now, will see later ;-) ) | 16:22 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Change fluentd to td-agent on CentOS https://review.opendev.org/675614 | 16:44 |
| mnasiadka | yoctozepto: happy now? ^^ | 16:44 |
| *** gfidente has quit IRC | 16:50 | |
| goldyfruit_ | mgoddard, thx for the review | 17:02 |
| *** cah_link has quit IRC | 17:02 | |
| *** absubram has joined #openstack-kolla | 17:11 | |
| yoctozepto | mnasiadka: always unsatisfied | 17:11 |
| yoctozepto | but thx for trying! | 17:12 |
| yoctozepto | (and seriously for your work on this thing) | 17:12 |
| mnasiadka | yoctozepto: now I’m just satisfying picky needs of reviewers :) | 17:13 |
| yoctozepto | mnasiadka: and later? | 17:14 |
| mgoddard | goldyfruit_: np | 17:15 |
| openstackgerrit | Mark Goddard proposed x/kayobe master: Sort overcloud inventory hosts and groups https://review.opendev.org/675994 | 17:15 |
| yoctozepto | mnasiadka: re: picky - I just know what Zuul will not like | 17:16 |
| yoctozepto | (and users probably ;p ) | 17:16 |
| * hrw off | 17:17 | |
| openstackgerrit | Merged openstack/kolla-ansible master: CI: Collect docker and systemd configs https://review.opendev.org/668445 | 17:19 |
| openstackgerrit | Merged openstack/kolla-ansible stable/stein: doc: Start using openstackdoctheme's extlink extension https://review.opendev.org/672309 | 17:19 |
| openstackgerrit | Merged openstack/kolla-ansible stable/stein: Fix the incorrect backup_driver configuration https://review.opendev.org/670143 | 17:19 |
| *** BjoernT_ is now known as BjoernT | 17:23 | |
| *** ivve has quit IRC | 17:27 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Workaround ironic inspector startup issue https://review.opendev.org/675893 | 17:37 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Check containers during kolla-ansible check https://review.opendev.org/599735 | 17:38 |
| *** whoami-rajat has quit IRC | 17:38 | |
| *** chrizl has quit IRC | 17:44 | |
| kplant | so if we have something sitting at -1 like https://review.opendev.org/#/c/632009/ and i wanted to crack at it, should i make a new merge request? i feel like there's a lot i would change about it | 17:45 |
| *** absubram has quit IRC | 17:47 | |
| *** rouk has joined #openstack-kolla | 17:53 | |
| *** atmark has quit IRC | 17:59 | |
| *** atmark has joined #openstack-kolla | 18:00 | |
| mgoddard | kplant: up to you, but I would get the authors permission to pick it up if you use the same change ID | 18:10 |
| mgoddard | kplant: seems like they have stalled though | 18:10 |
| *** luksky has joined #openstack-kolla | 18:12 | |
| kplant | i was thinking of changing IDs, instead of creating a new container and modifying the existing keystone containers in one i'd probably split it into two | 18:12 |
| kplant | though i'm not sure if i'm familiar enough with the codebase to start making changes like that yet :] | 18:14 |
| henriqueof | Has someone found problems with rabbitmq high cpu usage on Stein? My cloud is completely unstable. | 18:17 |
| kplant | i do not have that problem | 18:17 |
| kplant | how many nodes do you have? | 18:20 |
| henriqueof | It started when I tried to upload an volume as image, glance took all the controller resources and the control went down, when it finished all the APIs were incredibly slow, now rabbitmq is using 80~90% of CPU all the time. | 18:21 |
| henriqueof | kplant: I have only one controller and one compute node, it is a Stein test lab. | 18:21 |
| kplant | i have a few production stein clusters and numerous kolla aios, all on stein | 18:23 |
| kplant | i've never run into that | 18:23 |
| henriqueof | https://imgur.com/a/CRPmlFV | 18:24 |
| henriqueof | I am afraid I need to reboot the controller. | 18:24 |
| *** dpawlik has joined #openstack-kolla | 18:36 | |
| stackedsax | heya mgoddard: wanted to catch back up on what we've been up to re: ssl on internal network | 18:54 |
| stackedsax | let me know if you want to chat sometime | 18:54 |
| *** ivve has joined #openstack-kolla | 18:58 | |
| stackedsax | essentially, following Adam Harwell's suggestion, we're following the path of https://review.opendev.org/#/c/663555/, but more fleshed out | 19:10 |
| stackedsax | kklimonda: I think you were working on this stuff once upon a time | 19:11 |
| stackedsax | would love to catch up and get your perspective | 19:11 |
| *** generalfuzz has joined #openstack-kolla | 19:13 | |
| stackedsax | generalfuzz has done all the work on it so far | 19:14 |
| *** factor has quit IRC | 19:17 | |
| *** igordc has joined #openstack-kolla | 19:26 | |
| openstackgerrit | Merged openstack/kolla master: Support customising MariaDB log directory https://review.opendev.org/675386 | 19:38 |
| *** igordc has quit IRC | 19:41 | |
| *** kplant has quit IRC | 19:44 | |
| *** dave-mccowan has joined #openstack-kolla | 19:54 | |
| *** igordc has joined #openstack-kolla | 20:11 | |
| brtknr | how do i bump up the version of magnum that kolla image is using? | 20:26 |
| brtknr | it seems out of sync with stable/rocky branch of magnum | 20:27 |
| *** dpawlik has quit IRC | 20:42 | |
| *** pcaruana has quit IRC | 20:56 | |
| brtknr | how do i get rdo people to move the pointer forward for the yum package? | 20:57 |
| *** kplant has joined #openstack-kolla | 21:00 | |
| *** absubram has joined #openstack-kolla | 21:02 | |
| *** Reepicheep_ has quit IRC | 21:29 | |
| *** Reepicheep has joined #openstack-kolla | 21:29 | |
| *** henriqueof has quit IRC | 22:04 | |
| *** BjoernT has quit IRC | 22:07 | |
| *** vishwanathj has quit IRC | 22:10 | |
| *** mchlumsky has quit IRC | 22:14 | |
| *** ivve has quit IRC | 22:33 | |
| *** vishwanathj has joined #openstack-kolla | 22:58 | |
| *** vishwanathj has quit IRC | 23:03 | |
| *** dave-mccowan has quit IRC | 23:17 | |
| *** luksky has quit IRC | 23:31 | |
| *** dave-mccowan has joined #openstack-kolla | 23:37 | |
| *** absubram has quit IRC | 23:43 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!