| *** dave-mccowan has joined #openstack-kolla | 00:23 | |
| *** dave-mccowan has quit IRC | 00:48 | |
| *** ricolin has joined #openstack-kolla | 01:16 | |
| *** zhanglong has joined #openstack-kolla | 01:18 | |
| *** jcmdln has joined #openstack-kolla | 03:02 | |
| *** Tony31 has joined #openstack-kolla | 03:36 | |
| *** goldyfruit_ has quit IRC | 03:42 | |
| *** goldyfruit_ has joined #openstack-kolla | 03:42 | |
| openstackgerrit | Chason Chan proposed openstack/kolla-ansible master: [Docs] Pin kolla-anisble to the same version of quickstart guide https://review.opendev.org/707896 | 04:19 |
|---|---|---|
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Add support for encrypting backend HAProxy traffic https://review.opendev.org/664516 | 04:33 |
| *** skramaja has joined #openstack-kolla | 04:50 | |
| generalfuzz | It would be really awesome to have a couple reviews on the current approach we are taking for enabling backend TLS through HAProxy - https://review.opendev.org/664516. | 04:52 |
| patchbot | patch 664516 - kolla-ansible - Add support for encrypting backend HAProxy traffic - 15 patch sets | 04:52 |
| generalfuzz | Now that we have a couple people starting to tackle it service by service, it would be good to know that we are taking an approach the community approves of. | 04:52 |
| *** shyamb has joined #openstack-kolla | 05:07 | |
| *** shyamb has quit IRC | 05:08 | |
| Tony31 | good morning | 05:29 |
| *** evrardjp has quit IRC | 05:34 | |
| *** evrardjp has joined #openstack-kolla | 05:34 | |
| cosmicsound | yoctozepto , i am possitive | 05:46 |
| Tony31 | hi cosmicsound | 05:57 |
| Tony31 | I was thinking about your tls problem - I was wondering if you have the cert applied on the "internal" ansible variable and it enabled on external or something like that. | 05:58 |
| *** oyrogerg has joined #openstack-kolla | 06:06 | |
| cosmicsound | Hi Tony31 at first I used self generated SSL yet I had not FQDN set on globals.yml | 06:09 |
| cosmicsound | After I have explicitly setup TLS and FQDN for cloud.uhlhost.net domain and I added the right .PEM cert chain | 06:10 |
| cosmicsound | Internal VIP ip does not have FQDN in place | 06:10 |
| cosmicsound | https://mdb.uhlhost.net/uploads/e15c6e181f5415ce/cloud_uhlhost_net.pem this is the right SSL I put in place. I did not get any errors that the SSL is wrong or the chain. And I am positive this is the wildcard TLS i have in globals.yml | 06:12 |
| cosmicsound | https://mdb.uhlhost.net/uploads/e5657707f57704d2/image.png no other settings I have changed beside external TLS and FQDN | 06:13 |
| cosmicsound | https://mdb.uhlhost.net/uploads/850b3a78847ee23a/image.png and this is clearly showing that the PEM is right domain | 06:17 |
| cosmicsound | https://mdb.uhlhost.net/uploads/b316327bbc027ff4/image.png this is on server side, got really happy to see the FQDN work, yet TLS fails it made me wonder why I have used kolla certificates to generate these self made certs | 06:19 |
| Tony31 | I also have a wildcard SSL cert | 06:26 |
| Tony31 | brb | 06:26 |
| sorin-mihai | couple of days ago i tried to use the certs i have generated with certbot, but didn't manage to get it working | 06:27 |
| Tony31 | Hi Sorin-mihai - I think I have a solution for you also | 06:31 |
| Tony31 | Where did you see the error which you had seen? Was it during deploy or somewhere else? | 06:31 |
| Tony31 | sorin-mihai are you using kayobe or kolla-ansible ? | 06:31 |
| Tony31 | cosmicsound - I am using kayobe and I tried the "node_config" variable and I got an error (probably because I'm using kayobe, not kolla...). So I have `kolla_external_fqdn_cacert: "/etc/kolla/config/certificates/cacert-internal.pem"` and all that is in there is the CA bundle | 06:33 |
| yoctozepto | morning | 06:34 |
| Tony31 | morning ! :) | 06:35 |
| Tony31 | Did you have a good weekend yoctozepto | 06:35 |
| yoctozepto | Tony31: so-so but thank you for asking, and you? | 06:35 |
| Tony31 | Was great thank you :) | 06:36 |
| yoctozepto | :-) | 06:36 |
| Tony31 | Could you help us with certificates? Bit confused on it :) With Kayobe, I have the server cert and private key located directly into kolla.yml into `kolla_external_tls_cert:` and the CA cert goes into `kolla_external_fqdn_cacert: "/etc/kolla/config/certificates/cacert-internal.pem"` which I see has configured the kolla `globals.yml` variable | 06:39 |
| Tony31 | `kolla_external_fqdn_cacert:` But when checking the kolla docs for the same, I see `kolla_external_fqdn_cacert:` is explained as containing the server cert and I am wondering if this could be the cause of the problems for cosmicsound and sorin-mihai ? | 06:39 |
| *** sri_ has joined #openstack-kolla | 06:40 | |
| Tony31 | the server cert and private key goes into `/etc/kolla/certificates/haproxy.crt` | 06:45 |
| Tony31 | cosmicsound - for your external cert problem, check your server cert and private key (server cert pem first, then paste the private key underneath) within ` | 06:49 |
| *** riuzen has joined #openstack-kolla | 06:50 | |
| Tony31 | I would also like some help installing zun/kuryr. I get error `pull access denied for kolla/centos-binary-kuryr-libnetwork` so I tried setting `kuryr_install_type: "source"` but I still get the error when i try to deploy | 06:51 |
| *** rgogunskiy has joined #openstack-kolla | 06:53 | |
| cosmicsound | well i was thinking there is no mention of any private key inside globals.yml | 06:56 |
| cosmicsound | i did not see any variable to do with private key | 06:57 |
| cosmicsound | guess my private key for the wildcard is missing | 06:57 |
| Tony31 | cosmicsound no you are right about that, but there is text that says the server cert must contain the private key - it's a bit different but it works. I guess this is your problem. | 06:57 |
| cosmicsound | can you send me that link? | 06:58 |
| cosmicsound | i did not seen yet such details | 06:58 |
| Tony31 | https://docs.openstack.org/kolla-ansible/train/admin/advanced-configuration.html#tls-configuration | 06:58 |
| cosmicsound | and its maybe even a dsecurity breach | 06:58 |
| cosmicsound | in my knowledge priv keys are private for a reason | 06:58 |
| Tony31 | na - no breach unless the private key is used or given out | 06:59 |
| cosmicsound | adding them in the .pem chain will loose the prv key | 06:59 |
| cosmicsound | if an aatacker can get access to it | 06:59 |
| Tony31 | from what I can see, we're just presenting the info to kolla so that kolla can manage it | 06:59 |
| cosmicsound | ok | 06:59 |
| Tony31 | what kolla will need to do on the backend is install these certs in the right place to set up the chain. Kolla is just abstracting this config | 06:59 |
| *** shyamb has joined #openstack-kolla | 07:01 | |
| Tony31 | I understand your feeling though - like I said, a bit different to how I was expecting to approach this. Usually, the server cert and root / CA cert goes into one "bundle" and the private key is stored somewhere else. | 07:01 |
| cosmicsound | These two files are the server certificate with private key and the CA certificate with any intermediate certificates. | 07:01 |
| Tony31 | yes thats as I understood it. | 07:01 |
| cosmicsound | this is only line that mentions a private key | 07:01 |
| Tony31 | the CA cert file I have is specified with `kolla_external_fqdn_cacert:` | 07:02 |
| Tony31 | this line: `These two files are the server certificate with private key and the CA certificate with any intermediate certificates.` | 07:02 |
| cosmicsound | i added the ca bundle and made a right chsain | 07:02 |
| cosmicsound | that tls docs are confusing | 07:02 |
| cosmicsound | ... | 07:02 |
| cosmicsound | in end are 3 files | 07:02 |
| cosmicsound | and its totally wrong | 07:02 |
| Tony31 | no - 2 files. | 07:03 |
| Tony31 | Which 3 do you have? | 07:03 |
| cosmicsound | the private key does not come from a provider | 07:03 |
| cosmicsound | read a | 07:03 |
| cosmicsound | again | 07:03 |
| cosmicsound | 1 crt 1 ca bundle 1 private key | 07:03 |
| cosmicsound | get 3 files | 07:03 |
| Tony31 | I completely understand where you are coming from, but in this kolla context it does not really matter | 07:03 |
| Tony31 | OK do this: | 07:03 |
| cosmicsound | all i said i added my right ssl chain of trust | 07:04 |
| Tony31 | make 1 file in notepad++ called `server-private.crt` and paste in your server PEM. Then immediately underneath, paste in the private key | 07:04 |
| cosmicsound | and the pem is ignored | 07:04 |
| cosmicsound | no where i could deduct that the private key needs to be pasted inside the cert | 07:04 |
| Tony31 | Then make another file and call it CA bundle and paste in your CA bundle certs (i am not sure which way around they are meant to go. I usually do intermediate then root | 07:04 |
| cosmicsound | or i might not be ablee to read that english | 07:04 |
| Tony31 | you're not really making a cert for kolla, here. You're just placing the PEM text in the right location for kolla to pull it out and do the necessary things | 07:05 |
| *** jbadiapa has joined #openstack-kolla | 07:05 | |
| Tony31 | well, this is as I understand it and I am happy to be corrected | 07:05 |
| cosmicsound | i get this when ordering an ssl from sectigo/comodo | 07:06 |
| cosmicsound | https://mdb.uhlhost.net/uploads/467a8d4857286f82/image.png | 07:06 |
| cosmicsound | nothing more nothing less | 07:06 |
| Tony31 | thats fine | 07:06 |
| cosmicsound | i had added my ca-bundle into my crt | 07:06 |
| Tony31 | we can make this work with that | 07:06 |
| Tony31 | you just need your private key to hand | 07:06 |
| cosmicsound | to have the right chain of trust | 07:06 |
| cosmicsound | ok i ca. get the prv key | 07:07 |
| Tony31 | pull your CA bundle out of your cert because we just need within 1 file: `server cert / private key` | 07:07 |
| Tony31 | :) | 07:07 |
| *** thanhba has joined #openstack-kolla | 07:10 | |
| Tony31 | cosmicsound - I see that this is incorrect documentation on the link which I had sent to you: `The server certificate needs to be installed with the kolla deployment and is configured with the kolla_external_fqdn_cert or kolla_internal_fqdn_cert parameter.` | 07:11 |
| Tony31 | The variable described is for the CA bundle. Not server cert. | 07:11 |
| Tony31 | no wait, ignore that. I was mistaking these: `kolla_external_fqdn_cacert:` and `kolla_external_fqdn_cert:` | 07:12 |
| thanhba | I have problem when use kolla to deploy openstack rocky. | 07:14 |
| *** cah_link has joined #openstack-kolla | 07:15 | |
| thanhba | when running nova instance for a while. it lost directory '/sys/fs/cgroup/devices/machine.slice/' then i can't add volume to this instance. this is log http://paste.openstack.org/show/789628/ | 07:17 |
| thanhba | I have problem when use kolla to deploy openstack rocky. when running nova instance for a while. it lost directory '/sys/fs/cgroup/devices/machine.slice/' then i can't add volume to this instance. this is log http://paste.openstack.org/show/789628/ | 07:17 |
| thanhba | it is a bug? and how can i solve this problem? | 07:18 |
| Tony31 | thanhba - which `kolla` are you using and which version? | 07:21 |
| thanhba | i use kolla 7.x | 07:22 |
| Tony31 | I am not sure, do you need to use the matching openstack <-> kolla version ? | 07:22 |
| thanhba | http://paste.openstack.org/show/789629/ | 07:23 |
| thanhba | yes, i know that. This is how i install kolla. I install kolla with repo git. | 07:24 |
| Tony31 | ok - looks ok | 07:25 |
| *** shyamb has quit IRC | 07:31 | |
| *** Jeffrey4l has quit IRC | 07:41 | |
| *** Jeffrey4l has joined #openstack-kolla | 07:41 | |
| *** hrw has quit IRC | 07:47 | |
| *** hu_berlin_kalle has quit IRC | 07:48 | |
| Tony31 | is it possible to put compute nodes on different layer 3 networks when using kolla or kayobe ? | 07:57 |
| *** zhanglong has quit IRC | 08:01 | |
| *** zhanglong has joined #openstack-kolla | 08:02 | |
| yoctozepto | Tony31: zun&kuryr are source-only, yup | 08:04 |
| yoctozepto | Tony31: pure computes should be able to be on different subnets | 08:04 |
| *** bengates has joined #openstack-kolla | 08:04 | |
| yoctozepto | Tony31: only controllers/network nodes depend on having single subnet afair | 08:05 |
| yoctozepto | mostly due to keepalived | 08:05 |
| Tony31 | yoctozepto thank you - Do I need to configure routes for each interface? For example, the internal API interface, it will have a different subnet, so do I need to route out of this interface for the internal API? Does it make sense what I am trying to explain? | 08:06 |
| Tony31 | or, should I just keep it simple and have a single interface for everything | 08:06 |
| Tony31 | (on the remote compute node) | 08:06 |
| *** rpittau|afk is now known as rpittau | 08:10 | |
| *** gbatir_ has joined #openstack-kolla | 08:11 | |
| *** thanhba has quit IRC | 08:12 | |
| mnasiadka | morning | 08:12 |
| Tony31 | morning :D | 08:12 |
| yoctozepto | Tony31: uhm, all subnets should be routable | 08:13 |
| yoctozepto | Tony31: interfaces can be different | 08:13 |
| Tony31 | they are routable but I have one default gateway | 08:13 |
| Tony31 | (per system) | 08:13 |
| yoctozepto | Tony31: this is general networking, not really kolla-specific | 08:13 |
| yoctozepto | if that one gateway handles it well | 08:14 |
| yoctozepto | then it's all right ;-) | 08:14 |
| Tony31 | I think what I need to find out is if internal API traffic needs to leave and return to the same internal API interface between compute and controller on both sides. Hence, creating "routes" for the individual interfaces. | 08:15 |
| yoctozepto | well, they need to reach the required destination | 08:23 |
| yoctozepto | which is called routability | 08:23 |
| yoctozepto | I'm answering based on my k-a knowledge, kayobe might be forcing more, better ask mnasiadka/mgoddard | 08:23 |
| *** pbing19 has joined #openstack-kolla | 08:24 | |
| Tony31 | no - I'm fine with networks and routing :) My question is specifically about kolla/kolla-ansible/kayobe and how the separate interfaces are working when we have separate interfaces for the services such as `internal API` | 08:24 |
| Tony31 | Talking about the api interface for example and communication between compute node and controller node - they have API interfaces in different subnets. | 08:25 |
| Tony31 | So would I need host routes for both controller and compute node, which tells each host to use the API interface to reach the remote systems api interface in this respect? | 08:26 |
| mnasiadka | well, if you won't set the static routing to go through another gateway on the dedicated interface - it will go through the interface that reaches default gateway, right? | 08:27 |
| Tony31 | that's my point | 08:27 |
| mnasiadka | it all depends what you want to achieve, how your networks are configured, do you have security zones and security restrictions and so on | 08:27 |
| Tony31 | is it still a working scenario in that case? | 08:27 |
| *** tonythomas has joined #openstack-kolla | 08:28 | |
| Tony31 | looking at the controller node which is deployed with kayobe, I dont see any firewall configured there. | 08:28 |
| yoctozepto | I believe mnasiadka had in mind any external constraints | 08:29 |
| mnasiadka | Tony31: kayobe does not configure firewall | 08:29 |
| yoctozepto | k-a has nothing by design to prevent separating computes to different subnets | 08:29 |
| Tony31 | OK this is good - this is what I am looking for. Thank you | 08:29 |
| yoctozepto | and together we clear all doubts | 08:29 |
| Tony31 | So in this case, using separate subnets for example internal API, do I MUST configure routes ? or is it fine to allow sourcing of packets from another interface? For example, default gateway is on eth0 while internal API is on eth2. Since API of controller is on a separate layer 3 routed network, this system would have to send via it's gateway on | 08:31 |
| Tony31 | eth0 with a source address of the packet which is eth2. | 08:31 |
| Tony31 | I guess that would be poor design but would it work is the question :) | 08:32 |
| mnasiadka | yoctozepto: the single node wsrep errors are by date/time correlated with haproxy killing the connection - but question why neutron stalls for such a long time :) | 08:32 |
| yoctozepto | mnasiadka: yeah, it's a question to your friend | 08:32 |
| yoctozepto | mnasiadka: this looks like an internal process | 08:32 |
| yoctozepto | mnasiadka: despite having req number | 08:32 |
| mnasiadka | well, it's an upgrade job, so I guess mariadb comes down at some point, and this crappy neutron is not able to cope with that I guess - and get's stuck in some weird state | 08:33 |
| yoctozepto | Tony31: this really is not a kolla question, but networking in general / linux networking | 08:33 |
| yoctozepto | Tony31: if A can reach B on its designated IP address | 08:34 |
| yoctozepto | Tony31 and vice versa | 08:34 |
| yoctozepto | Tony31: then it is fine | 08:34 |
| yoctozepto | ;-) | 08:34 |
| yoctozepto | mnasiadka: yeah, that's what happens there | 08:34 |
| yoctozepto | mnasiadka: though not very precise | 08:34 |
| Tony31 | yoctozepto fantastic thank you for your help on this one to help me understand. | 08:35 |
| yoctozepto | yw, | 08:36 |
| mnasiadka | yoctozepto: well, it's interesting that the message "WSREP has not yet prepared node..." is an internal error, but lost connection to mysql during query is a connection error handled at oslo db layer | 08:36 |
| mnasiadka | yoctozepto: maybe that's the key | 08:36 |
| yoctozepto | mnasiadka: could be | 08:36 |
| mnasiadka | because from my perspective it should be handled in the same way | 08:36 |
| mnasiadka | let me raise a bug to Oslo, let's see :) | 08:37 |
| yoctozepto | mnasiadka: there was one already | 08:38 |
| yoctozepto | mnasiadka: from me | 08:38 |
| mnasiadka | yoctozepto: got a link? | 08:38 |
| yoctozepto | mnasiadka: looking now | 08:39 |
| yoctozepto | mnasiadka: not found it in lp | 08:40 |
| yoctozepto | mnasiadka: was it sb? | 08:40 |
| mnasiadka | yoctozepto: well, oslo wiki says oslo.db bugs are managed in launchpad | 08:41 |
| yoctozepto | mnasiadka: then can't find | 08:41 |
| yoctozepto | maybe it was not mine after all hmm | 08:41 |
| *** hu_berlin_kalle has joined #openstack-kolla | 08:41 | |
| mgoddard | morning all | 08:42 |
| mgoddard | I'm out today and tomorrow | 08:42 |
| mgoddard | please address all requests to mnasiadka :p | 08:42 |
| yoctozepto | mgoddard: will do | 08:42 |
| yoctozepto | mgoddard: have fun | 08:42 |
| yoctozepto | mnasiadka: ah, it was keystone then | 08:42 |
| yoctozepto | mnasiadka: and keystoneauth | 08:42 |
| yoctozepto | mnasiadka: so no, wsrep did not get being reported | 08:43 |
| mnasiadka | yoctozepto: I think we need 1047 or regexp in InternalError filters - https://github.com/openstack/oslo.db/blob/fe74320e8b2c0656ca4f1090a70c28d8294142ac/oslo_db/sqlalchemy/exc_filters.py#L391 | 08:43 |
| yoctozepto | mnasiadka: I guess we understand this all better now | 08:43 |
| yoctozepto | mnasiadka: most likely | 08:44 |
| yoctozepto | mnasiadka: oslo folks probably know it better | 08:45 |
| yoctozepto | mnasiadka: this is clearly a "retry" kind of message | 08:45 |
| yoctozepto | mnasiadka: and not fail critically | 08:45 |
| yoctozepto | mnasiadka: btw, did you get manila to work? you seem pretty content closing ceph bugs | 08:47 |
| *** shyamb has joined #openstack-kolla | 08:47 | |
| mnasiadka | I did get manila to work, I'll add manila to ceph-ansible CI - I'm only closing those that related to kolla-ceph - unless you want to invest cycles in fixing kolla-ceph in Train and earlier :) | 08:48 |
| *** rgogunskiy_ has joined #openstack-kolla | 08:48 | |
| mnasiadka | yoctozepto: this one is still a bit weird: https://zuul.opendev.org/t/openstack/build/bd8c3d336030431686071cb26f2793d2/log/primary/logs/kolla/neutron/neutron-server.txt#4361 | 08:50 |
| mnasiadka | but after this - we restart neutron-server and it gladly connects | 08:50 |
| yoctozepto | mnasiadka: what's weird about it though? | 08:51 |
| yoctozepto | mnasiadka: https://zuul.opendev.org/t/openstack/build/bd8c3d336030431686071cb26f2793d2/log/primary/logs/kolla/neutron/neutron-server.txt#260 it started early | 08:52 |
| Tony31 | hi mgoddard hope you had a good weekend | 08:52 |
| mgoddard | hi Tony31 thanks, and you | 08:52 |
| *** rgogunskiy has quit IRC | 08:52 | |
| mgoddard | how's the deployment going? | 08:52 |
| Tony31 | great thank you :) - deployment is good, unable to install swift at the moment. Michael logged that one. So I'm going to look at kubernetes (magnum) and try zun/kuryr. I have to set some variables to allow it to install from source. I think I missed one as I only listed kuryr. But otherwise it's great and I think what you guys have done here with | 08:54 |
| Tony31 | this is fantastic - more people should get onboard with this | 08:54 |
| yoctozepto | Tony31: I think I logged that | 08:59 |
| yoctozepto | Tony31: thanks for the good word | 08:59 |
| *** k_mouza has joined #openstack-kolla | 09:04 | |
| Tony31 | my bad. it was yoctozepto who was helpful over the weekend | 09:06 |
| *** ivve has joined #openstack-kolla | 09:07 | |
| *** k_mouza has quit IRC | 09:08 | |
| mnasiadka | well, 8 minutes after mariadb start - in theory we still get an error about node not ready for application use - but neutron-server restart does the job - funny, right? | 09:09 |
| mnasiadka | yoctozepto: https://bugs.launchpad.net/neutron/+bug/1863579 | 09:12 |
| openstack | Launchpad bug 1863579 in oslo.db "Unhandled error - WSREP has not yet prepared node for application use" [Undecided,New] | 09:12 |
| mnasiadka | let's see what happens | 09:12 |
| yoctozepto | mnasiadka: https://zuul.opendev.org/t/openstack/build/bd8c3d336030431686071cb26f2793d2/log/primary/logs/kolla/neutron/neutron-server.txt#4175 | 09:13 |
| yoctozepto | mnasiadka: it was critical | 09:13 |
| yoctozepto | and later kolla saves the day | 09:13 |
| *** dougsz has joined #openstack-kolla | 09:13 | |
| yoctozepto | mnasiadka: ok, I added myself as victim, subscribed, commented and changed to "confirmed" | 09:17 |
| yoctozepto | mnasiadka: and now we wait | 09:17 |
| mnasiadka | yup | 09:17 |
| *** FlorianFa has joined #openstack-kolla | 09:18 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Change /run bind mount for neutron/openvswitch https://review.opendev.org/707375 | 09:21 |
| yoctozepto | mnasiadka: any idea I could not reproduce this stuff around dbus leakage? | 09:22 |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: WIP: CentOS 8 https://review.opendev.org/707690 | 09:22 |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: WIP: Remove activate-virtualenv and deactivate-virtualenv roles https://review.opendev.org/708072 | 09:22 |
| yoctozepto | mnasiadka: I agree to limit the scope but prefer to understand | 09:22 |
| mnasiadka | yoctozepto: have no clue, we see it on some envs with high activity of neutron-rootwrap, but on some not really | 09:23 |
| yoctozepto | mnasiadka: very odd | 09:23 |
| yoctozepto | mnasiadka: and this fixes it? | 09:24 |
| mnasiadka | yoctozepto: yeah, Stig confirmed on an environment that it fixes it for him - and we don't break anything else | 09:24 |
| mnasiadka | wonder if that's not some... centos weirdness with systemd | 09:26 |
| mnasiadka | we usually see it on envs with sriov-agent enabled | 09:26 |
| mnasiadka | maybe that's also the key :) | 09:26 |
| yoctozepto | mnasiadka: could be, I have no sriov to play with in the first place :D | 09:28 |
| yoctozepto | mnasiadka: commented | 09:33 |
| yoctozepto | mnasiadka: a separate issue - don't other services need similar measures of isolation? | 09:34 |
| mnasiadka | yoctozepto: well, I guess they do - it's ironic, cinder and sahara I think - but I focused on this being the top talker :) | 09:35 |
| yoctozepto | mnasiadka: well, now I wonder if manila's bug you closed today was not really caused by this | 09:35 |
| Tony31 | Can I re-assign these variables on a per-host basis? `admin_oc_net_name` | 09:35 |
| yoctozepto | mnasiadka: as it has /run | 09:35 |
| yoctozepto | mnasiadka: and could guess it wants to call some systemd stuff | 09:35 |
| mnasiadka | yoctozepto: manila has /run only for iscsi I think | 09:36 |
| yoctozepto | mnasiadka: thus failing miserably | 09:36 |
| yoctozepto | mnasiadka: well, it somehow figured out to look for systemd ;D | 09:36 |
| yoctozepto | mnasiadka: could be manila though | 09:36 |
| mnasiadka | yoctozepto: I guess we don't need /run/openvswitch in linuxbridge agent? :D | 09:40 |
| yoctozepto | mnasiadka: yeah, and probably none other than ovs agent | 09:41 |
| mnasiadka | god knows, I can use the CI to test :D | 09:41 |
| yoctozepto | mnasiadka: yeah, I saw mgoddard adding more networking testing to kayobe | 09:42 |
| yoctozepto | mnasiadka: could help in k-a as well | 09:42 |
| mnasiadka | dhcp_agent has interface_driver = openvswitch setting | 09:42 |
| yoctozepto | mnasiadka: then just verify those and check any logs and we are clear | 09:42 |
| mnasiadka | so I guess it could use a connection to ovs socket | 09:42 |
| yoctozepto | mnasiadka: yeah, but it was supposed to be choosing agent | 09:43 |
| yoctozepto | mnasiadka: well, best test | 09:43 |
| yoctozepto | mnasiadka: add tests and test | 09:43 |
| yoctozepto | mnasiadka: let the ussuri cycle be the neutron cycle for kolla | 09:43 |
| mnasiadka | well, it has ovsdb connection using tcp, so it shouldn't need the mount | 09:43 |
| yoctozepto | mnasiadka: well, maybe none of these require this mount and we are discussing dump :-) | 09:44 |
| mnasiadka | let's see | 09:44 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Change /run bind mount for neutron/openvswitch https://review.opendev.org/707375 | 09:49 |
| mnasiadka | so let's test | 09:49 |
| mnasiadka | first the standard ovs stuff | 09:50 |
| mnasiadka | yoctozepto: might be a linuxbridge based CI would be useful | 09:50 |
| yoctozepto | mnasiadka: might be | 09:50 |
| mnasiadka | it seems some people are using it - I don't know if in k-a, but still | 09:51 |
| yoctozepto | mnasiadka: not sure if it passes now that you made openvswitch require this path to exist earlier | 09:51 |
| *** pbing19 has quit IRC | 09:51 | |
| mnasiadka | well, openvswitch creates this path | 09:52 |
| yoctozepto | mnasiadka: but not docker | 09:52 |
| mnasiadka | if it would fail for this reason - it would fail earlier as well | 09:52 |
| yoctozepto | I expect a failure to mount | 09:52 |
| yoctozepto | mnasiadka: earlier you mounted /run | 09:53 |
| yoctozepto | which exists | 09:53 |
| mnasiadka | i patchset 1 I mounted /run? | 09:53 |
| mnasiadka | where? | 09:53 |
| yoctozepto | mnasiadka: in ovs | 09:53 |
| yoctozepto | I don't mean neutron now, but ovs | 09:53 |
| mnasiadka | Yeah, and I didn't even edit ovs role in patchset 2, it was as it was in patchset 1 | 09:53 |
| mnasiadka | :) | 09:53 |
| yoctozepto | mnasiadka: ah, sorry, right | 09:54 |
| yoctozepto | mnasiadka: then it work | 09:54 |
| *** shyamb has quit IRC | 09:54 | |
| yoctozepto | works* | 09:54 |
| *** gfidente has joined #openstack-kolla | 09:54 | |
| yoctozepto | though docker usually fails if path does not exist previously | 09:54 |
| yoctozepto | do we touch host for this? | 09:54 |
| mnasiadka | nope, only extend_start in kolla I think creates that path if it doesn't exist | 09:55 |
| *** pbing19 has joined #openstack-kolla | 09:55 | |
| yoctozepto | mnasiadka: yeah, but that runs afterwards, weird | 09:55 |
| yoctozepto | mnasiadka: maybe it has something to do with ":shared" | 09:56 |
| mnasiadka | non-existent bind mounts from the docker engine will get initialized to an empty directory owned by root | 09:56 |
| yoctozepto | mnasiadka: our our module does the creation | 09:56 |
| mnasiadka | it only fails on :ro | 09:57 |
| yoctozepto | mnasiadka: mhm, possible | 09:57 |
| yoctozepto | mnasiadka: question for 100 points - why are we requiring dbus as kolla, and how is it used in deployed projects? (other than causing them memleaks) | 10:03 |
| yoctozepto | (I know systemd depends on dbus, but we make it explicit req in debian) | 10:03 |
| mnasiadka | yoctozepto: we required dbus for ceph-nfs on Ubuntu only | 10:03 |
| mnasiadka | because it used old ganesha, which had problems when dbus was missing | 10:03 |
| mnasiadka | now we could remove it | 10:03 |
| yoctozepto | mnasiadka: ah, this very old | 10:04 |
| yoctozepto | mnasiadka: because later I removed that dep | 10:05 |
| yoctozepto | mnasiadka: rocky: https://review.opendev.org/#/c/674889/1/ansible/roles/ceph/tasks/start_nfss.yml | 10:05 |
| patchbot | patch 674889 - kolla-ansible (stable/rocky) - ceph: fixes to deployment and upgrade (MERGED) - 1 patch set | 10:05 |
| mnasiadka | hmm, we have dbus in bindep? | 10:05 |
| yoctozepto | mnasiadka: exactly - why? | 10:05 |
| yoctozepto | mnasiadka: kayobe too for that matter ;-) | 10:05 |
| yoctozepto | mnasiadka: and also in zuul playbook | 10:06 |
| mnasiadka | seems ubuntu gate has problems with missing dbus | 10:06 |
| mnasiadka | but it was 3 years ago, change made by Steven Dake :D | 10:06 |
| yoctozepto | mnasiadka: from cisco | 10:06 |
| mnasiadka | yoctozepto: father of Kolla ;) | 10:07 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: Remove dbus from bindep https://review.opendev.org/708082 | 10:07 |
| mnasiadka | let's see ^^ | 10:07 |
| *** shyamb has joined #openstack-kolla | 10:09 | |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: WIP: Remove dbus from bindep and playbooks https://review.opendev.org/708082 | 10:12 |
| yoctozepto | mnasiadka: ^ to actually have an effect | 10:12 |
| yoctozepto | though it's most likely installed anyway due to systemd nowadays | 10:13 |
| mnasiadka | we installed it twice? lol :) | 10:13 |
| yoctozepto | mnasiadka: yeah, that's what I am talking about | 10:13 |
| Tony31 | need some help :) could you point me in the right direction? I would like to add a compute node but it's networks have been redefined with `host_vars` such as `admin_oc_net_name:` the problem is they are not being taken and the host build is assigning IPs from the wrong networks. | 10:13 |
| Tony31 | Can I use host_vars for this? If yes, then what name do I need to call the files there? | 10:14 |
| Tony31 | the goal is to install a compute node that is in a different layer 3 network | 10:14 |
| mnasiadka | you can call the files as you like, just put the vars inside | 10:14 |
| Tony31 | hmm thats what I thought | 10:15 |
| ivve | hello, i have a problem with the neutron-l3-agent container (possible keepalived issue), im looking for how to enable debuglogs for keepalived as its default is syslog and there is no syslog in the container.. how can i proceed? any pointers welcome | 10:16 |
| ivve | neutron debug already enabled but i can't seem to see much other than transition to backup/master | 10:16 |
| ivve | which i can see even if debug is not enabled | 10:17 |
| *** sri_ has quit IRC | 10:18 | |
| mnasiadka | ivve: tried looking in /var/lib/neutron/ha_confs/<router id>/ ? | 10:21 |
| ivve | mnasiadka: i have been looking how it creates debug log in keepalived.py in neutron and if logging.is_debug_enabled(cfg.CONF): | 10:21 |
| ivve | cmd.append('-D') | 10:21 |
| ivve | and it does that correctly | 10:22 |
| mnasiadka | ivve: yeah, but keepalived has two options - either log to console, or to syslog - syslog is not available inside Kolla docker image | 10:22 |
| ivve | so im just kinda wondering, am i seeing everything in /var/log/neutron/neutron-l3-agent.log? | 10:22 |
| mnasiadka | ivve: so, in theory neutron should store those logs in the directory I mentioned | 10:22 |
| ivve | i know, so the confs as by your suggestion (i have checked these before too) have nothing regarding logging in them | 10:23 |
| ivve | so im guessing it just goes into /dev/null then | 10:23 |
| mnasiadka | well, seems neutron enforces use_syslog on keepalived in l3 agent | 10:24 |
| ivve | ah okay | 10:25 |
| mnasiadka | ivve: https://github.com/openstack/neutron/blob/master/neutron/agent/l3/keepalived_state_change.py#L159 | 10:25 |
| ivve | i found the log and since -D flag is on all of them due to keepalived.py they should also be full logs with debug messages | 10:26 |
| ivve | thanks mnasiadka | 10:26 |
| mnasiadka | ivve: now you can go and complain to neutron guys :) | 10:26 |
| ivve | yes, i think it is their problem, just need to prove it. puh :( | 10:27 |
| ivve | just gonna figure out if this arping is failing first | 10:27 |
| ivve | i get two masters sometimes causing "splitbrain" on namespaces | 10:27 |
| ivve | and the best part, sometimes | 10:27 |
| *** riuzen has quit IRC | 10:28 | |
| Tony31 | I think this is a bug | 10:28 |
| ivve | could be keepalived bug/problem too i guess | 10:28 |
| ivve | but have to figure out what actually fails and promotes | 10:28 |
| Tony31 | mnasiadka I added a new network called `sanb` in kayobe networks.yml. And in the `group_vars/compute` I added the mapping `sanb_interface: eth4` Now I am trying to add a new compute node in another location and it is getting assigned an IP in `sanb` network when it's not configured for it | 10:30 |
| mnasiadka | ivve: well, as I said - keepalived can log to console or to syslog - kind of it's a keepalived weakness :) | 10:30 |
| hu_berlin_kalle | hi , I get the following error on kayobe overcloud host configure | 10:30 |
| hu_berlin_kalle | ERROR: Cannot uninstall 'ipaddress'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall. | 10:30 |
| hu_berlin_kalle | Did I mess omething up in the config somehow? (Or is this a bug?) | 10:31 |
| hu_berlin_kalle | (the error happens on all controllers and all compute node | 10:31 |
| Tony31 | mnasiadka - I am doing a grep | 10:31 |
| mnasiadka | Tony31: I guess this compute is still in the compute group? | 10:32 |
| mnasiadka | Tony31: group_vars will set a var for ALL hosts in a group | 10:32 |
| mnasiadka | so either create a group in inventory for those computes in another location | 10:32 |
| mnasiadka | or do host_vars instead (for each host) | 10:33 |
| Tony31 | mnasiadka thank you for your guidance! I think this is the issue | 10:33 |
| Tony31 | so it looks like etc/kayobe/compute.yml is taking precendence over the host_vars. Once I commented out the configuration in the compute.yml I now get the correct IPs allocated | 10:38 |
| *** shyamb has quit IRC | 10:46 | |
| Tony31 | What is this scanning for ssh keys and where is it scanning? https://pastebin.com/f7MK30mE | 10:55 |
| Tony31 | this is during the host configure. | 10:56 |
| Tony31 | I dont know how it was resolved before | 10:56 |
| Tony31 | I can ssh from the control host, no issue but the `configure` still fails at this step | 10:59 |
| *** rpittau is now known as rpittau|bbl | 11:03 | |
| *** Tony31 has quit IRC | 11:04 | |
| *** todin has joined #openstack-kolla | 11:18 | |
| cosmicsound | How do you do folks? | 11:18 |
| cosmicsound | yoctozepto , @? | 11:19 |
| *** shyamb has joined #openstack-kolla | 11:33 | |
| *** gfidente has quit IRC | 11:47 | |
| openstackgerrit | Yongjun Bai proposed openstack/kolla-ansible master: Add support for encrypting nova/heat api https://review.opendev.org/707131 | 11:48 |
| mnasiadka | yoctozepto: seems removing /run doesn't break ml2/ovs | 11:53 |
| *** pbing19 has quit IRC | 11:55 | |
| *** rlljorge has joined #openstack-kolla | 12:01 | |
| *** dougsz has quit IRC | 12:01 | |
| *** pbing19 has joined #openstack-kolla | 12:08 | |
| rlljorge | Hello, Someone there any idea about this problem https://bugs.launchpad.net/kolla-ansible/+bug/1863249 | 12:10 |
| openstack | Launchpad bug 1863249 in kolla-ansible "Cloudkitty not working in Stein" [Undecided,New] | 12:10 |
| *** Tony31 has joined #openstack-kolla | 12:10 | |
| *** jcmdln has quit IRC | 12:14 | |
| *** factor has joined #openstack-kolla | 12:15 | |
| *** kplant has joined #openstack-kolla | 12:18 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Stop using deprecated stores and default_store in glance https://review.opendev.org/708114 | 12:21 |
| *** sri_ has joined #openstack-kolla | 12:21 | |
| *** kplant has quit IRC | 12:22 | |
| *** kplant has joined #openstack-kolla | 12:22 | |
| Tony31 | Does anyone know about this ssh key scanning issue that I have? | 12:23 |
| Tony31 | `TASK [ssh-known-host : Scan for SSH keys] ************************************************************************************************************************************************************ | 12:23 |
| Tony31 | "changed": false, "cmd": ["ssh-keyscan"], "delta": "0:00:00.022516", "end": "2020-02-17 18:53:46.918114", "item": "", "msg": "non-zero return code", "rc": 1, "start": "2020-02-17 18:53:46.895598", "stderr": "usage: ssh-keyscan [-46cHv] [-f file] [-p port] [-T timeout] [-t type]\n\t\t [host | addrlist namelist] ...", "stderr_lines": ["usage: | 12:23 |
| Tony31 | ssh-keyscan [-46cHv] [-f file] [-p port] [-T timeout] [-t type]", "\t\t [host | addrlist namelist] ..."], "stdout": "", "stdout_lines": []}` | 12:23 |
| Tony31 | Where would it be trying to scan? | 12:23 |
| Tony31 | the playbook is located `kayobe/venvs/kayobe/share/kayobe/ansible/roles/ssh-known-host/tasks` | 12:28 |
| *** shyamb has quit IRC | 12:36 | |
| *** rpittau|bbl is now known as rpittau | 12:37 | |
| yoctozepto | mnasiadka: do we test instance connectivity though? | 12:38 |
| yoctozepto | mnasiadka: and are there no errors/warnings in neutron services logs? | 12:39 |
| yoctozepto | then I am very +2 on it | 12:39 |
| *** Luzi has joined #openstack-kolla | 12:44 | |
| mnasiadka | yoctozepto: I’m just thinking a stupid ssh to instance would be nice | 12:45 |
| yoctozepto | mnasiadka: would be lovely; as previously said, I think mgoddard did it in kayobe just recently | 12:46 |
| *** pbing19 has quit IRC | 12:49 | |
| *** dougsz has joined #openstack-kolla | 12:51 | |
| yoctozepto | mnasiadka: this looks pretty bad | 12:57 |
| yoctozepto | 2020-02-17 10:34:31.895 6 ERROR neutron.agent.linux.ip_lib [req-2a0b9de6-9a5b-4f25-b34f-a01c49f4dc16 - - - - -] Device tap28a1a217-3a cannot be used as it has no MAC address | 12:57 |
| yoctozepto | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_654/707375/3/check/kolla-ansible-ubuntu-source-multinode-ipv6/654000c/secondary1/logs/kolla/ | 12:57 |
| yoctozepto | mnasiadka: but not sure if it's CI in general or this new change | 12:58 |
| yoctozepto | mnasiadka: CI in general | 12:58 |
| mnasiadka | That’s why we need better network tests in CI :) | 12:58 |
| yoctozepto | mnasiadka: so nothing to worry about before further testing | 12:59 |
| yoctozepto | mnasiadka: yeah, indeed | 12:59 |
| yoctozepto | mnasiadka: only debian&ubuntu get this | 13:00 |
| yoctozepto | mnasiadka: so must be their misconfig | 13:01 |
| yoctozepto | mnasiadka: or it's just superfluous | 13:01 |
| yoctozepto | mnasiadka: odd, I checked several logs results and in non-upgrade success jobs it's the only error we are getting | 13:04 |
| yoctozepto | mnasiadka: and only ubuntu/debian | 13:04 |
| openstackgerrit | Yongjun Bai proposed openstack/kolla-ansible master: Add support for encrypting glance/heat api https://review.opendev.org/707131 | 13:05 |
| mnasiadka | Wonder what is this :) | 13:07 |
| yoctozepto | mnasiadka: only hrw is currently ubuntu/debian core folk | 13:11 |
| yoctozepto | mnasiadka: so not enough love | 13:11 |
| mnasiadka | yoctozepto: and he is not really into deploying :) | 13:12 |
| yoctozepto | mnasiadka: HE IS DA BUILDA | 13:12 |
| yoctozepto | :D | 13:12 |
| mnasiadka | Like Noe and the ark? | 13:12 |
| yoctozepto | kinda | 13:12 |
| yoctozepto | just no animal fetish | 13:12 |
| yoctozepto | I *think* | 13:12 |
| *** Tony31 has quit IRC | 13:18 | |
| *** gfidente has joined #openstack-kolla | 13:25 | |
| *** spiette_ has quit IRC | 14:04 | |
| *** spiette has joined #openstack-kolla | 14:07 | |
| *** dave-mccowan has joined #openstack-kolla | 14:11 | |
| *** bengates has quit IRC | 14:14 | |
| *** pbing19 has joined #openstack-kolla | 14:26 | |
| *** rgogunskiy_ has quit IRC | 14:28 | |
| *** rgogunskiy has joined #openstack-kolla | 14:29 | |
| *** Luzi has quit IRC | 14:30 | |
| *** rgogunskiy has quit IRC | 14:31 | |
| *** riuzen has joined #openstack-kolla | 14:33 | |
| *** zhanglong has quit IRC | 14:36 | |
| *** sri_ has quit IRC | 14:37 | |
| riuzen | Hi, i wanna ask about magnum on openstack-kolla. So create k8s cluster but stuck when creating master node. I setup my openstack to do this: request between service using HTTP API (Internal API) and for external access using HTTPS API (External API).How to make my magnum service request from internal API? From magnum log I got this error: Failed to contact the endpoint at https://10.30.30.101:5000 for discovery. Fallback to using that endpoint as | 14:41 |
| riuzen | the base url.: SSLError: SSL exception connecting to https://10.30.30.101:5000: HTTPSConnectionPool(host='10.30.30.101', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),)) | 14:41 |
| *** pbing19 has quit IRC | 14:47 | |
| *** pbing19 has joined #openstack-kolla | 14:47 | |
| *** pbing19 has quit IRC | 14:49 | |
| *** pbing19 has joined #openstack-kolla | 14:49 | |
| *** hu_berlin_kalle has quit IRC | 14:49 | |
| *** cah_link has quit IRC | 14:50 | |
| todin | Hi, is ovs-dpdk still suposed to work right now? I get this error Interface name undefined for network 'dpdk_tunnel' (set 'dpdk_tunnel_interface'). And the codes was introduced with bc053c0 an there it says "No idea whether ovs-dpdk works at all atm." | 14:55 |
| *** TrevorV has joined #openstack-kolla | 14:58 | |
| *** bengates has joined #openstack-kolla | 15:01 | |
| *** bengates has quit IRC | 15:02 | |
| *** bengates has joined #openstack-kolla | 15:02 | |
| *** gfidente has quit IRC | 15:06 | |
| *** pbing19 has quit IRC | 15:08 | |
| *** pbing19 has joined #openstack-kolla | 15:08 | |
| sorin-mihai | what is the default location for kvm/qemu VMs, /var/lib/libvirt or somewhere else? | 15:15 |
| kplant | if you're using libvirt, the default image directory is /var/lib/libvirt/images | 15:17 |
| kplant | the metadata is in /var/lib/libvirt/qemu | 15:17 |
| *** gfidente has joined #openstack-kolla | 15:19 | |
| todin | kplant: but that dir is only accesible within of the container, from the point of view of the hostsystem it is a docker-volume | 15:20 |
| kplant | they didn't ask about kolla nova_libvirt, just kvm/qemu... so that's what i assumed | 15:20 |
| sorin-mihai | i'm only interested from the host point of view, planning to separate the docker stuff from libvirt so that i can backup with lvm. i see that the kolla_logs is also set somewhere in the docker folder, is there a point to not keep it in the real /var/log? | 15:21 |
| todin | kplant: fair enough | 15:21 |
| sorin-mihai | let me rephrase then, when i use 'openstack server create' where are the 'disk' files stored? | 15:23 |
| kplant | what's your storage backend? | 15:24 |
| sorin-mihai | lvm | 15:24 |
| kplant | right, so cinder volumes would likely be in the "cinder-volumes" vg | 15:25 |
| kplant | and nova volumes would likely be in /var/lig/nova/instances/ | 15:25 |
| kplant | s/lig/lib | 15:26 |
| sorin-mihai | so nova instances would be separated from /var/lib/libvirt/images or is some sort of relation between the two? | 15:28 |
| sorin-mihai | as an extreme case, i could create instances in the host with virsh along those created with nova? | 15:29 |
| kplant | in the interest of avoiding an XY problem | 15:30 |
| kplant | what are you trying to do? | 15:31 |
| sorin-mihai | planing partitioning for a aio host. some of my previous backup plans assumed that /var/lib/libvirt contains (almost) everything needed to 'restore' kvm/qemu images managed through libvirt, at least the metadata and the raw qcow files. i'm sure is not the best aproach... | 15:33 |
| *** skramaja has quit IRC | 15:33 | |
| sorin-mihai | i have a 4tb raid1 for the OS and a 20tb raid50 for hot backups. both 'drives' will be managed with lvm, i'm thinking of separating some things in different volumes, like /var/log, /var/lib/libvirt/, /var/lib/docker and some others | 15:36 |
| kplant | are you thinking of doing an aio for production use? | 15:37 |
| *** bengates has quit IRC | 15:37 | |
| *** bengates has joined #openstack-kolla | 15:38 | |
| sorin-mihai | it's a dev shop, just 1 server for now, if it proves it's use in the development cycle we get more servers and move to a decent multi node setup. initially i planned the server only for libvirt instances, but by the time the server arrived i changed my mind. lol | 15:38 |
| *** gbatir_ has quit IRC | 15:40 | |
| *** bengates has quit IRC | 15:46 | |
| *** bengates has joined #openstack-kolla | 15:48 | |
| kplant | got it | 15:50 |
| kplant | so as todin pointed out, libvirt is ran in a docker container with k-a | 15:51 |
| kplant | so all of the libvirt metadata lives within the volume | 15:51 |
| kplant | and i don't think there are any binds to pull guest images out | 15:51 |
| *** riuzen has quit IRC | 15:52 | |
| sorin-mihai | for me, this is strange now. why run libvirt inside a docker and force the hypervisor to downgrade to qemu, instead of running libvirt along docker in the host? | 15:53 |
| sorin-mihai | or this is only in the aio case? | 15:54 |
| kplant | just because libvirtd is running in a container doesn't mean the host can't use qemu-kvm | 15:55 |
| sorin-mihai | yeah, i know. just trying to understand what would be the benefit(s) | 15:56 |
| noxoid_ | greater control over the libvirt/qemu versions and upgrade process | 16:00 |
| sorin-mihai | so, would the disk files used by nova instances be raw qcow on top of the filesystem, or that would happen only if using cinder? | 16:00 |
| sorin-mihai | or if using cinder they will actually be lv and not qcow files? /me goes rtfm about cinder | 16:05 |
| noxoid_ | do not confuse cinder volume storage and nova storage. they can be configured to use separate backends | 16:05 |
| noxoid_ | if you have nova configured to use lvm via https://docs.openstack.org/nova/stein/configuration/config.html#libvirt.images_type then the data will be on an LV with the metadata in the aforementioned /var location | 16:09 |
| noxoid_ | iirc the default, if you made no changes, is qcow2 files in the aforementioned /var location | 16:10 |
| sorin-mihai | yeah, i got that. though, with the minimal configuration needed to get things running cinder is not enabled, so i didn't think about that too much during the tests i've run so far (bear with me please, i had to also adjust to centos after more than 10 years of "anything else but rpm") | 16:11 |
| *** bengates has quit IRC | 16:13 | |
| *** bengates has joined #openstack-kolla | 16:27 | |
| *** bengates has quit IRC | 16:29 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Stop using deprecated stores and default_store in glance https://review.opendev.org/708114 | 16:31 |
| *** bengates has joined #openstack-kolla | 16:31 | |
| *** bengates has quit IRC | 16:34 | |
| *** bengates has joined #openstack-kolla | 16:35 | |
| openstackgerrit | Hongbin Lu proposed openstack/kolla-ansible master: [WIP] Zun: Add zun-cni-daemon to compute node https://review.opendev.org/708213 | 16:46 |
| *** bengates has quit IRC | 16:48 | |
| *** bengates has joined #openstack-kolla | 16:48 | |
| *** rpittau is now known as rpittau|afk | 16:54 | |
| openstackgerrit | Hongbin Lu proposed openstack/kolla-ansible master: [WIP] Zun: Add zun-cni-daemon to compute node https://review.opendev.org/708213 | 16:55 |
| yoctozepto | todin: hi, it is surely supposed to by general means; the commit is about IPv6 and it most likely does not work with ovsdpdk atm due to aforementioned oddity | 16:57 |
| yoctozepto | todin: please raise a bug about ovsdpdk if you think it's broken, we likely have shortage of its users | 16:57 |
| *** ivve has quit IRC | 16:58 | |
| *** bengates has quit IRC | 16:59 | |
| *** gfidente has quit IRC | 17:18 | |
| openstackgerrit | Jason Anderson proposed openstack/kolla-ansible master: [gnocchi] Disable statsd daemon by default https://review.opendev.org/671618 | 17:28 |
| *** factor has quit IRC | 17:30 | |
| *** evrardjp has quit IRC | 17:34 | |
| *** evrardjp has joined #openstack-kolla | 17:34 | |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: WIP: CI: Refactor VXLAN overlay setup https://review.opendev.org/708217 | 17:46 |
| *** cz3 is now known as tequilasunset | 17:55 | |
| *** dougsz has quit IRC | 17:56 | |
| *** tequilasunset is now known as cz3 | 17:56 | |
| openstackgerrit | wes hayutin proposed openstack/kolla master: change tripleo centos-7 build containers to centos-8 https://review.opendev.org/708218 | 17:59 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla master: Get rid of Python 2 support https://review.opendev.org/691316 | 18:01 |
| weshay|ruck | FYI.. folks https://review.opendev.org/708218 remove the blocking centos-7 tripleo build containers job | 18:03 |
| patchbot | patch 708218 - kolla - change tripleo centos-7 build containers to centos-8 - 1 patch set | 18:03 |
| *** aleccoder has quit IRC | 18:06 | |
| *** klippo has quit IRC | 18:07 | |
| r3ap3r | I have two questions for the Dev team when anyone gets a second. Have yall moved to all Python 3 in your Gitlab Senarios? If so, do you have any "hey, look out for this" if I try a deployment using only Python 3 on CentOS 7? | 18:09 |
| r3ap3r | I meant "all" of your Gitlab Senarios. | 18:09 |
| yoctozepto | weshay|ruck: thanks, already +2 | 18:10 |
| yoctozepto | r3ap3r: c7 py3 is probably still missing selinux bindings | 18:10 |
| weshay|ruck | np :) thanks to you folks ( in general ) | 18:10 |
| weshay|ruck | think that is the case re: selinux | 18:10 |
| yoctozepto | r3ap3r: and we never moved to py3 on c7 | 18:11 |
| yoctozepto | r3ap3r: we are moving to c8 :-) | 18:11 |
| r3ap3r | yoctozepto: yeah, I saw that when looking up the deps manually first. | 18:11 |
| yoctozepto | r3ap3r: btw, we don't have gitlab but zuul | 18:11 |
| yoctozepto | r3ap3r: nice piece of software | 18:11 |
| yoctozepto | r3ap3r: yeah, it's breaking for ansible | 18:11 |
| yoctozepto | r3ap3r: not much to do for us unless you want to hack ansible for yourself | 18:12 |
| yoctozepto | r3ap3r: or provide selinux bindings | 18:12 |
| *** dannins has quit IRC | 18:13 | |
| r3ap3r | yoctozepto: yeah, I'm aware of the move to C8, just didn't know if it was ready or not, was going to run through a fresh deploy today. I have not messed with zuul, I will have to take a peak. I'm working on my "DevOps" skillset but not quite there with being comfortable enough to contribute to a project yet. ;-) | 18:14 |
| yoctozepto | r3ap3r: I see, zuul is openstack (well, actually opendev atm) -wide solution for CI/CD | 18:16 |
| yoctozepto | r3ap3r: https://zuul-ci.org/docs/zuul/index.html | 18:16 |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Add support for encrypting backend HAProxy traffic https://review.opendev.org/664516 | 18:17 |
| r3ap3r | yoctozepto: Cool, will check it out. Thanks. :-) | 18:18 |
| *** tonythomas has quit IRC | 18:20 | |
| *** aleccoder has joined #openstack-kolla | 18:26 | |
| openstackgerrit | Jason Anderson proposed openstack/kolla-ansible master: [haproxy] Support interfaces with '-' chars https://review.opendev.org/663325 | 18:36 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Stop using deprecated stores and default_store in glance https://review.opendev.org/708114 | 18:42 |
| *** klippo has joined #openstack-kolla | 18:47 | |
| *** Tengu has quit IRC | 19:07 | |
| *** Tengu has joined #openstack-kolla | 19:09 | |
| *** Tengu has quit IRC | 19:14 | |
| *** Tengu has joined #openstack-kolla | 19:15 | |
| *** pbing19 has quit IRC | 19:19 | |
| *** Tengu has quit IRC | 19:21 | |
| *** Tengu has joined #openstack-kolla | 19:22 | |
| openstackgerrit | Jason Anderson proposed openstack/kolla-ansible master: [haproxy] Support interfaces with '-' chars https://review.opendev.org/663325 | 19:32 |
| *** rlljorge has quit IRC | 19:38 | |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: CI: Refactor VXLAN overlay setup https://review.opendev.org/708217 | 19:43 |
| *** Tengu has quit IRC | 19:44 | |
| *** Tengu has joined #openstack-kolla | 19:46 | |
| *** klippo has quit IRC | 19:46 | |
| *** aleccoder has quit IRC | 19:46 | |
| *** openstackstatus has quit IRC | 19:49 | |
| *** Tengu has quit IRC | 19:51 | |
| *** Tengu has joined #openstack-kolla | 19:55 | |
| *** adeberg has quit IRC | 20:12 | |
| *** aleccoder has joined #openstack-kolla | 20:17 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: CI: test-core-openstack: add floating ip and ssh to the instance https://review.opendev.org/708250 | 20:21 |
| *** kplant has quit IRC | 20:23 | |
| todin | yoctozepto: Thanks for your response, right now I am trying to figure out if it its broken or just badly documented, as soon as I know more I will let you know | 20:29 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: WIP: CI: Refactor VXLAN overlay setup https://review.opendev.org/708217 | 20:30 |
| *** klippo has joined #openstack-kolla | 20:32 | |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Add support for encrypting backend HAProxy traffic https://review.opendev.org/664516 | 20:33 |
| yoctozepto | todin: could be both! | 20:34 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: CI: Refactor VXLAN overlay setup https://review.opendev.org/708217 | 20:40 |
| todin | yoctozepto: maybe, but the kolla projekt is still awsome, thank to you all for that | 20:41 |
| yoctozepto | todin: thanks! | 20:42 |
| *** TrevorV has quit IRC | 21:33 | |
| openstackgerrit | Hongbin Lu proposed openstack/kolla-ansible master: [WIP] Zun: Add zun-cni-daemon to compute node https://review.opendev.org/708213 | 21:37 |
| *** lennyb has quit IRC | 22:17 | |
| *** lennyb has joined #openstack-kolla | 22:20 | |
| *** openstackstatus has joined #openstack-kolla | 23:02 | |
| *** ChanServ sets mode: +v openstackstatus | 23:02 | |
| *** igordc has joined #openstack-kolla | 23:03 | |
| r3ap3r | `kolla-ansible deploy` and `kolla-ansible reconfigure` seem to be looking for `haproxy-internel.pem` but it doesn't appear to be generated by the `certificates` playbook? Am I missing something? Notes: AIO Deployment on CentOS 7 from Source: any other questions are welcome. Please see below pastebin for further details about the playbook errors. | 23:06 |
| r3ap3r | https://pastebin.com/3kNYD3i1 | 23:06 |
| openstackgerrit | Hongbin Lu proposed openstack/kolla master: Zun: add zun-cni-daemon image https://review.opendev.org/708273 | 23:07 |
| openstackgerrit | Hongbin Lu proposed openstack/kolla master: [WIP] Zun: add zun-cni-daemon image https://review.opendev.org/708273 | 23:07 |
| r3ap3r | I also looked in the `/etc/kolla/certificates/` directory and searched for the `haproxy-internal.pem` throughout the entire drive and it doesn't exists. | 23:07 |
| openstackgerrit | Hongbin Lu proposed openstack/kolla-ansible master: [WIP] Zun: Add zun-cni-daemon to compute node https://review.opendev.org/708213 | 23:08 |
| r3ap3r | I'm also looking at the `generate.yml` on the Kolla Github project and it also doesn't reference creating the `haproxy-internal.pem` either. Maybe it is supposed to be somewhere else? | 23:12 |
| *** goldyfruit_ has quit IRC | 23:51 | |
| *** negronjl has quit IRC | 23:55 | |
| *** negronjl has joined #openstack-kolla | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
