Monday, 2020-05-04

openstackgerritjacky06 proposed openstack/kayobe master: [WIP]Remove six  https://review.opendev.org/72515503:08
openstackgerritjacky06 proposed openstack/kayobe master: Remove six  https://review.opendev.org/72515503:29
*** ricolin_ has joined #openstack-kolla03:49
*** JangwonLee has joined #openstack-kolla04:17
*** ykarel|away is now known as ykarel04:19
*** JangwonLee_ has quit IRC04:20
*** evrardjp has quit IRC04:35
*** evrardjp has joined #openstack-kolla04:36
*** skramaja has joined #openstack-kolla04:37
*** skramaja has quit IRC05:05
*** skramaja has joined #openstack-kolla05:06
openstackgerritjacky06 proposed openstack/kolla-cli master: Remove six  https://review.opendev.org/72516405:17
*** cah_link has joined #openstack-kolla05:26
*** e0ne has joined #openstack-kolla05:43
*** vishalmanchanda has joined #openstack-kolla05:43
*** e0ne has quit IRC05:48
sri_morning06:20
*** ykarel is now known as ykarel|afk06:21
*** e0ne has joined #openstack-kolla06:24
*** dciabrin has joined #openstack-kolla06:29
*** e0ne has quit IRC06:29
*** rpittau|afk is now known as rpittau06:33
*** ykarel|afk is now known as ykarel06:37
*** seco has joined #openstack-kolla06:52
*** nathharp has quit IRC07:03
*** e0ne has joined #openstack-kolla07:05
openstackgerritjacky06 proposed openstack/kolla-cli master: Remove six  https://review.opendev.org/72516407:06
*** e0ne has quit IRC07:10
*** jbadiapa has joined #openstack-kolla07:19
*** bengates has joined #openstack-kolla07:26
*** bengates has quit IRC07:26
*** bengates has joined #openstack-kolla07:26
*** born2bake has joined #openstack-kolla07:30
*** e0ne has joined #openstack-kolla07:47
*** e0ne has quit IRC07:52
*** k_mouza has joined #openstack-kolla07:56
*** nathharp has joined #openstack-kolla07:58
*** k_mouza has quit IRC08:00
*** stingrayza has quit IRC08:15
mgoddardmorning08:16
mgoddarddking: still around?08:17
*** stingrayza has joined #openstack-kolla08:18
*** nathharp has quit IRC08:24
*** nathharp has joined #openstack-kolla08:25
erolgmorning08:27
*** hamzaachi has joined #openstack-kolla08:28
*** nathharp has quit IRC08:30
*** stingrayza has quit IRC08:32
*** stingrayza has joined #openstack-kolla08:49
*** nathharp has joined #openstack-kolla08:53
*** nathharp has quit IRC08:55
*** gfidente has joined #openstack-kolla08:57
*** nathharp has joined #openstack-kolla08:57
*** stingrayza has quit IRC09:03
*** k_mouza has joined #openstack-kolla09:03
*** e0ne has joined #openstack-kolla09:04
*** stingrayza has joined #openstack-kolla09:07
*** e0ne has quit IRC09:09
hrwmorning09:10
*** hamzaachi has quit IRC09:14
*** hamzaachi has joined #openstack-kolla09:14
*** abdysn has joined #openstack-kolla09:15
*** hamzaachi_ has joined #openstack-kolla09:17
*** hamzaachi has quit IRC09:19
*** nathharp has quit IRC09:21
*** e0ne has joined #openstack-kolla09:22
*** ykarel is now known as ykarel|lunch09:22
*** nathharp has joined #openstack-kolla09:26
*** stingrayza has quit IRC09:29
*** nathharp has quit IRC09:33
hrwmgoddard: what do you think about merging https://review.opendev.org/721329 as it is + update to c-r-openstack-train once it finally lands?09:43
patchbotpatch 721329 - kolla (stable/train) - switch to official CentOS 8 repositories - 10 patch sets09:43
hrwmgoddard: soon will be 2 weeks while we wait for it09:43
mgoddardhrw: yoctozepto suggested we should wait09:43
hrwok then09:43
mgoddardthe gpgcheck is an issue for me09:43
hrwso we wait09:44
mgoddardhrw: we're still blocked on releasing09:44
mgoddardyeah09:44
hrwin meaintime I ask daily ;D09:44
mgoddardI'm thinking I will release everything else, and just wait for kolla09:44
mgoddardplease do :D09:44
openstackgerritMark Goddard proposed openstack/kolla master: Use Python 3 for mod_wsgi in cyborg and monasca on Debian/Ubuntu  https://review.opendev.org/72522109:45
hrwmgoddard: https://review.opendev.org/725032 can be merged in meantime09:46
patchbotpatch 725032 - kolla - CentOS: disable more repos - 1 patch set09:46
*** nathharp has joined #openstack-kolla09:47
*** nathharp has quit IRC09:49
*** nathharp has joined #openstack-kolla09:55
*** stingrayza has joined #openstack-kolla10:01
openstackgerritMark Goddard proposed openstack/kayobe master: CI: Test host configuration persistence  https://review.opendev.org/72481910:03
openstackgerritMark Goddard proposed openstack/kayobe master: CentOS 8: Fix network configuration persistence  https://review.opendev.org/72522410:03
hrwmgoddard: thx for looking - started test build to be sure10:09
*** rpittau is now known as rpittau|bbl10:14
hrwfsck. we have opstools and centos-opstools...10:15
hrwsame repo twice10:15
hrwwill sort it10:17
mgoddarddoh10:18
*** hamzaachi_ has quit IRC10:24
hrwmgoddard: https://review.rdoproject.org/r/2715110:28
hrwmgoddard: turns out that there are several images in a need of opstools ;D10:30
hrwmgoddard: the problem is that openstack-base is one of them so all it's children gets it enabled too10:32
mgoddardhrw: what needs it in openstack-base?10:32
*** ykarel|lunch is now known as ykarel10:32
hrwpython3-sqlalchemy10:33
hrwcommit 310802d2f63197d46f6e46f0f573dc1e21981c2910:34
hrwsee10:34
mgoddardok. I guess it's not a huge problem10:36
hrwdo not think so10:38
hrwmostly jansson10:39
hrwit is in 6 images so not yet valid to be moved to openstack-base ;D10:40
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: CentOS: disable more repos  https://review.opendev.org/72503210:46
hrww8. one more change10:46
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: CentOS: disable more repos  https://review.opendev.org/72503210:47
hrwnow should be fine10:47
*** cah_link1 has joined #openstack-kolla11:01
*** cah_link has quit IRC11:03
*** cah_link has joined #openstack-kolla11:03
*** kevinz has quit IRC11:04
*** cah_link1 has quit IRC11:05
mgoddardkolla & kolla ansible stable releases: https://review.opendev.org/72523311:15
patchbotpatch 725233 - releases - Release stable branches for Kolla and Kolla-Ansible - 1 patch set11:15
mgoddardexcept kolla for train, waiting on centos811:15
hrw+111:37
*** born2bake has quit IRC11:41
openstackgerritMark Goddard proposed openstack/kolla master: Use Python 3 for mod_wsgi in cyborg and monasca on Debian/Ubuntu  https://review.opendev.org/72522111:46
*** hamzaachi_ has joined #openstack-kolla11:51
*** abdysn has quit IRC12:04
*** born2bake has joined #openstack-kolla12:05
maharg101does kolla generally support the use of network bridges in place of interfaces e.g. network_interface: "brmgmt"12:07
*** born2bake has quit IRC12:07
*** dmellado has quit IRC12:10
maharg101https://docs.openstack.org/kolla-ansible/train/admin/production-architecture-guide.html#network-configuration would imply it does, but we hit problems in `kolla_address` function when using bridges12:13
*** dmellado has joined #openstack-kolla12:16
*** rpittau|bbl is now known as rpittau12:22
openstackgerritMark Goddard proposed openstack/kayobe master: CI: Test host configuration persistence  https://review.opendev.org/72481912:30
*** ykarel is now known as ykarel|afk12:38
mgoddardmaharg101: bridges should be fine12:42
mgoddardmaharg101: historically there was a problem with dashes in interface names, although I think this may work since train12:43
maharg101mgoddard: many thanks12:43
dkingmgoddard: I was in the channel, but just getting in for the morning now.12:51
*** cz3 is now known as poweredgelord12:53
openstackgerritMark Goddard proposed openstack/kayobe master: CI: Test host configuration persistence  https://review.opendev.org/72481912:53
dkingSo, I'm getting my cloud server to boot, but the network configuration isn't correct. There's no bond or vlan created. One reason might be that I'm seeing once booted, it's using different interface names (eth0 and eth1 as opposed to eno1 and eno2).12:56
dkingI updated network-interfaces, but that didn't seem to change anything, and I wanted to make sure the changes were being recognized, or see if there were some other problem.12:57
*** poweredgelord is now known as cz312:57
dkingmgoddard: What would be the best way of investigating where those interfaces are being setup? This is for an overcloud deployment.13:09
*** ricolin_ has quit IRC13:17
*** hamzaachi_ has quit IRC13:19
*** hamzaachi_ has joined #openstack-kolla13:19
mgoddarddking: which command is failing?13:20
dkingkayobe overcloud provision. It's failing at "Wait for SSH access to the nodes". It's failing because, even though it has the right IP and the IP is setup, it's not on a VLAN.13:22
*** hamzaachi_ has quit IRC13:26
*** hamzaachi__ has joined #openstack-kolla13:26
*** hamzaachi__ has quit IRC13:31
*** hamzaachi__ has joined #openstack-kolla13:31
*** KeithMnemonic has joined #openstack-kolla13:31
*** cah_link has quit IRC13:33
dkingmgoddard: This should have the relevant information: http://paste.openstack.org/show/793051/13:35
dkingAt the moment, I'm trying to understand exactly how the network information is supposed to be setup. I see that the deploy image requests a configdrive-*.iso.gz image, so I'm assuming that's it. I just don't understand what role creates that. I saw a role jriguera.configdrive, but that seems to only be called by seed-vm-provision.yml?13:37
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: CentOS: disable more repos  https://review.opendev.org/72503213:39
hrwnow it should pass13:39
*** cah_link has joined #openstack-kolla13:39
*** TrevorV has joined #openstack-kolla13:48
mgoddarddking: bifrost creates the configdrives for overcloud hosts13:52
*** k_mouza has quit IRC13:52
dkingOkay, so that's all on Bifrost, then? I'm thinking that if I can find where it hands that off, I can ask more intelligent questions to the bifrost folks.13:53
mgoddarddking: have you set management_vlan to 83 in networks.yml?13:53
*** hamzaachi_ has joined #openstack-kolla13:53
dkingNo, I haven't. That might be it. I knew that I was missing something simple.13:54
dkingBut it doesn't set the bond. Let me give that a try.13:54
*** hamzaachi__ has quit IRC13:56
openstackgerritMark Goddard proposed openstack/kayobe master: CI: Test host configuration persistence  https://review.opendev.org/72481913:57
*** k_mouza has joined #openstack-kolla13:59
mgoddarddking: after the provision, bifrost only configures one interface - the admin interface. Other networks are configured afterwards by kayobe, in 'overcloud host configure'14:01
dkingThat's good to know. I figured that it would do some more work during that later step. I just have to get it there. I'm hoping that the fact that I'm needing a vlan won't be a problem.14:03
*** k_mouza has quit IRC14:03
*** nathharp has quit IRC14:04
dkingmgoddard: I deprovisioned and then provisioned the overcloud. I saw that configdrive-*.iso.gz was updated, but still no bond, and still no VLAN. How far back should I have to go to see those changes applied, as in, would I have to do any other commands, such as seed host configure?14:08
mgoddarddking: there is no bond support in bifrst14:09
mgoddarddking: the provision command should regenerate the bifrost variables14:09
mgoddarddking: on the seed, check /etc/kolla/bifrost/inventory/14:10
mgoddardhost_vars should include the vlan14:10
*** ykarel|afk is now known as ykarel14:10
*** nathharp has joined #openstack-kolla14:12
dkingOkay. I'm gong to try again. I think this might answer my questions. I'll confirm in just a moment.14:14
*** cdearborn has joined #openstack-kolla14:14
*** hamzaachi__ has joined #openstack-kolla14:17
mgoddarddking: https://opendev.org/openstack/kayobe/src/branch/master/ansible/kolla-bifrost-hostvars.yml is where that is generated14:20
*** hamzaachi_ has quit IRC14:20
*** k_mouza has joined #openstack-kolla14:21
openstackgerritMark Goddard proposed openstack/kayobe master: CI: Test host configuration persistence  https://review.opendev.org/72481914:21
*** nathharp has quit IRC14:28
dkingmgoddard: Thank you very much! This has been very helpful. Now, I'm seeing the vlan information set. I'm just having trouble because the interface doesn't actually seem to come up on the host. So, I'm going to try sorting through that. Now that I know where things are set, I think I can make it further.14:42
mgoddarddking: cloud-init logs are probably the next thing to check14:42
mgoddardswitching to a vlan has always been a little bit tricky14:42
dkingRight now, the problem seems to be that if I run "ifup eth0.83", I get "Error: Connection activation failed: No suitable device found for this connection (device eth0 not available because profile is not compatible with device (mismatching interface name))." Even though eth0 shows up in "ip a".14:43
dkingI think it's on the right track, because ifcfg-eth0.83 seems to have the correct info in it. At least, it shows the VLAN and the IPs as I expect.14:45
*** nathharp has joined #openstack-kolla14:47
mgoddarddking: we don't use NetworkManager. If you run the host configure command it should improve things14:55
dkingThe provision won't succeed yet due to the fact that the ansible host can't connect over the management interface. It's stuck waiting for SSH access that will never come.14:56
*** seco has quit IRC15:07
mgoddarddking: if the interface has an IP on the right VLAN then it sounds like an L2 issue15:14
mgoddardswitch has the right VLAN?15:14
*** nathharp has quit IRC15:14
dkingThe port should be a trunk on the switch.15:15
dkingAnd the interface won't bring up the IP.15:16
dkinghttp://paste.openstack.org/show/793066/15:16
dkingThe IP that's setup is for a different network, the provisioning network. I have that set for oob_oc_net_name, provision_oc_net_name, and oob_wl_net_name, but won't expect that to be available once it's deployed.15:19
mgoddarddking: are you using centos 7 or 815:20
dking815:20
mgoddardah15:20
mgoddardpossibly you need to install network-scripts package and stop NetworkManager15:20
mgoddardI haven't tried using a VLAN for the admin network on CentOS 815:21
*** nathharp has joined #openstack-kolla15:21
dkingI'm using 8 for the deploy ramdisk and overcloud. My ansible host is using 7.15:21
dkingI don't have admin_oc_net_name using the provisioning network. It's using my management network, which uses the VLAN.15:22
dkingSo, maybe that's the problem?15:22
*** hamzaachi_ has joined #openstack-kolla15:23
dkingAlso, when you say to install network-scripts, would that be somewhere in the deploy ramdisk, or where would I set that up to occur when running kayobe overcloud provision before it tries to test the ssh connection?15:24
*** hamzaachi__ has quit IRC15:25
*** born2bake has joined #openstack-kolla15:27
openstackgerritMerged openstack/kolla master: debian: binary target is x86 only  https://review.opendev.org/72461715:29
mgoddarddking: the admin network needs to be on the same physical network as the provisioning network15:32
mgoddardusing a vlan on that network should be supported, but as I said so far untested on centos 715:33
mgoddardit also doesn't really play nicely with bonds15:33
mgoddardinstalling network scripts would need to be done in the overcloud image, but I would probably try to manually get the network connectivity working first15:34
mgoddardobviously it's a bit hard if you don't have network connectivity15:34
mgoddardconfig for the overcloud image is here: https://docs.openstack.org/kayobe/latest/configuration/bifrost.html#overcloud-root-disk-image-configuration15:35
dkingOkay. Well, maybe I need to make sure I'm on the right track. So, I'm going to want a single ansible host (maybe seed host as well) to be able to provision multiple overclouds. Once the overclouds are configured and deployed, I want none of them to share any networks with each other. Each overcloud should have it's own set of VLANs, and preferrably shouldn't be able to communicate back to the ansible/seed host. I will15:38
dkingep BMC access to check on their power status, at least. Does it sounds like I'm on the right path for that?15:38
*** xclan has joined #openstack-kolla15:39
hrwmgoddard: train/c8 gets stuck for a while.15:40
hrwmgoddard: centos 8.2 release work got team busy15:41
*** hamzaachi_ has quit IRC15:42
*** ykarel is now known as ykarel|away15:47
*** hamzaachi has joined #openstack-kolla15:47
*** hamzaachi has quit IRC15:49
*** hamzaachi_ has joined #openstack-kolla15:49
*** hamzaachi__ has joined #openstack-kolla15:58
mgoddarddking: makes sense, yes. It should work, and if it doesn't then it's a bug15:59
mgoddarddking: is that same NIC part of a bond?15:59
*** hamzaachi_ has quit IRC16:00
dkingEventually, it should be. That's because the servers have two NICs and we're bonding them for performance and redundancy. I don't care as much about that during provisioning.16:02
*** nathharp has quit IRC16:03
*** rpittau is now known as rpittau|afk16:08
xclanI've upgraded kolla-ansible package from 7.0.1 to 7.2.1 (rocky) and started kolla-ansible deploy (also I've used reconfigure option), but it seems docker images versions were not changed. Should I use kolla-ansible upgrade inside a single release or just pull images by docker command? Thanks in advance.16:14
noxoidi would do kolla-ansible pull followed by kolla-ansible upgrade16:16
noxoidpull to pre-stage the images and upgrade to deploy everything and apply any database migrations16:16
noxoidim not sure if thats the correct way to do it but it has worked for me16:21
xclanThanks. Yes, I agree. Indeed, looks like there is no difference beetween monor or major upgrade - the meaning and approach is the same.16:24
xclanAs for database mogration - should I make some manual actions to achieve that?16:25
noxoidno that should be handled by the "upgrade" action16:25
noxoidhttps://opendev.org/openstack/kolla/src/branch/stable/rocky/docker/magnum/magnum-api/extend_start.sh#L616:27
noxoidhttps://opendev.org/openstack/kolla-ansible/src/branch/stable/rocky/ansible/roles/magnum/tasks/bootstrap_service.yml16:27
noxoidas examples16:27
noxoidbootstrap_service.yml is called during the upgrade action, which sets KOLLA_BOOTSTRAP env var, which extend_start.sh detects and runs the database upgrade/migrate  command16:27
*** hamzaachi_ has joined #openstack-kolla16:29
xclanExactly! I'm just performing upgrade for these changes in docker images where the behavior of database upgrade has changed: https://review.opendev.org/#/c/655859/1/docker/monasca/monasca-api/extend_start.sh16:32
patchbotpatch 655859 - kolla (stable/rocky) - Use alembic migrations to deploy Monasca DB (MERGED) - 1 patch set16:32
*** hamzaachi__ has quit IRC16:32
xclanfunny coincidence16:32
*** hamzaachi__ has joined #openstack-kolla16:35
*** hamzaachi_ has quit IRC16:35
*** evrardjp has quit IRC16:35
*** evrardjp has joined #openstack-kolla16:36
*** bengates has quit IRC16:46
*** born2bake has quit IRC16:47
*** bornie2bake has joined #openstack-kolla16:51
*** skramaja has quit IRC16:54
*** bornie2bake has quit IRC16:54
*** bornie2bake has joined #openstack-kolla16:55
*** Fl1nt has joined #openstack-kolla16:56
Fl1ntGood Evening everyone!16:56
dkingFl1nt: Good evening.16:59
Fl1ntHope you all doing it good :D17:00
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Use FQDN for kibana and elasticsearch  https://review.opendev.org/72534017:04
*** hamzaachi__ has quit IRC17:08
*** bornie2bake has quit IRC17:08
*** bornie2bake has joined #openstack-kolla17:11
*** vishalmanchanda has quit IRC17:15
*** bornie2bake has quit IRC17:20
openstackgerritEmilien Macchi proposed openstack/kolla master: Remove tripleoclient container from Kolla  https://review.opendev.org/72535217:35
*** born2bake has joined #openstack-kolla17:40
*** e0ne has quit IRC17:46
*** mrunge_ has joined #openstack-kolla17:55
*** mrunge has quit IRC17:56
*** Fl1nt has quit IRC17:57
*** born2bake has quit IRC18:06
*** jonaspaulo has joined #openstack-kolla18:19
*** Jeffrey4l has quit IRC18:23
*** Jeffrey4l has joined #openstack-kolla18:31
*** timburke has joined #openstack-kolla18:33
*** Jeffrey4l has quit IRC18:36
*** Jeffrey4l has joined #openstack-kolla18:43
*** nathharp has joined #openstack-kolla18:54
*** nathharp has quit IRC19:00
*** nathharp has joined #openstack-kolla19:01
*** nathharp has quit IRC19:05
*** nathharp has joined #openstack-kolla19:05
openstackgerritMark Goddard proposed openstack/kayobe master: CI: Test host configuration persistence  https://review.opendev.org/72481919:09
*** e0ne has joined #openstack-kolla19:19
*** k_mouza has quit IRC19:46
*** cah_link has quit IRC20:21
*** threestrands has joined #openstack-kolla20:34
*** nathharp has quit IRC20:37
*** nathharp has joined #openstack-kolla20:38
*** nathharp has quit IRC20:40
*** nathharp_ has joined #openstack-kolla20:40
*** ysirndjuro has joined #openstack-kolla21:19
*** hamzaachi__ has joined #openstack-kolla21:20
*** e0ne has quit IRC21:36
*** hamzaachi_ has joined #openstack-kolla21:41
*** hamzaachi__ has quit IRC21:44
*** k_mouza has joined #openstack-kolla21:47
*** k_mouza has quit IRC21:52
*** e0ne has joined #openstack-kolla22:09
*** e0ne has quit IRC22:24
*** jonaspaulo has quit IRC22:28
*** hamzaachi_ has quit IRC22:36
*** TrevorV has quit IRC23:00
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Include SSL Client Certificate in service API execution  https://review.opendev.org/72421723:32
*** xclan has quit IRC23:38

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!