Friday, 2020-01-31

« Thursday, 2020-01-30 Index Saturday, 2020-02-01 »
*** billp has quit IRC00:07
*** billp has joined #openstack-kolla00:17
*** calbers has quit IRC00:24
*** calbers has joined #openstack-kolla00:27
*** ricolin has joined #openstack-kolla00:48
*** igordc has quit IRC01:43
*** rlljorge has quit IRC01:52
openstackgerritMerged openstack/kolla master: CentOS 8: Remove shellinabox from ironic-conductor  https://review.opendev.org/70481302:05
*** k_mouza_ has quit IRC02:56
*** negronjl has quit IRC02:58
*** negronjl has joined #openstack-kolla02:59
*** k_mouza has joined #openstack-kolla03:03
*** k_mouza has quit IRC03:08
*** k_mouza has joined #openstack-kolla03:37
*** k_mouza has quit IRC03:42
openstackgerritAntony Messerli proposed openstack/kolla master: Adds python3-systemd for ansible deploy interface  https://review.opendev.org/70502104:00
*** mixor has joined #openstack-kolla04:10
*** mixor has quit IRC04:32
*** skramaja has joined #openstack-kolla04:38
*** evrardjp has quit IRC05:33
*** evrardjp has joined #openstack-kolla05:33
*** k_mouza has joined #openstack-kolla05:38
*** k_mouza has quit IRC05:42
*** tony31 has joined #openstack-kolla05:56
*** jbadiapa has quit IRC06:20
*** cah_link has joined #openstack-kolla06:22
*** goldyfruit_ has quit IRC06:24
*** goldyfruit has joined #openstack-kolla06:24
tony31good morning06:26
tony31I'm running a VM of kayobe universe from nothing on top of Red Hat Virtualisation (KVM) which is basically the open source `oVirt`. When I execute the seed-deploy.sh I get an error that it does not understand the kvm hypervisor:06:33
tony31TASK [stackhpc.libvirt-vm : Ensure the VM is defined] ****************************************************************************************************************06:33
tony31arch=x86_64 domaintype=kvm "}06:33
tony31I was able to resolve that issue by installing extra packages and then rebooting the VM. `yum install qemu-kvm libvirt libvirt-python libguestfs-tools virt-install`06:50
tony31the `libvirtd` service was already running and restarting it didnt resolve that issue. Trying the reboot did resolve. Not sure why, as checking the enabled kernel module showed it had been enabled for virtualisation anyway06:55
tony31this is for `kayobe`06:55
osmanlicilegimorning06:59
*** Jeffrey4l has quit IRC07:06
*** Jeffrey4l has joined #openstack-kolla07:12
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible upgrade jobs  https://review.opendev.org/70498407:14
mnasiadkamorning07:14
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible upgrade jobs  https://review.opendev.org/70498407:27
*** k_mouza has joined #openstack-kolla07:39
*** jbadiapa has joined #openstack-kolla07:39
*** iputra has joined #openstack-kolla07:43
*** k_mouza has quit IRC07:43
*** luksky has joined #openstack-kolla07:51
*** luksky has quit IRC08:02
openstackgerritDincer Celik proposed openstack/kolla-ansible master: Follow up on Ansible version checking  https://review.opendev.org/70517108:07
*** bengates has joined #openstack-kolla08:08
*** rpittau|afk is now known as rpittau08:16
bersaceHow to open a mariadb shell ?08:21
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible upgrade jobs  https://review.opendev.org/70498408:23
*** gfidente has joined #openstack-kolla08:23
tony31@ Mark - when deploying the seed with ./dev/seed-deploy.sh it kept failing with a libvirt not found error. I managed to fix it by installing more packages and then rebooting `yum install qemu-kvm qemu-img virt-manager libvirt libvirt-python libvirt-client virt-install virt-viewer bridge-utils`08:23
openstackgerritAndreas Jaeger proposed openstack/kayobe master: Cleanup tox.ini deps  https://review.opendev.org/70517308:24
tony31not sure which package fixed it. The error was `/var/run/libvirt/libvirt-sock no such file or directory`. I dont get this error anymore, but this file still does not exist08:24
tony31bersace - what have you tried?08:25
bersacetony31: I'm using `docker run --rm -it --volumes-from mariadb --name dbrestore    --volume mariadb_backup:/backup --tmpfs /backup/restore    kolla/ubuntu-binary-mariadb:train    /bin/bash`08:28
bersaceThen: `mysql -h openstack.virt -u nova -D nova_cell0 -p`08:29
bersaceIt's an all-in-one deployment.08:29
tony31I am not that experienced with docker, but you would need to get a shell into the container running mariadb first. Then you can connect to the db with `mysql`08:30
tony31are you able to get a shell where mariadb is running?08:30
bersaceI got a shell, but nevermind, it looks like openstack solve the issue itself, odd :/08:31
*** iputra has quit IRC08:32
tony31once you got the shell, you can check mariadb is listening with `ss -an | grep LIST | grep 3306` <- that will show you the listening port for mariadb connections. Then you should just be able to do `mysql` and see what happens. I'm just unsure if any other steps are required for within docker08:36
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible upgrade jobs  https://review.opendev.org/70498408:40
*** tonythomas has joined #openstack-kolla08:43
*** jbadiapa has quit IRC08:43
*** goldyfruit has quit IRC08:46
*** luksky has joined #openstack-kolla08:54
*** jbadiapa has joined #openstack-kolla08:59
*** tony31 has quit IRC09:06
mgoddardmorning09:10
mgoddardyoctozepto, mnasiadka, hrw, osmanlicilegi: very easy centos8 review: https://review.opendev.org/#/c/704965/109:11
patchbotpatch 704965 - kolla - CentOS 8: Relax ironic iPXE EFI bootloader location - 1 patch set09:11
mgoddardhrw, osmanlicilegi: centos8 & ceph review: https://review.opendev.org/#/c/70433709:11
patchbotpatch 704337 - kolla - CentOS 8: Use upstream Ceph/master - 8 patch sets09:11
mgoddardwould be great to get centos8 deploy jobs merged before the weekend09:11
*** k_mouza has joined #openstack-kolla09:15
openstackgerritRadosław Piliszek proposed openstack/kolla master: nova-libvirt: add UEFI packages to support UEFI instances  https://review.opendev.org/68228009:19
openstackgerritRadosław Piliszek proposed openstack/kolla master: nova-libvirt: add UEFI packages to support UEFI instances  https://review.opendev.org/68228009:20
yoctozeptomgoddard, mnasiadka: https://review.opendev.org/682280 - uefi fix09:21
patchbotpatch 682280 - kolla - nova-libvirt: add UEFI packages to support UEFI in... - 4 patch sets09:21
mgoddardCentOS 8 images properly tagged on Dockerhub \o/ https://hub.docker.com/layers/kolla/centos-source-glance-base/master-centos8/images/sha256-3ad788ff10f8620d75f4adcc2f4288b436a30051f831244d4cf678d0f64c000209:27
mnasiadkayoctozepto: I think we fixed that in Ubuntu some cycles ago - funny it doesn't work in CentOS :)09:27
yoctozeptomgoddard: yay09:28
yoctozeptomnasiadka: maybe, did not check :D09:28
mnasiadkayoctozepto: I know, I used Ubuntu in a previous-previous-company :)09:28
mnasiadka(against company policies btw)09:29
*** mrunge has quit IRC09:29
*** reph has joined #openstack-kolla09:30
*** mrunge has joined #openstack-kolla09:33
*** mrunge_ has joined #openstack-kolla09:35
openstackgerritAndreas Jaeger proposed openstack/kayobe master: Fix error logging of dev/functions  https://review.opendev.org/70518009:37
*** mrunge has quit IRC09:39
*** mrunge_ has quit IRC09:41
*** mrunge has joined #openstack-kolla09:46
*** k_mouza has quit IRC09:47
*** k_mouza has joined #openstack-kolla09:47
*** hjensas has quit IRC09:54
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible upgrade jobs  https://review.opendev.org/70498409:55
*** goldyfruit has joined #openstack-kolla10:47
*** rpittau is now known as rpittau|bbl11:06
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible stable/train: CI: Add ceph-ansible related files to ignores in check-config.sh  https://review.opendev.org/70519411:25
mnasiadkamgoddard, yoctozepto: ^^ simple pre-req change to support master upgrade-ceph-ansible job11:27
yoctozeptomnasiadka: I thought I made stuff like this depend on currently checked version11:35
yoctozeptomnasiadka: I would go for removing it from checks on master, more messy than worth it now11:35
mnasiadkayoctozepto: no, checking out version to upgrade to is later than check-config.sh11:36
mnasiadkaso it uses code from previous_release11:36
yoctozeptomnasiadka: yes, I understand you11:37
yoctozeptostill, it is of little worth11:37
mnasiadkamaybe it is, but removing it in master and backporting is way more work - and I guess we would like to remove kolla-ceph ASAP, so nobody is really surprised...11:38
yoctozeptomnasiadka: fine, it's just CI11:52
*** cah_link has quit IRC11:56
*** skramaja has quit IRC11:56
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: OVN Support  https://review.opendev.org/69684111:57
*** cah_link has joined #openstack-kolla12:05
*** kplant has joined #openstack-kolla12:12
*** mixor has joined #openstack-kolla12:26
*** mixor has quit IRC12:30
*** rpittau|bbl is now known as rpittau13:08
*** k_mouza has quit IRC13:18
*** k_mouza has joined #openstack-kolla13:19
*** dougsz has joined #openstack-kolla13:31
*** dougsz has quit IRC13:39
*** luksky has quit IRC13:49
*** rlljorge has joined #openstack-kolla13:53
rlljorgeHi there !13:54
mgoddardhi rlljorge14:01
*** cah_link1 has joined #openstack-kolla14:04
*** cah_link has quit IRC14:04
*** cah_link1 is now known as cah_link14:04
*** reph has quit IRC14:16
mgoddardsome centos8 k-a patches: https://review.opendev.org/#/c/704629/ https://review.opendev.org/#/c/697690/16 https://review.opendev.org/#/c/704960/114:21
patchbotpatch 704629 - kolla-ansible - CI: Reduce unnecessary gate image builds - 1 patch set14:21
patchbotpatch 697690 - kolla-ansible - CentOS 8: Deploy CentOS 8 containers - 16 patch sets14:21
patchbotpatch 704960 - kolla-ansible - CI: build swift images for swift scenario - 1 patch set14:21
openstackgerritMerged openstack/kolla-ansible stable/train: CI: Add ceph-ansible related files to ignores in check-config.sh  https://review.opendev.org/70519414:23
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible upgrade jobs  https://review.opendev.org/70498414:26
*** TrevorV has joined #openstack-kolla14:28
*** cah_link has quit IRC15:03
*** luksky has joined #openstack-kolla15:11
rlljorgeGuys, deploy a new cluster using train release is a mistake ? I should use Stein for example ?15:19
kplanti'm running stable/train in production15:19
rlljorgekplant: do you use ceph storage?15:19
kplantyes i do15:19
yoctozeptorlljorge: train is nice and stable15:20
rlljorgekplant: do you have this issue https://bugs.launchpad.net/kolla-ansible/+bug/1860739 ?15:20
openstackLaunchpad bug 1860739 in kolla-ansible "Cinder backup failed for restore volume snapshot" [Undecided,New]15:20
yoctozeptoand works with ceph15:20
yoctozeptoah, this15:20
yoctozeptoI did not get testing it yet15:21
yoctozeptoany help appreciated15:21
rlljorgeyoctozepto: if possible can you try and share the result.15:22
kplanttrying to test right now15:22
yoctozeptorlljorge: yeah, I have this in my todo, overloaded15:23
yoctozeptokplant: thanks, appreciated15:23
kplantmy backup restoration was successful15:23
kplantlet me re-read the bug15:23
rlljorgekplant: thanks15:23
*** admin0 has joined #openstack-kolla15:25
kplantyeah i followed the spirit of the bug and it worked fine. let me follow the order defined exactly15:25
kplant'Status15:27
kplantError Restoring'15:27
kplantmight be something here15:28
rlljorgeKplant: in your test ?15:28
kplantyeah15:28
kplantif i snapshot the volume, it works fine.. if i snapshot the instance like stated in the bug, it breaks15:28
rlljorgekplant: I will try now with snapshot volume too15:29
*** pcaruana has quit IRC15:30
yoctozeptokplant, rlljorge: then it might be the case it's nova+cinder problem15:34
kplantyeah, that makes sense15:37
yoctozeptokplant, rlljorge: please update the report with details, I'll mark it appropriately then15:37
kplantadded15:39
yoctozeptomnasiadka: https://review.opendev.org/682280 :D15:39
patchbotpatch 682280 - kolla - nova-libvirt: add UEFI packages to support UEFI in... - 4 patch sets15:39
yoctozeptokplant: thx15:40
yoctozeptokplant: also, if you could click on tthat "affects me" it boosts it a little in nova/cinder ranks :-)15:41
kplantfair point15:42
yoctozeptokplant, rlljorge: either party does something wrong there15:43
yoctozeptonothing for kolla to fix (unfortunately!)15:43
kplantyep - totally understand15:43
yoctozeptorlljorge: might be best bet to attack #openstack-cinder and/or #openstack-nova now15:43
rlljorgeyoctozepto kplant i am try cinder irc without sucess ... I will try nova now !!!!15:44
rlljorgekplant yoctozepto thanks for confirmation the bug15:45
rlljorgeI am not alone15:45
*** luksky has quit IRC15:45
cosmicsoundgood day15:46
yoctozeptorlljorge: yes, that's reassuring... I think15:46
cosmicsoundTASK [keystone : Initialise fernet key authentication] *******************************************************************************************************************************************************15:46
cosmicsoundFAILED - RETRYING: Initialise fernet key authentication (10 retries left).15:46
cosmicsoundfatal: [sc]: FAILED! => {"msg": "The conditional check 'fernet_create.stdout.split()[2] == 'SUCCESS' or fernet_create.stdout.find('Key repository is already initialized') != -1' failed. The error was: error while evaluating conditional (fernet_create.stdout.split()[2] == 'SUCCESS' or fernet_create.stdout.find('Key repository is already15:46
cosmicsoundinitialized') != -1): list object has no element 2"}15:46
yoctozeptoit happens from time to time15:46
cosmicsoundgot this when run a simple deploy once kolla upgraded15:46
yoctozeptoyeah, it's random15:46
yoctozeptomnasiadka working on the fix (for mnasiadka: fernet stuff ;-) )15:46
mnasiadkaOh boy, fernet again15:47
mnasiadkaI’ll force myself to work on it15:47
*** mixor has joined #openstack-kolla15:49
*** pcaruana has joined #openstack-kolla15:49
cosmicsounddoes kolla 9.1.0 train has something extra one needs to run/15:49
*** luksky has joined #openstack-kolla15:49
cosmicsoundwas thinking maybe i miss some gen-passwd or such15:50
cosmicsoundgen certs15:50
cosmicsoundif any new implemented15:50
yoctozeptocosmicsound: no, it's random15:50
yoctozeptodestroy and try again probably works15:51
cosmicsoundok15:53
cosmicsoundhow is a server destroyed if it was previously removed by force/decomissioned15:54
yoctozeptomnasiadka: https://review.opendev.org/682280 quickie15:56
patchbotpatch 682280 - kolla - nova-libvirt: add UEFI packages to support UEFI in... - 4 patch sets15:56
yoctozeptocosmicsound: server? hammer comes in handy :-)15:57
mnasiadkayoctozepto: I would rather use a flex saw - more fun15:58
yoctozeptoo to to :D15:58
mnasiadkaok, weekend time :)15:59
cosmicsound:)15:59
cosmicsoundTASK [destroy : Destroying all Kolla containers and volumes] *************************************************************************************************************************************************16:01
cosmicsoundfatal: [sc]: FAILED! => {"changed": true, "msg": "non-zero return code", "rc": 1, "stderr": "Shared connection to sc closed.\r\n", "stderr_lines": ["Shared connection to sc closed."], "stdout": "Some qemu processes were detected.\r\nDocker will not be able to stop the nova_libvirt container with those running.\r\nPlease clean them up before16:01
cosmicsoundrerunning this script.\r\n", "stdout_lines": ["Some qemu processes were detected.", "Docker will not be able to stop the nova_libvirt container with those running.", "Please clean them up before rerunning this script."]}16:01
cosmicsoundIf this error present16:01
cosmicsoundSHould there be a force to it16:01
cosmicsoundAnd just wondering why isnt this able to force close all needed?16:01
cosmicsoundSince one already give his agreement16:01
*** rpittau is now known as rpittau|afk16:05
cosmicsoundworst its i runned all16:11
cosmicsoundprune and nothin helps16:11
yoctozeptocosmicsound: best just reboot16:12
yoctozeptoor try killall qemu16:13
yoctozeptoor killall qemu-kvm16:13
yoctozepto(on that target)16:13
cosmicsounddoing a reboot16:13
cosmicsoundgot one more month to proove kolla is worthy for our cloud project16:14
cosmicsound:D16:14
cosmicsoundweird16:16
cosmicsoundreboot didnt helped16:16
cosmicsound(venv3) root@ss:~/kolla-ansible_setup# killall qemu16:17
cosmicsoundqemu: no process found16:17
cosmicsound(venv3) root@ss:~/kolla-ansible_setup# killall qemu-kvm16:17
cosmicsoundqemu-kvm: no process found16:17
cosmicsoundweird00016:17
cosmicsoundoww second16:18
cosmicsoundwas second s16:20
cosmicsoundhost16:20
cosmicsoundhave a good weekend folks16:24
mgoddardcosmicsound: what is blocking approval currently?16:31
cosmicsoundmgoddard , seems all good16:31
mgoddardgreat :)16:32
cosmicsoundredeploying as we speak16:32
*** jopdorp has joined #openstack-kolla16:33
jopdorphi!16:33
jopdorpanyone know about this error for kolla AIO?16:33
jopdorpTASK [mariadb : Copying over galera.cnf]16:33
jopdorpAn exception occurred during task execution. To see the full traceback, use -vvv. The error was: TypeError: unicode argument expected, got 'str'16:34
*** negronjl has quit IRC16:36
yoctozeptomnasiadka: thx16:36
yoctozeptojopdorp: odd error, which versions of kolla-ansible, ansible and python (also on target host)?16:36
jopdorpI'll have a look16:37
jopdorpI'm on kolla ansible master branch16:37
jopdorpPython 2.7.1716:38
jopdorpansible 2.9.416:38
jopdorpI'll try stable/train16:39
jopdorpI'm on kolla ansible master branch from git16:40
jopdorphow do i see the version of kolla-ansible itself?16:40
jopdorpi think i'm on 1.25.816:41
jopdorpno sorry16:41
*** negronjl has joined #openstack-kolla16:41
jopdorpI'9.1.0.dev23616:41
jopdorpRequirement already satisfied: kolla-ansible in /home/jopdorp/kolla-venv/lib/python2.7/site-packages (9.1.0.dev236)16:41
*** TrevorV has quit IRC16:42
*** bengates has quit IRC16:43
openstackgerritMark Goddard proposed openstack/kayobe master: WIP: Use python3 for local kolla-ansible execution  https://review.opendev.org/70500016:44
*** luksky has quit IRC16:44
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Fix qemu loading of ceph.conf (permission error)  https://review.opendev.org/68074016:53
mgoddardosmanlicilegi: seeing ansible version check issue in kayobe: https://3a40dd74807b16b826bd-be2647135e35587b155fc0ff22bf704c.ssl.cf5.rackcdn.com/705000/5/check/kayobe-overcloud-centos/189efd6/primary/ansible/overcloud-deploy17:12
mgoddardERROR: Ansible version should be between 2.8 and 2.9. Current version is  which is not supported.17:12
mgoddardno ANSIBLE_DEBUG17:13
mgoddardansible 2.8.817:13
mgoddardcould be an issue on our end17:14
mgoddardERROR! Unexpected Exception, this is probably a bug: Failed to detect selinux python bindings at ['/usr/local/lib64/python3.6/site-packages', '/usr/local/lib/python3.6/site-packages', '/usr/lib64/python3.6/site-packages', '/usr/lib/python3.6/site-packages']17:14
cosmicsoundwhat is weird after clean deployment and running init-runonce i get "This tool should only be run once per deployment."17:14
openstackgerritwes hayutin proposed openstack/kolla master: DNM: Add dnf modules for rhel8/py3 containers  https://review.opendev.org/67118317:16
openstackgerritwes hayutin proposed openstack/kolla master: DNM: Add rhel8 support for early CentOS8 testing  https://review.opendev.org/66947117:25
*** jopdorp has quit IRC17:25
*** jopdorp has joined #openstack-kolla17:28
jopdorpthumbs up for stable/train17:28
jopdorpi got it to work17:28
jopdorpby the way, I read that the ceph role is deprecated17:28
jopdorpdoes anyone know why that is and what replaces it?17:28
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Improve Ansible version check error handling  https://review.opendev.org/70526117:30
*** evrardjp has quit IRC17:33
*** evrardjp has joined #openstack-kolla17:34
openstackgerritMerged openstack/kolla master: nova-libvirt: add UEFI packages to support UEFI instances  https://review.opendev.org/68228017:57
cosmicsoundjopdorp , ceph-ansible might be used instead or any other ceph deployment/manager17:57
jopdorpokay, thanks!17:59
*** jopdorp has quit IRC18:23
*** rlljorge has quit IRC18:27
osmanlicilegimgoddard: I was out all day. I'm gonna check the logs now.18:33
yoctozeptoyeah, running master for production is asking for trouble :-)18:40
*** luksky has joined #openstack-kolla18:42
*** tonythomas has quit IRC18:50
*** gfidente has quit IRC19:01
*** igordc has joined #openstack-kolla19:03
*** TrevorV has joined #openstack-kolla19:05
openstackgerritMerged openstack/kolla master: CentOS 8: Use upstream Ceph/master  https://review.opendev.org/70433719:26
openstackgerritRadosław Piliszek proposed openstack/kolla stable/train: nova-libvirt: add UEFI packages to support UEFI instances  https://review.opendev.org/70528719:29
*** pcaruana has quit IRC19:30
dking_desktopI'm working on setting up baremetal nodes. They seem to be created, and I know that they can be powered on/off through openstack. I filled in the server properties manually. However, if I try to create a server with one, it says that "No valid host was found". I see that "openstack hypervisor show UUID", it shows some details about the server, but the properties, such as local_gb and memory_mb, are still showi19:34
dking_desktopeason why nova believe that there is no valid host?19:34
noxoidno valid host i believe is returned by the scheduler service. you may have a misconfiguration between nova and ironic where nova is asking for hardware that doesnt exist in ironic19:36
dking_desktopThanks.19:36
yoctozeptoneed checking logs for nova, ironic and placement services19:37
yoctozeptosomewhere it went wrong :-)19:37
*** kplant has quit IRC19:54
*** k_mouza has quit IRC20:12
*** k_mouza has joined #openstack-kolla20:22
*** k_mouza has quit IRC20:27
*** k_mouza has joined #openstack-kolla20:59
*** k_mouza has quit IRC21:03
*** Trevor_V has joined #openstack-kolla21:20
*** TrevorV has quit IRC21:23
*** Trevor_V has quit IRC21:29
*** KeithMnemonic has quit IRC22:06
*** mixor has quit IRC22:06
openstackgerritMerged openstack/kolla-ansible master: CI: Reduce unnecessary gate image builds  https://review.opendev.org/70462922:11
*** k_mouza has joined #openstack-kolla23:19
*** igordc has quit IRC23:52
« Thursday, 2020-01-30 Index Saturday, 2020-02-01 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!