Saturday, 2020-02-08

*** hongbin has joined #openstack-kolla01:10
*** hongbin has quit IRC01:28
*** gfidente|afk has quit IRC01:41
*** hongbin has joined #openstack-kolla01:43
*** zhanglong has joined #openstack-kolla02:43
*** jcmdln has quit IRC02:57
*** hongbin has quit IRC04:50
*** evrardjp has quit IRC05:34
*** evrardjp has joined #openstack-kolla05:34
*** jcmdln has joined #openstack-kolla05:49
*** bjolo has quit IRC06:46
*** k_mouza has joined #openstack-kolla06:53
*** k_mouza has quit IRC06:58
*** pbing19 has joined #openstack-kolla07:57
openstackgerritDincer Celik proposed openstack/kolla stable/train: [train] Bump versions  https://review.opendev.org/70665908:33
openstackgerritDincer Celik proposed openstack/kolla stable/stein: [stein] Bump versions  https://review.opendev.org/70666008:35
openstackgerritDincer Celik proposed openstack/kolla stable/rocky: [rocky] Bump versions  https://review.opendev.org/70666108:40
*** zhanglong has quit IRC08:42
*** zhanglong has joined #openstack-kolla09:29
*** pbing19 has quit IRC09:38
*** pbing19 has joined #openstack-kolla09:39
*** zhanglong has quit IRC09:57
*** zhanglong has joined #openstack-kolla09:58
*** zhanglong has quit IRC10:19
*** jopdorp has joined #openstack-kolla10:20
jopdorphi all10:30
jopdorpI know it's the weekend, but I've been facing this issue since yesterday and don't know what to do with it10:31
jopdorpthis step fails for a multinode deployment with ubuntu as base os rabbitmq : Waiting for rabbitmq to start on first node10:31
jopdorpI'm using private key for ssh10:32
jopdorpI saw that there used to be some problems with the rabbitmq container constantly restarting10:32
jopdorpcould that still be an issue?10:32
jopdorpyeah, it's restarting10:35
jopdorpit looks like this issue, https://bugs.launchpad.net/kolla-ansible/+bug/183769910:39
openstackLaunchpad bug 1837699 in kolla-ansible train "[ubuntu host] Handle duplicated hostnames in /etc/hosts file (127.0.1.1 problem)" [High,Fix released] - Assigned to Dincer Celik (osmanlicilegi)10:39
jopdorpbut i think it should have been fixed?10:40
jopdorphH10:40
jopdorphaha10:40
jopdorpyeah so exactly that one10:40
jopdorpshould i remove the entries in etc hosts in all my nodes or just in control?10:41
jopdorpi removed them just from control, think it's enough, thanks!10:50
*** jopdorp has quit IRC11:13
*** vesper11 has joined #openstack-kolla12:08
*** vesper has quit IRC12:08
*** mloza has quit IRC12:52
*** negronjl has quit IRC12:54
*** negronjl has joined #openstack-kolla12:55
*** admin0 has joined #openstack-kolla13:44
admin0quick question .. how big ( in terms of cpu/ram/disk ) resource would be required to test kolla all-in-one with all services enabled ..  where the goal is to test 1 service at a time and not run 100s of it13:48
admin0just to say "ok.. it works"13:48
admin0trying to run kolla aio  trein .. got error on: TASK [mariadb : Copying over galera.cnf]  The error was: TypeError: unicode argument expected, got 'str' .. i did not even touch this file ..13:56
admin0is this a fault in our QA :) ?14:00
*** pbing19 has quit IRC14:35
yoctozeptoadmin0: this sounds like using python2 on deployment machine and python3 on target (or vice versa, never remember)15:04
yoctozepto;-)15:04
r3ap3ryoctozepto: so does that mean at this point in time, we can drop python2 and strictly use python3 for Kolla deployments?15:05
yoctozeptor3ap3r: ansible works reliably with the same python version on both sides15:06
r3ap3ryoctozepto: Ok cool. I will try a python3 deployment sometime in the next couple of weeks and see how my mileage goes. Thanks.15:07
yoctozeptor3ap3r: feature-wise it ends up the same because containers are separate15:09
*** vesper11 has quit IRC15:09
*** vesper11 has joined #openstack-kolla15:11
r3ap3ryoctozepto: +115:12
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Add --clean to kolla-mergepwd  https://review.opendev.org/70666815:28
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Docs: fix cells upgrade guide  https://review.opendev.org/70666915:30
openstackgerritDincer Celik proposed openstack/kolla stable/train: [train] Bump versions  https://review.opendev.org/70665915:50
openstackgerritDincer Celik proposed openstack/kolla stable/train: [train] Bump versions  https://review.opendev.org/70665915:54
openstackgerritDincer Celik proposed openstack/kolla stable/stein: [stein] Bump versions  https://review.opendev.org/70666015:56
openstackgerritDincer Celik proposed openstack/kolla stable/rocky: [rocky] Bump versions  https://review.opendev.org/70666115:58
admin0yoctozepto, its a all in one to test first .. i followed steps on https://docs.openstack.org/kolla-ansible/latest/user/quickstart.html16:01
admin0the document does not say anywhere regarding python versions16:02
yoctozeptoadmin0: well, it's hard to get wrong python versions in an aio - did you use virtualenv with custom python?16:08
yoctozeptoi.e. it's a different python than if you now ssh'ed back to the machine16:08
admin0all i did was apt-get install python-virtualenv on ubuntu 18.04 and then source it16:23
admin0and proceeded with the rest of the copy/paste commands16:23
admin0let me destroy this box and do it again one more time .. will document what i did16:24
*** pbing19 has joined #openstack-kolla16:44
*** evrardjp has quit IRC17:34
*** evrardjp has joined #openstack-kolla17:34
*** devfaz has quit IRC17:49
*** devfaz has joined #openstack-kolla17:49
*** pbing19 has quit IRC17:54
*** born2bake has joined #openstack-kolla17:56
admin0yoctozepto, this time different output: https://gist.github.com/a1git/788efba8948d5695e19b39c51442e7e2  --- please suggest where I missed following the documentation18:03
yoctozeptoadmin0: the bad part is where you did the git clone and left it at the default branch - you got master which is for ussuri18:10
yoctozeptothis is also the reason for step duplication18:10
yoctozeptothey are duplicated because you did both the "deployment/eval" part and "devel" part18:10
admin0so #1. we need to add git checkout (correct branch) in the documentation ?18:11
admin0the deployment/eval an devel part, i did only in the gen-passwords18:11
yoctozeptoI think we need to rewrite it so that nobody thinks these are the next steps18:11
yoctozeptowell, first you installed k-a from pypi18:12
yoctozeptoand then from git18:12
yoctozeptoso you rewrote train stable release with ussuri master18:12
yoctozeptoI agree that docs are not the best, I'll see to having them more clear18:13
admin0i see it18:13
admin0i will correct this, move forward and document where i get stuck again18:14
*** pbing19 has joined #openstack-kolla18:14
admin0i still need to be root for the all-in-one which is on localhost when doing the precheck, bootstrap etc right ?18:15
dking_desktopI notice that when k-a setups up ovs, all of the internal ports are down. I can bring them up by manually creating the ifcfg-* files with IP and netmask added and then ifup, but is there some place in the configuration where that could be done?18:16
yoctozeptoadmin0: thanks, that's helpful the way you give feedback18:16
yoctozeptoadmin0: or have passwordless sudo18:16
yoctozeptoadmin0: this is general ansible knowledge but it might be wise to repeat it here, indeed18:16
yoctozeptodking_desktop: why do you want them to be up?18:16
yoctozeptodking_desktop: these are nothing to worry about18:16
admin0i started the same way when starting with osa also (many years ago) .. making sure the documentation is easy to follow for everyone to get the basics up and running18:17
dking_desktopyoctozepto: I'm still working on getting Ironic to work with my untagged networks. Maybe I'm going about it the wrong way, though.18:17
yoctozeptodking_desktop: http://paste.openstack.org/show/789321/ <- with these STATES all networking works fine18:18
yoctozeptoso you've got DOWN on 4 common18:19
yoctozeptoand UNKNOWN on specific18:19
yoctozeptoand life is good18:19
dking_desktopI have two NICs which for redundancy need to be using a bond (bond0). All of the rest of my connections use VLANs, which is working fine. I set bond0 as neutron_external_interface in globals.yml, but saw that this overwrites any IP and I can't use that interface for ironic_dnsmasq. However, if I manually bring br-ex up, it seemed to work.18:20
yoctozeptoadmin0: yeah, the thing with docs is that people perceive them differently and it's no easy task to get them really right18:20
dking_desktop...or maybe I should just be making a separate port?18:20
admin0yoctozepto, but we can also separate this to an all-in-one ..  like its done in ansible: https://docs.openstack.org/openstack-ansible/latest/user/aio/quickstart.html18:21
admin0there is just one set of instructions to follow .. avoiding confusion18:22
admin0yoctozepto, maybe i will contribute and create this document and submit a request18:22
yoctozeptoadmin0: yeah, I was thinking about separate aio/multinode and separate eval+deploy/dev18:22
yoctozeptoadmin0: plus mentioning definite release, not any18:23
yoctozeptodking_desktop: bond0 is fine; the thing with neutron_external_interface it gets captured as bridge for physnet118:23
yoctozeptoso yeah, it loses any ip addressing18:23
yoctozeptoironic_dnsmasq is for provisioning18:24
yoctozeptoafair ;-)18:24
admin0how do i contribute to the docs ? is it the same way as in openstack-ansible ?18:24
yoctozeptoso you probably want to separate a tagged interface for it and use it18:24
yoctozeptobond0.xxx18:24
yoctozeptoe.g. bond0.12318:24
dking_desktopOkay. I'm sure I'm asking the wrong question, then. What should I use for ironic_dnsmasq_interface?18:24
yoctozeptowell, that xxx was most unfortunate as it is probably some pr0n site, oh well18:25
yoctozepto;D18:25
dking_desktopWell, the problem there is that my servers can't handle VLANs for PXE, and the servers need to be on trunk ports.18:25
yoctozeptoadmin0: yeah, the gerrit review18:25
admin0ok18:25
yoctozeptodking_desktop: yeah, you don't need to tag these for pxe18:26
dking_desktoplol. Maybe that's IEEE 802.1xxx?18:26
yoctozeptoit could be untagged on them18:26
yoctozeptobut tagged on port to neutron18:26
yoctozeptolemme find usage of ironic_dnsmasq in the code18:26
dking_desktopIt looks like ironic_dnsmasq_interface sets interface in dnsmasq.conf inside of the ironic_dnsmasq container.18:27
yoctozeptooh, it's for inspector18:27
yoctozeptoyeah18:27
yoctozeptoso it's for initial enrollment18:28
dking_desktopCorrect. And once they are up, they will need to be able to communicate through multiple VLANs,  so they are trunked.18:30
yoctozeptoyeah, trunked but the native/default/whatever should be this one18:31
dking_desktopYep. I've tried about every other combination I could think of. Our switches have VLAN 1 set for the native VLAN, so I tried bond0.1. I see that sending from that becomes untagged, which is great, but if I set dnsmasq's interface to bond0.1, it still doesn't see the packets.18:32
yoctozeptodking_desktop: they need to be tagged on that switch's port to that server with dnsmasq18:33
dking_desktopThe only way I've been able to get the controller to accept untagged frames so far has been to use an internal port.18:33
yoctozeptowhat is an internal port?18:34
dking_desktopI meant the port type internal through ovs, such as br-ex. Really, that's the only one I tried so far, but that worked fine.18:34
yoctozeptoah, you mean you hacked on kolla ovs internals, not nice! :D18:36
yoctozeptowhat you need to do is configure the switch port to dnsmasq to tag that vlan 118:37
dking_desktoplol. Maybe not, but it wasn't being too nice to me. And since the documentation hasn't been helping me, I thought I would try to dig until I could find something that worked, and then work backwards from that.18:37
yoctozeptobecause otherwise you can't separate that traffic18:37
yoctozeptodking_desktop: I was joking :-) but that does not change the fact that it's far from a supported solution18:37
dking_desktopTrue. I just haven't figured out a supported solution that works. The documentation looks to me like it assumes that I have a spare NIC. Would you know where I could find the proper documentation?18:40
admin0yoctozepto, https://gist.github.com/a1git/9fc701e4d2aaec41f46499cefe2ede78 ... looks like i am still missing something18:43
dking_desktopI'm hoping to avoid too many custom changes to the switch port. I might try that, but it would be Monday before I could give it a shot.18:44
yoctozeptodking_desktop: yeah, spare, but .123 is also a spare18:46
yoctozeptothe issue really is linux does not differentiate between "any" and "untagged"18:46
yoctozeptoso it's best done on switch side18:46
dking_desktopIt's a spare VLAN, though. But perhaps other folks aren't attempting to provision baremetal nodes that use trunked ports.18:47
yoctozeptoadmin0: very weird18:49
yoctozeptodking_desktop: trunk is a must18:49
admin0yoctozepto, on my 8 months old attempt, documented here: https://www.openstackfaq.com/openstack-kolla-all-in-one/ i see a line where it says pip install kolla-ansible docker ( when not using virualenv)18:51
yoctozeptoadmin0: could you sahre the logs from bootstrap?18:51
admin0sure18:51
admin0let me run it again18:51
yoctozeptoadmin0: yeah, it does install it in bootstrap18:51
yoctozeptono idea why it did not now ;p18:52
admin0it now installed docker, on the 2nd run18:53
admin0first run: https://gist.github.com/a1git/06ed6534203e41c0af2456991a2b126b   second run: https://gist.github.com/a1git/0d0916c52d16423767d67264fa62f4a118:55
admin0 i did not used -v in first, used -v in 2nd18:55
admin0running prechecks now :)18:55
admin0which failed again in the same point .. failed in checking docker sdk version18:56
admin0ImportError: No module named docker18:56
admin0i think inside the virtualenv, its needed to do pip install docker18:56
admin0which is in my old documentation, but unable to find in new one18:56
admin0it returned with requirement already satisfied18:57
*** vmixor has joined #openstack-kolla18:59
dking_desktopyoctozepto: Thanks for the tips. I'm going to think about that and maybe re-visit the switch configuration on Monday. In the meantime, I might push ahead with what I've got manually. Thank you.19:01
yoctozeptodking_desktop: yw, good luck!19:01
admin0i am going to retry wtihout using virtualenv19:01
yoctozeptoadmin0: hmm, odd, it installed it in the venv19:02
yoctozeptoadmin0: I wonder if ansible started doing us a "favor" there19:02
admin0well, i am still nowhere to start :) and test19:03
yoctozeptoadmin0: https://docs.openstack.org/kolla-ansible/latest/reference/deployment-and-bootstrapping/bootstrap-servers.html#initial-bootstrap-considerations19:04
yoctozeptowell, this mentions it19:04
yoctozeptoso quickstart is bad at suggesting venv for beginners :-)19:05
*** pbing19 has quit IRC19:07
yoctozeptowell, no, it does not make sense still19:14
yoctozeptoit explicitly says it's not using virtualenv there19:14
yoctozeptowhy did it pick up this pip...19:14
*** vmixor has quit IRC19:23
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: DNM: Test venv  https://review.opendev.org/70667319:25
yoctozeptoadmin0: I've set up some poc in CI ^19:25
yoctozeptoI wonder what happens in there19:25
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: DNM: Test venv  https://review.opendev.org/70667319:36
admin0"so quickstart is bad at suggesting venv for beginners :-)" -- yes .. hours lost19:48
admin0trying to follow the docs19:48
yoctozeptoadmin0: well, the poc actually works so I'm still unsure what happens in your case...20:04
yoctozepto"/usr/local/bin/pip2",20:12
yoctozeptodespite using venv20:12
yoctozeptohmm20:13
*** goldyfruit has quit IRC20:17
*** goldyfruit has joined #openstack-kolla20:18
*** goldyfruit has quit IRC20:18
*** goldyfruit has joined #openstack-kolla20:19
*** goldyfruit has quit IRC20:20
*** goldyfruit has joined #openstack-kolla20:20
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: DNM: Test venv  https://review.opendev.org/70667320:29
*** negronjl has quit IRC20:33
*** negronjl has joined #openstack-kolla20:37
*** goldyfruit has quit IRC21:07
*** bjolo has joined #openstack-kolla21:17
yoctozeptoadmin0: oddly all are passing21:38
yoctozeptoadmin0: I thought it could be due to localhost but it passed anyways21:38
admin0i will redo a fresh  ubuntu install tomorrow and retry21:39
yoctozeptoadmin0: and I'll do a poc for train - could be ussuri is fine but train is not - since you move farther with ussuri previously21:43
yoctozeptogood night21:44
admin0good night .. thanks .. will test tomororw22:10
*** goldyfruit has joined #openstack-kolla23:46

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!