Friday, 2019-06-21

*** weifan has quit IRC00:01
openstackgerritMerged openstack/openstack-ansible-os_heat stable/rocky: Updated from OpenStack Ansible Tests  https://review.opendev.org/66668200:08
openstackgerritMerged openstack/openstack-ansible-os_neutron stable/stein: Updated from OpenStack Ansible Tests  https://review.opendev.org/66671300:11
*** markvoelker has joined #openstack-ansible00:40
*** markvoelker has quit IRC00:59
*** schwicht has quit IRC01:07
*** sreejithp has quit IRC01:21
snadgei was all happy about finally getting an OA stein AIO to deploy.. rebooted it, and my primary network interface is missing :(01:47
snadgewhy would ens160 just disappear01:47
snadgeit got renamed to eth0 .. 🤔01:52
*** markvoelker has joined #openstack-ansible01:56
snadgewhat mechanism changes the hostname from whatever it was before i ran the playbooks, to aio?01:57
*** markvoelker has quit IRC02:01
*** markvoelker has joined #openstack-ansible02:57
*** markvoelker has quit IRC03:02
openstackgerritLogan V proposed openstack/openstack-ansible-os_nova master: Install netcat on compute hosts  https://review.opendev.org/66675903:17
*** markvoelker has joined #openstack-ansible03:58
*** joshualyle has joined #openstack-ansible04:02
*** markvoelker has quit IRC04:02
*** udesale has joined #openstack-ansible04:05
*** dave-mccowan has quit IRC04:09
*** pcaruana has joined #openstack-ansible04:45
*** markvoelker has joined #openstack-ansible04:59
*** markvoelker has quit IRC05:04
*** kopecmartin|off is now known as kopecmartin05:12
*** jbadiapa has quit IRC05:13
ChosSimbaOneMorning. Just to clear things up, external_lb_vip_address does not have to be an url, IP should work just aswell, right? in openstack_user_config.yml05:25
*** markvoelker has joined #openstack-ansible06:00
*** markvoelker has quit IRC06:04
*** ivve has joined #openstack-ansible06:15
jrossersnadge: either ip or fqdn is fine, so long as there is a proper dns entry for the fqdn06:31
jrosserOops... ChosSimbaOne ^06:31
jrossersnadge: using netplan?06:31
snadgeyes.. i suspect that's half the drama06:32
snadgethe rocky release doesn't do it.. which is odd.. just stein.. network interface changes from ens160 to eth0 after deploying06:32
snadgeand the hostname reverts from "aio1" to "ubuntu-stack" (which is what it was before deploying) after rebooting06:32
snadgewhich causes some issues with neutron .. ie.. it detects multiple hypervisors, one with the new name "aio1" and one with the original hostname06:33
snadgeits not a biggy and its probably easily fixed in both cases06:34
jrosserWell, you’ve got a mix of stuff going on as the AIO uses systemd-networkd to configure the host networking, so that may well be fighting with netplan?06:35
*** udesale has quit IRC06:36
jrosserAnd also an AIO is sort of an emphemeral rest thing, on a production deploy you’d have to configure the host and it’s networking yourself, so may be better to put effort into getting that nailed instead06:37
jrosser*test thing06:37
*** udesale has joined #openstack-ansible06:37
snadgeyeah I guess i should stop playing around with AIOs .. but its a convenient sandbox to play with and learn things06:39
jrosserYou can take the resulting config from /etc/openstack_deploy, stick it in a git repo and use it as the starting point for a prod deploy06:43
*** luksky has joined #openstack-ansible06:43
jrosserThe thing you do get on an AIO is everything is natted behind the host ip which is just what you want for a test06:44
jrosserBut not prod like at all06:44
*** jbadiapa has joined #openstack-ansible07:00
*** markvoelker has joined #openstack-ansible07:01
*** rgogunskiy has joined #openstack-ansible07:12
*** tosky has joined #openstack-ansible07:21
*** markvoelker has quit IRC07:28
ChosSimbaOnejrosser: Okay, still bangning my head against the wall with the haproxy and some of the containers on other servers... Tried setting up an additional interface for testing, but that did not change anything. Will have to dig deeper. Is it possible to disable the keepalived on haproxy, so the ip don't "float" from infrastructure host to infrastructure host?07:40
*** cshen has joined #openstack-ansible07:43
*** rpittau|afk is now known as rpittau07:47
CeeMacChosSimbaOne: the IP shouldn't 'float' unless the box that has the VIP is restarted.07:52
CeeMacChosSimbaOne: you can set an override in user_variables.yml haproxy_use_keepalived: False.  Then you'd want to change your openstack_user_config.yml to just have a single haproxy_host I guess07:53
CeeMacdid you re-run the haproxy playbook after setting up the new interface ChosSimbaOne ?07:53
openstackgerritMerged openstack/openstack-ansible-os_trove stable/rocky: Updated from OpenStack Ansible Tests  https://review.opendev.org/66669607:54
ChosSimbaOneCeeMac: Hi. Thanks, well just realised that the problem is i cannot access the ext. lb ip which is secondary ip on the 1st infrastructure host. So disabling the keepalived will not solve this issue i guess, wasn't thinking. yes rerun both setup-host and setup-infrastructure.08:02
CeeMacChosSimbaOne: yeah, that wouldn't help. Looks like you have some network issues to resolve :)08:08
*** udesale has quit IRC08:11
*** dirk has joined #openstack-ansible08:11
ChosSimbaOneyes, I think you are right :-S08:11
*** udesale has joined #openstack-ansible08:12
CeeMacChosSimbaOne: did you not add the second interface to use for the external vip?08:16
openstackgerritMerged openstack/openstack-ansible-os_nova stable/rocky: Updated from OpenStack Ansible Tests  https://review.opendev.org/66668908:17
ChosSimbaOneyes i added an extra nic and configured that i user_variable as the external interface.08:18
*** gkadam has joined #openstack-ansible08:19
CeeMacand you can ping the default interface IP of the new nic from each of the infrastructure nodes?08:20
ChosSimbaOneI did not set any default ip on the new nic, I reconned that would be handled by the playbooks. btw should the lb ip be part of the used ips list?08:30
openstackgerritMerged openstack/openstack-ansible-os_keystone stable/stein: Updated from OpenStack Ansible Tests  https://review.opendev.org/66670808:32
*** cshen has quit IRC08:35
jrosserChosSimbaOne: for a custom network setup you need to make all the interface config yourself including assigning ip to the interfaces/bridges as needed08:36
jrosserThe “interface” to OSA is br-mgmt, br-storage, and so on08:38
CeeMacChosSimbaOne: assuming you're using a spereate subnet, the 'external' network wouldn't be specified as a container network, so you wouldn't need to exlcude it from the the used ips list.08:39
CeeMacs/spereate/separate08:40
*** cshen has joined #openstack-ansible08:40
openstackgerritMerged openstack/openstack-ansible-os_tacker stable/stein: Updated from OpenStack Ansible Tests  https://review.opendev.org/66671708:41
openstackgerritMerged openstack/openstack-ansible-os_sahara stable/stein: Updated from OpenStack Ansible Tests  https://review.opendev.org/66671608:42
*** cshen has quit IRC08:45
jrossernoonedeadpunk: for Zun I think we need to remove galera_client, I will look next week unless you get to it first08:48
ChosSimbaOnejrosser and CeeMac Okay,. Background is that i am running it all on the same vlan 172.19.20.0/24 so br-storage br-vlan br-mgmt all has addresses on this network. The nodes in my setup has address ont th bridge according to the ip assignments table on https://docs.openstack.org/openstack-ansible/latest/user/ceph/full-deploy.html Therefore i gathered that the lb ips was handled by the playbook and all i needed to do was to configure08:52
ChosSimbaOnethe address and interface in openstack_user_config and user_variable. Is that i wrong assumption? Should i before i run the playbook configure the ip as a secondary ip on all br-mgmt bridges on the control host, and then add the lb ip as a used ip in he openstack_user_confg?08:52
ChosSimbaOneexample of network/interfaces http://paste.openstack.org/show/753249/08:53
jrosserKeepalived handles the secondary ip, you don’t need to do that08:54
*** cshen has joined #openstack-ansible09:01
CeeMacChosSimbaOne: i'm not convinced having all of the bridges on the same vlan is going to work09:01
ChosSimbaOnejrosser: Super, that was what i thought :-)09:02
CeeMacnot that I'm saying it won't having never tried it, but it 'feels' wrong09:02
CeeMacits certainly going to add a layer of difficulty to your troubleshooting09:03
*** joshualyle has quit IRC09:03
ChosSimbaOneCeeMac: Okay, well the reason behind it is purely for testing. Id did do a run of the test example https://docs.openstack.org/openstack-ansible/latest/user/test/example.html09:04
ChosSimbaOneIf iirc i had the br-vxlan on a 10.14.x.x range as this is not routed right? With regards to br-storage and br-mgmt the test server and ceph OSD is on the same network, so that is way that is necessary.09:05
CeeMacits a bit of a catch-22 i guess, i understand keeping things simple for testing, but sometimes if its too simple it can cause other problems :)09:05
CeeMactechnically, none of the networks are routed, except the public one :)09:06
*** tosky__ has joined #openstack-ansible09:06
ChosSimbaOneYeah, i belive you are very right, and I am trying to follow the recommendations as close as possible. Will see if can come up with something better.09:06
ChosSimbaOneWill try just to put br-vxlan on a 10.14.x.x as with my first test and see if that helps me out :-)09:07
CeeMaci don't know that anything specifically 'calls' the br-storage network explicitly, you might get away with not having it if your storage is accessible on the br-mgmt network09:07
ChosSimbaOneCeeMac and jrosser thank you for all the help btw.09:07
CeeMaci have  br-storage configured to match the reference architecture, but my storage is on a seperate out of band bonded interface09:07
CeeMacand things work fine like that :)09:08
*** tosky has quit IRC09:08
CeeMacfor me, the simplest tests should be minimal server install, but network setup like prod09:08
CeeMacas networks are nearly always a pain point somewhere :)09:09
*** cshen has quit IRC09:09
CeeMaci'd be tempted to put external on a different vlan too,  if thats where you've having your main problems09:09
CeeMacactually ChosSimbaOne it might be worth leaving all of your networking as is, except make a new vlan for external09:10
CeeMacas that is where your problem lies right now, then you're only making 1 change so it makes troubleshooting easier09:10
*** gkadam has quit IRC09:11
ChosSimbaOneCeeMac: okay, I am trying to deploy against a existing ceph cluster for glance and cinder. Yeah you are right.09:11
ChosSimbaOnegood idea, so having an vlan added to the trunk for lb_external09:12
*** rgogunskiy has quit IRC09:17
*** vnogin has joined #openstack-ansible09:17
*** rgogunskiy has joined #openstack-ansible09:20
CeeMacChosSimbaOne: sounds good, then you can have a vlan interface assigned as the external nic for the LB09:20
CeeMacChosSimbaOne: haproxy_keepalived_external_interface in user_variables in case you don't have it :)09:22
*** rgogunskiy has quit IRC09:22
CeeMacChosSimbaOne: just checking, you have these set up? https://docs.openstack.org/openstack-ansible-haproxy_server/latest/configure-haproxy.html#making-haproxy-highly-available09:23
*** rgogunskiy has joined #openstack-ansible09:27
jrosserHmm I wonder if keepalived likes running twice on the same interface if internal and external vip are on the same bridge......09:34
*** rgogunskiy has quit IRC09:37
*** rgogunskiy has joined #openstack-ansible09:38
CeeMacjrosser: that would probably confuse it09:41
CeeMacmaybe09:41
* CeeMac shrugs09:41
*** hamzaachi has joined #openstack-ansible09:42
*** tosky__ is now known as tosky09:43
*** cshen has joined #openstack-ansible09:44
*** rgogunskiy has quit IRC09:47
*** cshen has quit IRC09:49
*** rgogunskiy has joined #openstack-ansible09:50
*** rgogunskiy has quit IRC09:52
openstackgerritMerged openstack/openstack-ansible-os_glance stable/stein: Updated from OpenStack Ansible Tests  https://review.opendev.org/66670509:52
*** rgogunskiy has joined #openstack-ansible09:59
*** cshen has joined #openstack-ansible10:05
noonedeadpunkjrosser: I looks exactly like situation with undefined variable for tempest with it's call of meta handlers10:05
*** vnogin has quit IRC10:06
noonedeadpunkI was thinking about to try import instead of include10:06
*** spsurya has joined #openstack-ansible10:07
*** rgogunskiy has quit IRC10:09
openstackgerritMerged openstack/openstack-ansible-os_designate stable/stein: Updated from OpenStack Ansible Tests  https://review.opendev.org/66670410:10
*** cshen has quit IRC10:10
*** hamzaachi has quit IRC10:24
*** cshen has joined #openstack-ansible10:26
*** hamzaachi has joined #openstack-ansible10:28
*** cshen has quit IRC10:31
*** udesale has quit IRC10:31
*** vnogin has joined #openstack-ansible10:32
*** cshen has joined #openstack-ansible10:47
*** cshen has quit IRC10:52
noonedeadpunkjrosser: strange that failure was only at the gate here https://review.opendev.org/#/c/657270/310:54
openstackgerritMerged openstack/openstack-ansible master: Remove credential sharing with nova  https://review.opendev.org/65817810:54
*** cshen has joined #openstack-ansible11:02
*** rgogunskiy has joined #openstack-ansible11:06
*** cshen has quit IRC11:06
*** kplant has joined #openstack-ansible11:07
*** jbadiapa has quit IRC11:09
jrossernoonedeadpunk: I thought it looked different with an included role using the vars from the parent, rather than itself11:12
jrosserBut all of our other roles except os_zun have dropped galera_client which would make that error go away anyway11:12
*** cshen has joined #openstack-ansible11:13
jrosserLook at the line in the output where it loads the distro specific vars in galera_client, it actually loads os_zun/vars/redhat.yml and that’s why the galera vars are missing11:14
noonedeadpunkBut galera should load it's own vars, doesn't she?11:16
jrosserWell yes11:16
noonedeadpunkI mean, that it seems to be not only about zun and galera, but either about our usage of includes or ansible bug...11:17
jrosserBut iirc we’ve dropped usage of that role everywhere else11:17
jrosserSo we need to ditch galera_client from zun11:17
jrosserBut like you say I think there is something we are overlooking with the behaviour of import vs include11:17
*** vnogin has quit IRC11:18
*** vnogin has joined #openstack-ansible11:21
noonedeadpunkso yeah, the easy solution is to just drop galera_client from zun (and I think we need to do this anyway)11:21
jrosserRight, and we need to sanity check the way these imports work11:24
jrosserFeels like it’s easy to make a mistake there11:24
noonedeadpunkyep, so I'll try to spend some time to get it fixed in the current state, as it should be smth easy to reproduce and to debug11:25
*** vnogin has quit IRC11:31
*** schwicht has joined #openstack-ansible11:35
*** rgogunskiy has quit IRC11:35
*** rgogunskiy has joined #openstack-ansible11:36
*** hamzaachi has quit IRC11:38
*** vnogin has joined #openstack-ansible11:40
*** schwicht has quit IRC11:57
*** udesale has joined #openstack-ansible12:10
*** vnogin has quit IRC12:16
*** schwicht has joined #openstack-ansible12:21
*** schwicht has quit IRC12:29
*** schwicht has joined #openstack-ansible12:31
*** cshen has quit IRC12:36
*** cshen has joined #openstack-ansible12:42
*** cshen has quit IRC12:46
*** cshen has joined #openstack-ansible12:48
*** jbadiapa has joined #openstack-ansible12:48
mnaserhi al12:52
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_zun master: Convert dynamic includes to static imports  https://review.opendev.org/66682912:53
CeeMachi mnaser12:55
*** schwicht has quit IRC12:56
*** schwicht has joined #openstack-ansible12:57
*** schwicht has quit IRC12:58
*** martinkennelly has joined #openstack-ansible13:19
*** schwicht has joined #openstack-ansible13:22
*** aedc has joined #openstack-ansible13:26
*** markvoelker has joined #openstack-ansible13:29
*** dave-mccowan has joined #openstack-ansible13:30
noonedeadpunkjrosser: interestingly https://review.opendev.org/#/c/657270/ passed check before https://review.opendev.org/#/c/656499/ got merged, so seems that smth really changed inside ansible13:30
jrosserWow, wonder what....13:32
noonedeadpunkthat's really good question :)13:34
mgariepy https://review.opendev.org/#/c/656499/ << got merged in may ?13:35
mgariepyho sorry..13:35
noonedeadpunkyep:) but https://review.opendev.org/#/c/657270/ passed on 6th of may)13:35
* mgariepy going back to my corner13:35
jrosserUnfortunately I have no time till next week to look but I’ve seen a bunch of Debian jobs timing out on linters13:39
jrosserAnd systemd-networkd role is having a bad time on centos, consistently looses connection to zuul I think13:40
*** cshen has quit IRC13:43
*** BjoernT has joined #openstack-ansible13:45
guilhermespmornings all!13:47
*** hamzaachi has joined #openstack-ansible13:56
*** markvoelker has quit IRC13:57
*** cshen has joined #openstack-ansible14:00
*** joshualyle has joined #openstack-ansible14:02
*** cshen has quit IRC14:08
*** cshen has joined #openstack-ansible14:25
*** BjoernT has quit IRC14:33
*** BjoernT has joined #openstack-ansible14:34
openstackgerritMerged openstack/openstack-ansible-os_ironic stable/stein: Updated from OpenStack Ansible Tests  https://review.opendev.org/66670714:35
*** BjoernT has quit IRC14:36
*** BjoernT_ has joined #openstack-ansible14:36
*** joshualyle has quit IRC14:39
openstackgerritMerged openstack/openstack-ansible-os_sahara stable/rocky: Updated from OpenStack Ansible Tests  https://review.opendev.org/66669114:46
openstackgerritMerged openstack/openstack-ansible-os_zun stable/rocky: Updated from OpenStack Ansible Tests  https://review.opendev.org/66670314:47
*** kopecmartin is now known as kopecmartin|off14:47
CeeMachey guilhermesp :)14:51
*** markvoelker has joined #openstack-ansible14:54
*** spatel has joined #openstack-ansible14:55
CeeMacguilhermesp: I've filed a bug for the glance cinder store issues we've looked at https://bugs.launchpad.net/openstack-ansible/+bug/183372514:56
openstackLaunchpad bug 1833725 in openstack-ansible "Glance cinder store missing some configuration" [Undecided,New]14:56
CeeMaci'll try and work on a patch next week if i get time14:57
spatelQuick question related Octavia  lb-mgmt network, I am build in 3 node production controller, do i need to create lb-mgmt network or it will get auto create ?14:57
*** markvoelker has quit IRC14:58
*** KurtB has joined #openstack-ansible15:03
*** KurtB has quit IRC15:04
spatelI get it i need to create  "br-lbaas" bridge like i created others br-mgmt, br-host etc..15:04
*** rgogunskiy has quit IRC15:06
*** rgogunskiy has joined #openstack-ansible15:08
*** cshen has quit IRC15:14
*** rgogunskiy has quit IRC15:14
*** gkadam has joined #openstack-ansible15:15
openstackgerritMerged openstack/openstack-ansible-os_zun stable/stein: Updated from OpenStack Ansible Tests  https://review.opendev.org/66671915:16
*** tosky has quit IRC15:18
*** ansmith has quit IRC15:22
*** ansmith has joined #openstack-ansible15:45
*** gyee has joined #openstack-ansible15:46
*** ansmith has quit IRC15:51
*** markvoelker has joined #openstack-ansible15:55
*** rpittau is now known as rpittau|afk15:57
*** markvoelker has quit IRC15:59
*** spsurya has quit IRC16:24
openstackgerritMerged openstack/openstack-ansible-os_heat stable/stein: Updated from OpenStack Ansible Tests  https://review.opendev.org/66670616:26
*** ckonstanski has joined #openstack-ansible16:31
*** martinkennelly has quit IRC16:37
*** udesale has quit IRC16:39
*** udesale has joined #openstack-ansible16:39
*** udesale has quit IRC16:50
*** weifan has joined #openstack-ansible16:55
*** markvoelker has joined #openstack-ansible16:56
*** gkadam has quit IRC17:00
*** markvoelker has quit IRC17:01
*** aedc has quit IRC17:07
*** ansmith has joined #openstack-ansible17:26
*** hwoarang has quit IRC17:31
guilhermespnice CeeMac thanks for that. I with no time too unfortunately but if you do before me I will review it o/17:32
*** hwoarang has joined #openstack-ansible17:32
CeeMacguilhermesp: sounds good :)17:42
*** markvoelker has joined #openstack-ansible17:56
*** markvoelker has quit IRC18:01
*** hwoarang has quit IRC18:13
*** hwoarang has joined #openstack-ansible18:14
*** tosky has joined #openstack-ansible18:19
*** hamzaachi has quit IRC18:20
*** BjoernT has joined #openstack-ansible18:35
*** BjoernT_ has quit IRC18:37
*** markvoelker has joined #openstack-ansible18:57
*** hamzaachi has joined #openstack-ansible18:59
*** hamzaachi has quit IRC19:18
*** markvoelker has quit IRC19:26
BjoernT@spatel the network will autocreate once you add it to octavia_service_net_setup, octavia_management_net_subnet_allocation_pool, octavia_provider_segmentation_id and set octavia_service_net_setup  to true19:35
*** ansmith has quit IRC19:37
*** ansmith has joined #openstack-ansible19:37
*** hamzaachi has joined #openstack-ansible19:39
*** kplant has quit IRC19:56
*** schwicht has quit IRC20:00
*** tosky__ has joined #openstack-ansible20:03
*** tosky has quit IRC20:03
*** markvoelker has joined #openstack-ansible20:23
*** markvoelker has quit IRC20:28
*** schwicht has joined #openstack-ansible20:29
*** BjoernT has quit IRC20:33
*** hamzaachi has quit IRC20:54
*** pcaruana has quit IRC21:16
*** joshualyle has joined #openstack-ansible21:17
*** mmercer has quit IRC21:20
*** joshualyle has quit IRC21:21
*** schwicht has quit IRC21:22
*** markvoelker has joined #openstack-ansible21:24
*** joshualyle has joined #openstack-ansible21:27
*** markvoelker has quit IRC21:29
*** joshualyle has quit IRC21:29
*** joshualyle has joined #openstack-ansible21:29
*** openstackgerrit has quit IRC21:33
*** joshualyle has quit IRC21:34
*** hamzaachi has joined #openstack-ansible21:35
*** joshualyle has joined #openstack-ansible21:39
*** joshualyle has quit IRC21:44
*** persia_ is now known as persia21:45
*** schwicht has joined #openstack-ansible21:45
*** spatel has quit IRC21:49
ChosSimbaOnejrosser and CeeMac Thanks for all the suggestion, did not get to try it out today due to long meeting, so will go back to the drawing board monday and see if i can get it working. Have a nice weekend :-)21:52
*** joshualyle has joined #openstack-ansible21:52
*** joshualyle has quit IRC21:56
*** schwicht has quit IRC21:57
*** schwicht has joined #openstack-ansible22:00
*** hamzaachi has quit IRC22:02
*** tosky__ is now known as tosky22:14
*** markvoelker has joined #openstack-ansible22:25
*** markvoelker has quit IRC22:30
*** schwicht has quit IRC22:31
*** gyee has quit IRC22:40
*** spatel has joined #openstack-ansible22:43
*** rgogunskiy has joined #openstack-ansible22:45
*** spatel has quit IRC22:47
*** dave-mccowan has quit IRC22:49
*** rgogunskiy has quit IRC22:50
*** schwicht has joined #openstack-ansible23:00
*** weifan has quit IRC23:16
*** weifan has joined #openstack-ansible23:18
*** weifan has quit IRC23:19
*** weifan has joined #openstack-ansible23:20
*** tosky has quit IRC23:22
*** weifan has quit IRC23:25
*** markvoelker has joined #openstack-ansible23:26
*** joshualyle has joined #openstack-ansible23:27
*** joshualyle has quit IRC23:27
*** joshualyle has joined #openstack-ansible23:27
*** markvoelker has quit IRC23:30
*** luksky has quit IRC23:31
*** weifan has joined #openstack-ansible23:42
*** weifan has quit IRC23:44
*** weifan has joined #openstack-ansible23:45
*** weifan has quit IRC23:45

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!