| *** hamzaachi has quit IRC | 00:05 | |
| *** tosky has quit IRC | 00:14 | |
| *** nurdie has joined #openstack-ansible | 00:16 | |
| *** sdake has quit IRC | 00:25 | |
| *** sdake has joined #openstack-ansible | 00:29 | |
| *** sdake has quit IRC | 00:30 | |
| *** ansmith_ has joined #openstack-ansible | 00:41 | |
| *** TxGirlGeek has quit IRC | 00:44 | |
| *** schwicht has joined #openstack-ansible | 00:46 | |
| *** sdake has joined #openstack-ansible | 00:56 | |
| *** cmart has quit IRC | 01:03 | |
| *** sdake has quit IRC | 01:03 | |
| *** nurdie has quit IRC | 01:16 | |
| *** nurdie has joined #openstack-ansible | 01:17 | |
| *** ansmith_ has quit IRC | 01:19 | |
| *** sdake has joined #openstack-ansible | 01:19 | |
| *** sdake has quit IRC | 01:27 | |
| *** cmart has joined #openstack-ansible | 01:35 | |
| *** sdake has joined #openstack-ansible | 01:35 | |
| *** cmart has quit IRC | 01:39 | |
| *** macza has quit IRC | 01:49 | |
| *** nurdie has quit IRC | 01:50 | |
| *** nurdie has joined #openstack-ansible | 01:51 | |
| *** sdake has quit IRC | 01:52 | |
| *** nurdie has quit IRC | 01:55 | |
| *** macza has joined #openstack-ansible | 02:09 | |
| *** sdake has joined #openstack-ansible | 02:20 | |
| *** gyee has quit IRC | 02:35 | |
| *** cmart has joined #openstack-ansible | 02:47 | |
| *** cmart has quit IRC | 03:03 | |
| cloudnull | evenings | 03:06 |
|---|---|---|
| *** nurdie has joined #openstack-ansible | 03:15 | |
| *** chandankumar has joined #openstack-ansible | 03:17 | |
| *** sdake has quit IRC | 03:17 | |
| *** DanyC has joined #openstack-ansible | 03:17 | |
| *** sdake has joined #openstack-ansible | 03:18 | |
| *** bgmccollum has quit IRC | 03:19 | |
| *** DanyC has quit IRC | 03:22 | |
| *** bgmccollum has joined #openstack-ansible | 03:24 | |
| *** sdake has quit IRC | 03:35 | |
| *** sohny has joined #openstack-ansible | 03:42 | |
| *** dave-mccowan has quit IRC | 03:44 | |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_tempest master: Ping router once it is created https://review.openstack.org/633883 | 03:50 |
| *** chandankumar has quit IRC | 03:51 | |
| *** sdake has joined #openstack-ansible | 03:57 | |
| *** sohny has quit IRC | 04:14 | |
| *** sdake has quit IRC | 04:16 | |
| *** gkadam__ has joined #openstack-ansible | 04:17 | |
| *** udesale has joined #openstack-ansible | 04:17 | |
| *** chandankumar has joined #openstack-ansible | 04:32 | |
| *** chandankumar is now known as chkumar|ruck | 04:32 | |
| *** sdake has joined #openstack-ansible | 04:43 | |
| *** macza has quit IRC | 04:45 | |
| *** slaweq has joined #openstack-ansible | 04:55 | |
| *** chkumar|ruck has quit IRC | 04:56 | |
| *** chkumar|ruck has joined #openstack-ansible | 04:57 | |
| *** shyamb has joined #openstack-ansible | 05:06 | |
| *** macza has joined #openstack-ansible | 05:08 | |
| openstackgerrit | Merged openstack/ansible-config_template master: Replace Chinese quotes with English quotes https://review.openstack.org/607104 | 05:24 |
| *** chkumar|ruck has quit IRC | 05:48 | |
| *** chkumar|ruck has joined #openstack-ansible | 05:48 | |
| *** shyamb has quit IRC | 05:48 | |
| *** slaweq has quit IRC | 05:49 | |
| *** sohny has joined #openstack-ansible | 05:50 | |
| *** macza has quit IRC | 05:50 | |
| *** hamzaachi has joined #openstack-ansible | 05:56 | |
| *** shyamb has joined #openstack-ansible | 06:01 | |
| *** sdake has quit IRC | 06:03 | |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_tempest master: Ping router once it is created https://review.openstack.org/633883 | 06:07 |
| *** shyamb has quit IRC | 06:10 | |
| *** shyamb has joined #openstack-ansible | 06:10 | |
| *** hamzaachi has quit IRC | 06:11 | |
| *** cmart has joined #openstack-ansible | 06:15 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/ansible-config_template master: Correct "changed" to work with the latest ansible https://review.openstack.org/634856 | 06:16 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-plugins master: Update the config_template plugin from upstream https://review.openstack.org/634857 | 06:19 |
| *** cmart has quit IRC | 06:23 | |
| *** markvoelker has joined #openstack-ansible | 06:45 | |
| *** slaweq has joined #openstack-ansible | 07:00 | |
| *** chkumar|ruck has quit IRC | 07:10 | |
| *** markvoelker has quit IRC | 07:18 | |
| *** Emine has joined #openstack-ansible | 07:43 | |
| *** pcaruana has joined #openstack-ansible | 07:45 | |
| *** chkumar|ruck has joined #openstack-ansible | 07:47 | |
| *** shyamb has quit IRC | 07:58 | |
| *** kopecmartin|off is now known as kopecmartin | 08:03 | |
| *** markvoelker has joined #openstack-ansible | 08:15 | |
| *** nurdie has quit IRC | 08:28 | |
| *** nurdie has joined #openstack-ansible | 08:29 | |
| *** nurdie has quit IRC | 08:33 | |
| *** shyamb has joined #openstack-ansible | 08:45 | |
| *** markvoelker has quit IRC | 08:49 | |
| *** tosky has joined #openstack-ansible | 08:50 | |
| *** priteau has joined #openstack-ansible | 08:51 | |
| *** priteau has quit IRC | 08:51 | |
| *** priteau has joined #openstack-ansible | 08:52 | |
| *** shardy has joined #openstack-ansible | 08:58 | |
| *** shardy has quit IRC | 08:59 | |
| *** shardy has joined #openstack-ansible | 09:00 | |
| *** rgogunskiy has joined #openstack-ansible | 09:16 | |
| *** DanyC has joined #openstack-ansible | 09:24 | |
| *** nurdie has joined #openstack-ansible | 09:29 | |
| *** nurdie has quit IRC | 09:34 | |
| openstackgerrit | Erik Berg proposed openstack/openstack-ansible-galera_server stable/rocky: Fix ansible deprecation warnings https://review.openstack.org/634893 | 09:45 |
| *** markvoelker has joined #openstack-ansible | 09:46 | |
| chkumar|ruck | arxcruz, hello | 09:48 |
| arxcruz | chkumar|ruck: hello | 09:48 |
| chkumar|ruck | arxcruz, I need some help on this patch https://review.openstack.org/#/c/633883/ | 09:48 |
| *** aedc has joined #openstack-ansible | 09:48 | |
| chkumar|ruck | arxcruz, I am trying to get the value from this dict http://logs.openstack.org/83/633883/9/check/openstack-ansible-functional-centos-7/c151930/logs/ara-report/result/28d009bb-6711-4ec8-9f68-f6743f453236/ | 09:49 |
| chkumar|ruck | arxcruz, but I am not able to retrive it | 09:49 |
| chkumar|ruck | arxcruz, please have a look! | 09:50 |
| arxcruz | chkumar|ruck: isn't _add_router['router']['external_gateway_info'][['...'] ? | 09:55 |
| arxcruz | chkumar|ruck: http://man.hubwiz.com/docset/Ansible.docset/Contents/Resources/Documents/docs.ansible.com/ansible/os_router_module.html at the bottom of the page | 09:56 |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_tempest master: Ping router once it is created https://review.openstack.org/633883 | 09:58 |
| chkumar|ruck | arxcruz, I thought it is dumping all the values in _add_router var | 09:58 |
| chkumar|ruck | arxcruz, updated, let's see | 09:58 |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_heat master: [DNM] heat tempest tests finding https://review.openstack.org/630694 | 10:04 |
| *** chkumar|ruck has quit IRC | 10:09 | |
| *** chandankumar has joined #openstack-ansible | 10:15 | |
| *** chandankumar is now known as chkumar|ruck | 10:15 | |
| *** gillesMo has joined #openstack-ansible | 10:18 | |
| *** markvoelker has quit IRC | 10:19 | |
| *** shyamb has quit IRC | 10:32 | |
| CeeMac | jamesdenton, jrosser: as discussed yesterday - https://bugs.launchpad.net/openstack-ansible/+bug/1814686 | 10:32 |
| openstack | Launchpad bug 1814686 in openstack-ansible "host_bind_override issues for br-vlan network provider" [Undecided,New] | 10:33 |
| *** slaweq has quit IRC | 10:33 | |
| jrosser | CeeMac: great that it works - i have exactly the same problem with the vlan interfaces only existing on the network hosts | 10:36 |
| CeeMac | jrosser, I remember chatting with you a while back about your environment, it sounds very similar to how I'm hoping to get mine up and running | 10:36 |
| jrosser | yeah, i have seperate hosts for pretty much everything, network, haproxy, infra and so on | 10:37 |
| CeeMac | actually, seeing as you're here :) what are your thoughts/understandings on performance/stability of l2pop vs multicast for the vxlan communication | 10:38 |
| CeeMac | just wondering if its worth my time batting my demons and looking at implementing multicast in the core for this, are there other elements that would benefit from it to? | 10:38 |
| jrosser | i am using multicast, no experience directly with l2pop other than thinking it's a bit wierd | 10:38 |
| CeeMac | haha, weird how? | 10:38 |
| jrosser | well, just replicating something thats inherent in vxlan, feels more like a historical workaround in opnestack rather than a "proper" thing | 10:39 |
| jrosser | maybe i'm too much of a purist there :) | 10:39 |
| jrosser | if you ever want to interoperate with a physical vtep in a switch for example...... | 10:40 |
| jrosser | anyway, i have a routed network, you need to watch out for multicast ttl must be > 1 | 10:41 |
| jrosser | i put the tunnel networks in their own VRF on the switches so i only have multicast configured there, not universally | 10:41 |
| CeeMac | thats a good shout | 10:42 |
| CeeMac | what switching gear are you using? | 10:42 |
| jrosser | usual PIM + anycast RP on my spine switches, fairly standard | 10:42 |
| jrosser | nexus 9k | 10:42 |
| CeeMac | ah, ok | 10:42 |
| CeeMac | big boy switches :) | 10:42 |
| jrosser | if you find a cisco or juniper reference for setting up vlxan overlay even without openstack all the same stuff holds | 10:43 |
| *** shyamb has joined #openstack-ansible | 10:43 | |
| jrosser | however for a small setup you can easily do it in one switch just with igmp snooping turned on | 10:44 |
| jrosser | i would highly recommend using the asmping tools to verify you've got everything working properly | 10:45 |
| *** hwoarang has quit IRC | 10:47 | |
| CeeMac | unfortunately, although i have dedicated compute for the environment its still connected over the production core switch (juniper) | 10:47 |
| CeeMac | so yeah. i'll need to check the docs for vxlan with multicast in a routing-instance | 10:47 |
| CeeMac | i'll keep a note of the asmping tools | 10:48 |
| CeeMac | i like the simplicity of l2pop, but for production i'd worry there would be an overhead | 10:50 |
| jrosser | i've always wanted to leave the option open interoperating with a hardware vtep, so i've stuck firmly to standards based stuff | 10:51 |
| jrosser | but it depends entirely on your use case | 10:52 |
| jrosser | use what works for you :) | 10:52 |
| CeeMac | i like to future proof new platforms i'm working on | 10:52 |
| CeeMac | there is definitely scope for breaking out into physical networking | 10:52 |
| CeeMac | juniper integrates well with openstack for RaaS, FWaaS etc | 10:52 |
| CeeMac | and i try to adopt an open standard approach as well, where possible | 10:53 |
| CeeMac | but for this RC, I'm happy with the "make it work now" approach due to management pressures for a working demo | 10:53 |
| CeeMac | i'll trying to keep notes on things to loop back on for production and "best practice" implementation | 10:54 |
| *** bgmccollum has quit IRC | 11:05 | |
| *** kopecmartin has quit IRC | 11:05 | |
| *** mwe1 has quit IRC | 11:06 | |
| *** mwe1 has joined #openstack-ansible | 11:09 | |
| *** kopecmartin has joined #openstack-ansible | 11:10 | |
| *** bgmccollum has joined #openstack-ansible | 11:12 | |
| *** markvoelker has joined #openstack-ansible | 11:15 | |
| *** udesale has quit IRC | 11:28 | |
| *** ansmith_ has joined #openstack-ansible | 11:36 | |
| *** slaweq has joined #openstack-ansible | 11:46 | |
| *** markvoelker has quit IRC | 11:48 | |
| *** ansmith_ has quit IRC | 11:55 | |
| *** slaweq has quit IRC | 12:09 | |
| jamesdenton | Thanks, CeeMac. The bug report looks great and I'll work on some docs soon and send them your way for review, if thats OK | 12:24 |
| CeeMac | no problem. yeah, happy to do that | 12:26 |
| jamesdenton | As for l2pop vs multicast - There has been some overhead and caveats to l2pop in the past. One of my major grievances was the difference in behavior between linuxbridge and OVS. Namely, that if for any reason the FDB entry did not exist for a given MAC, then traffic would just be dropped. What this meant was that in the event the agent didn't properly program the entry, nothing went anywhere. With OVS, at least, the | 12:28 |
| jamesdenton | traffic would be broadcast as a last resort and an FDB could be learned. At the time, this was a limitation of the kernel module and there were workarounds implemented IIRC (to also help with allowed address pairs). Anyway, we defaulted it to off in OSA around the Mitaka timeframe IIRC. But it's necessary if you don't have a working multicast setup | 12:28 |
| jamesdenton | just enabling igmp snooping may be enough to get it going, i don't remember | 12:29 |
| CeeMac | yeah, i think i'm going to have to bite the bullet and jump in to multicast setup | 12:30 |
| CeeMac | i know there are a few other systems we have that could benefit from it if i can implement it correctly in a controlled way within virtual routers | 12:31 |
| CeeMac | as an aside, on the LB/OVS front, is OVS still a pre-requisite for distributed VRs | 12:31 |
| jamesdenton | Yes, OVS is required for DVR | 12:32 |
| *** samc-bbc has quit IRC | 12:32 | |
| jamesdenton | we've got some folks using OVS within OSA, along with DVR, that i think are getting along well | 12:32 |
| CeeMac | they can be set up concurrently in the same environment albeit it on different hosts right? | 12:33 |
| jamesdenton | As in, LinuxBridge and OVS in the same environment? | 12:34 |
| CeeMac | yeah | 12:34 |
| CeeMac | or is that only true in a non-osa driven build? | 12:34 |
| jamesdenton | Yes, that is doable, albeit not really tested beyond the lab, at least for me anyway | 12:34 |
| jamesdenton | The mechanism to deploy it that way looks a lot like what you did yesterday | 12:35 |
| jamesdenton | I think where things could get hairy is in the routing | 12:35 |
| CeeMac | i'm still stuck in the do-it-quick vs do-it-right duality. | 12:35 |
| jamesdenton | To me, there isn't much to gain in a dual setup like that in production | 12:36 |
| CeeMac | i have 1 compute node on LB right now, to get up and running. Was thinking I could slot in a second one on OVS and transition across maybe? | 12:36 |
| jamesdenton | unless you're transitioning or something | 12:36 |
| CeeMac | yeah, this would be transitional in the RC | 12:36 |
| CeeMac | in prod I would go OVS if we decided to go with DVR | 12:36 |
| CeeMac | not sure if there is a benefit of OVS over LB in general outwith the case for DVR? | 12:37 |
| jamesdenton | Well in that case, yes, you could do that. There are a couple of user_variables to implement, too, but it's fairly straightforward | 12:37 |
| CeeMac | is that document or can i just ping you a message if/when i come to it? | 12:37 |
| jamesdenton | Well, feature-wise DVR is the one that immediately comes to mind. There is also the benefit of OVS firewall driver vs iptables. | 12:38 |
| jamesdenton | yeah, just ping me | 12:38 |
| *** slaweq has joined #openstack-ansible | 12:38 | |
| CeeMac | cool, thanks | 12:38 |
| jamesdenton | You can also get DPDK w/ OVS down the road | 12:38 |
| jamesdenton | If I have some free cycles I'll throw up a blog on how to do it so there's something to reference | 12:38 |
| CeeMac | that would be great, if you had the time | 12:39 |
| CeeMac | I'll probably park the networking config decision for a little while as I have a few other things to get through to provide a "limited production capable" demo | 12:40 |
| CeeMac | which I'll no doubt be back on here to ask for help with at some point :) | 12:41 |
| jamesdenton | :) | 12:41 |
| jamesdenton | jrosser Re: LB + VPP. I spent some time on this recently, and the issue I'm coming up against is that while it's possible to enable multiple mechanism drivers, getting OSA to run the respective provider tasks is tricky. For something like LB+OVS, those mech drivers are built in so it's not a problem. Same goes for adding sriovnicswitch or l2population. When it comes to something like VPP, there are pip packages that | 12:45 |
| jamesdenton | need to be installed not only on the compute node but within the neutron-server container. | 12:45 |
| *** markvoelker has joined #openstack-ansible | 12:46 | |
| jamesdenton | Right now, neutron_plugin_type is expected to be a single value "ml2.lxb, ml2.ovs, ml2.vpp, etc", and this code block includes the respective task file: https://github.com/openstack/openstack-ansible-os_neutron/blob/master/tasks/main.yml#L77-L86 | 12:46 |
| jamesdenton | I'm soliciting advice on how to process that as a list, and iterate thru the elements and include the respective file(s). I'm running into the issue of a loop-within-a-loop, though | 12:46 |
| *** gkadam__ has quit IRC | 12:58 | |
| *** slaweq has quit IRC | 12:59 | |
| *** shyamb has quit IRC | 13:03 | |
| *** sdake has joined #openstack-ansible | 13:06 | |
| *** sohny has quit IRC | 13:16 | |
| *** markvoelker has quit IRC | 13:19 | |
| *** ansmith_ has joined #openstack-ansible | 13:25 | |
| *** sdake has quit IRC | 13:54 | |
| chkumar|ruck | jrosser, odyssey4me hello | 13:54 |
| chkumar|ruck | jrosser, odyssey4me http://logs.openstack.org/94/630694/9/check/openstack-ansible-deploy-aio_metal_heat-centos-7/a655e4c/job-output.txt.gz#_2019-02-05_11_28_53_271932 | 13:55 |
| chkumar|ruck | jrosser, odyssey4me Collecting heat\n\n:stderr: The repository located at mirror.ord.rax.openstack.org is not a trusted or secure host and is being ignored. If this repository is available via HTTPS we recommend you use HTTPS instead, otherwise you may silence this warning and allow it anyway with '--trusted-host mirror.ord.rax.openstack.org'.\n The repository located at mirror.ord.rax.openstack.org is not a trusted | 13:55 |
| chkumar|ruck | or secure host and is being ignored. If this repository is available via HTTPS we recommend you use HTTPS instead, otherwise you may silence this warning and allow it anyway with '--trusted-host mirror.ord.rax.openstack.org'.\ | 13:55 |
| *** sdake has joined #openstack-ansible | 14:03 | |
| *** chkumar|ruck has quit IRC | 14:05 | |
| *** dave-mccowan has joined #openstack-ansible | 14:09 | |
| jamesdenton | jrosser CeeMac Let me know what you think of the language in this example: http://paste.openstack.org/show/744567/ | 14:11 |
| jamesdenton | I am thinking of creating a new scenario based on this: https://docs.openstack.org/openstack-ansible/latest/user/prod/example.html, along with a diagram that might help clear it up | 14:11 |
| *** dave-mccowan has quit IRC | 14:13 | |
| *** markvoelker has joined #openstack-ansible | 14:16 | |
| *** sdake has quit IRC | 14:17 | |
| CeeMac | jamesdenton, looks good. | 14:24 |
| CeeMac | i like how the groups are used in all of the examples so its easy to see which provider networks are going to apply to which nodes | 14:25 |
| jamesdenton | cool. I will draw a diagram that matches up the example and drive it home | 14:25 |
| jamesdenton | thanks | 14:25 |
| CeeMac | that'd be good. the diagrams, although good, seem very genericised | 14:27 |
| jamesdenton | thats true | 14:29 |
| CeeMac | and it can be tricky to try and map a non-generic scenario through them | 14:29 |
| CeeMac | certainly in my experience | 14:29 |
| CeeMac | although I understand the need for homogenousity (if thats a word?) | 14:29 |
| CeeMac | completely unrelated question, config file overrides can be done in user_variables.yml right? or through individual .yml files in the environment directory? (/etc/openstack_deploy by default?) | 14:31 |
| jamesdenton | it just sorta depends on how you want them applied, generally or to a host or groups | 14:32 |
| jamesdenton | but yes, they often go into user_variables | 14:32 |
| *** udesale has joined #openstack-ansible | 14:32 | |
| CeeMac | in this scenario it would be for a 3rd party cinder volume driver | 14:33 |
| jamesdenton | i would think user_variables would be OK for that | 14:33 |
| CeeMac | ok cool | 14:33 |
| CeeMac | just trying to work out when branching out into multiple individual variable files will be more practical in the long term than having everything in user_variables | 14:34 |
| *** sdake has joined #openstack-ansible | 14:34 | |
| *** gshippey has quit IRC | 14:36 | |
| *** cmart has joined #openstack-ansible | 14:36 | |
| *** sdake has quit IRC | 14:45 | |
| *** sdake has joined #openstack-ansible | 14:47 | |
| *** markvoelker has quit IRC | 14:49 | |
| *** sdake has quit IRC | 14:56 | |
| *** priteau has quit IRC | 14:58 | |
| *** fnpanic has joined #openstack-ansible | 15:01 | |
| fnpanic | hi | 15:01 |
| CeeMac | jamesdenton, is there a way to bounce a neutron router interface? | 15:01 |
| *** nurdie has joined #openstack-ansible | 15:02 | |
| fnpanic | i wanted to add some host var to a specific host for ceph ansible but it did not work out | 15:02 |
| *** nurdie has joined #openstack-ansible | 15:02 | |
| fnpanic | i added in openstack_user_config.yml a host_vars: under one ceph osd and added devices: | 15:03 |
| fnpanic | is this supposed to work? | 15:03 |
| *** sdake has joined #openstack-ansible | 15:03 | |
| *** mattheca has quit IRC | 15:05 | |
| *** udesale has quit IRC | 15:06 | |
| *** nurdie has quit IRC | 15:07 | |
| *** slaweq has joined #openstack-ansible | 15:08 | |
| fnpanic | https://docs.openstack.org/openstack-ansible/rocky/reference/configuration/using-overrides.html | 15:13 |
| jamesdenton | CeeMac You try can to set admin state up/down on the port | 15:13 |
| CeeMac | yeah, i've tried --disable and --enable on the port and its admin up but state down | 15:14 |
| CeeMac | trying to recall if it was up before we made the changes yesterday | 15:14 |
| jamesdenton | is it not working? | 15:14 |
| CeeMac | you can get in to the router from the o/s to run diagnostics? | 15:14 |
| CeeMac | it is not | 15:14 |
| jamesdenton | sure. | 15:15 |
| jamesdenton | it's a 'qrouter' network namespace | 15:15 |
| CeeMac | VM times out at gateway getting to internet | 15:15 |
| jamesdenton | ip netns list will list them | 15:15 |
| jamesdenton | so you can ping the router from the VM but can't get out? | 15:15 |
| CeeMac | the ids should match back to openstack? | 15:15 |
| jamesdenton | yes | 15:15 |
| CeeMac | yep | 15:15 |
| CeeMac | traceroute fails from external interface "no route to destination" | 15:16 |
| jamesdenton | can you send the output of "ip netns exec qrouter-xxxxxxxxx ip addr" | 15:18 |
| CeeMac | yep | 15:18 |
| CeeMac | getting incomplete arp after trying to ping gateway on lan | 15:18 |
| jamesdenton | k | 15:18 |
| cloudnull | mornings | 15:18 |
| cloudnull | if folks have a moment, it'd be great to get this pr reviewed https://review.openstack.org/#/c/634857 | 15:19 |
| jamesdenton | also send the output of "ip netns exec qrouter-xxxxxxxxx ip route" and "ip netns exec qrouter-xxxxxxxxx ip neigh show" | 15:19 |
| CeeMac | http://paste.openstack.org/show/744572/ | 15:19 |
| CeeMac | http://paste.openstack.org/show/744573/ | 15:20 |
| jamesdenton | how about "brctl show" on the node itself | 15:21 |
| CeeMac | http://paste.openstack.org/show/744574/ | 15:22 |
| jamesdenton | Well, the plumbing looks OK. I see the QG interface in the bridge along with enp2s0f1.310. Is enp2s0f1 up? | 15:23 |
| jamesdenton | For test purposes, you can throw an IP on brq31a89156-4e and ping the external gateway. If it works, it will help eliminate some external possibilities | 15:24 |
| CeeMac | hmm | 15:25 |
| CeeMac | now you mention it | 15:25 |
| CeeMac | i don't know if enp2s0f1 is up >_> | 15:25 |
| CeeMac | i had to bounce the switch to enable jumbo frames yesterday, i wonder if thats upset it | 15:26 |
| jamesdenton | Does it look up from the OS standpoint? "ip link show enp2s0f1" | 15:27 |
| CeeMac | down | 15:28 |
| jamesdenton | so, you can set it up temporarily with "ip link set enp2s0f1 up" | 15:29 |
| CeeMac | i couldnt see any packets going through the counter in ifconfig either | 15:29 |
| CeeMac | just getting on to the switch now | 15:29 |
| jamesdenton | Well, do you see NO_CARRIER? | 15:29 |
| jamesdenton | it may just be down on the OS side | 15:29 |
| CeeMac | no | 15:29 |
| CeeMac | thats brought it up | 15:29 |
| jamesdenton | ok cool. It may be the netplan config | 15:29 |
| CeeMac | got reachable in router now for gateway address | 15:30 |
| CeeMac | and i can ping the gateway | 15:30 |
| jamesdenton | ok nice. Can you send your netplan config for that interface? | 15:30 |
| CeeMac | yeah, i was just looking that | 15:31 |
| CeeMac | its pretty sparse, lol | 15:31 |
| CeeMac | http://paste.openstack.org/show/744575/ | 15:32 |
| CeeMac | http://paste.openstack.org/show/744576/ | 15:34 |
| CeeMac | from our chat i think we resolved that br-vlan didn't need any interfaces, and the host_bind_override would allocate the provider vlan networks directly to the interface | 15:35 |
| CeeMac | which it must be doing as there is an interface there for it | 15:35 |
| jamesdenton | right | 15:35 |
| jamesdenton | this is a ... bug? caveat? to netplan | 15:35 |
| CeeMac | could this be the thing i read somewhere where in netplan if the interface has no config it doesn't come up? | 15:35 |
| jamesdenton | There are some heavy assumptions made by netplan devs (from what i read) that are slightly annoying | 15:36 |
| CeeMac | and you have to do something else | 15:36 |
| jamesdenton | yes | 15:36 |
| jamesdenton | the workaround was to create a systemd unit to do it | 15:36 |
| CeeMac | i had a page up about that somewhere in my gazillion chrome tabs | 15:36 |
| jamesdenton | https://askubuntu.com/questions/1037276/bring-up-but-dont-assign-address-with-netplan | 15:36 |
| CeeMac | that sounds about right | 15:36 |
| jrosser | CeeMac: if you are in trouble with netplan you can install the ifupdown package on ubuntu | 15:36 |
| CeeMac | just when i start to think netplan is alright, lol | 15:36 |
| CeeMac | jrosser, yeah, i kinda didn't wanna tho | 15:37 |
| jamesdenton | You can also do that ^^^, or just use networkd directly | 15:37 |
| jamesdenton | https://askubuntu.com/questions/1046420/why-is-netplan-networkd-not-bringing-up-a-static-ethernet-interface | 15:37 |
| jrosser | and everything works just like it used to on xenial <- this is what i'm doing for path of least resistance xenial->bionic upgrades | 15:37 |
| jamesdenton | netplan is the man-in-the-middle that really just complicates things | 15:37 |
| jamesdenton | IMO | 15:37 |
| CeeMac | yeah | 15:38 |
| CeeMac | netplan is proprietary to canonical? I think it read that somewhere? | 15:38 |
| jamesdenton | i think so | 15:38 |
| CeeMac | hmm | 15:38 |
| * cloudnull hates netplan | 15:39 | |
| CeeMac | you can use networkd directly to configure interfaces without ifupdown or? | 15:39 |
| jamesdenton | yes | 15:39 |
| cloudnull | its a ubuntu thing, which aims to make networkd easier. | 15:39 |
| jamesdenton | cloudnull has the goods | 15:39 |
| cloudnull | we have a role for it | 15:40 |
| cloudnull | https://github.com/openstack/ansible-role-systemd_networkd | 15:40 |
| cloudnull | I also have an example networkd setup which i use in my lab | 15:40 |
| cloudnull | https://gist.github.com/cloudnull/f38516639a31ef42b91fa3acbf7e28a0 | 15:41 |
| *** nurdie has joined #openstack-ansible | 15:41 | |
| *** rgogunskiy has quit IRC | 15:41 | |
| cloudnull | if you just want to create a static config without using the role | 15:41 |
| CeeMac | cool. thanks, i'll look at that | 15:41 |
| jrosser | this is a useful example of that role being used too https://github.com/openstack/openstack-ansible/blob/master/tests/roles/bootstrap-host/tasks/prepare_networking.yml#L16-L137 | 15:41 |
| CeeMac | i wanted to try and adopt netplan as "its the way forward" and I wanted to keep things vanilla as far as possible, but it seems to cause more issues than it fixes | 15:41 |
| CeeMac | jrosser, is that from the aio? | 15:42 |
| cloudnull | that said, if you need ifup/down on 1804 you can install the iftools with `apt install ifupdown` | 15:42 |
| cloudnull | though I would strongly suggest moving away from those if at all possible | 15:42 |
| jrosser | although i have to say networkd is not a silver bullet, you are at the mercy of the systemd version and there is a whole chunk of my life i will never get back from trying to make that work on bionic | 15:42 |
| cloudnull | ^ truth :) | 15:42 |
| CeeMac | haha | 15:43 |
| CeeMac | so I'm doomed basically, whatever way I go :D | 15:43 |
| jrosser | CeeMac: yes that is how the AIO sets up the networking | 15:43 |
| CeeMac | what is the most "standard" way of configuring interfaces? | 15:43 |
| jamesdenton | There isn't one, really. Varies by OS. The Ubuntu (reliable) way may be ifupdown for now, but i think networkd moving forward? | 15:44 |
| CeeMac | ah the joys of open source :) | 15:45 |
| CeeMac | its not a problem, its an opportunity right? | 15:45 |
| jamesdenton | perfect opportunity for you to create the next standard | 15:45 |
| *** markvoelker has joined #openstack-ansible | 15:46 | |
| CeeMac | there's already a macvlan. maybe i can create a macnet or something? | 15:46 |
| jrosser | if you want to get running as quickly as possible, probably ifupdown with the example configs from the openstack-ansible repo | 15:46 |
| jrosser | you can then play with these other things without messing up delivering your POC | 15:46 |
| CeeMac | jrosser, i'm pretty much up and running from a networking perspective now (as long as i don't reboot the network node and forget to bring the external interface up) | 15:47 |
| *** Emine has quit IRC | 15:47 | |
| CeeMac | I'll loop network configuration testing into my "deploy metal HA" phase | 15:48 |
| CeeMac | where I'll be putting in second/third hosts for everything | 15:48 |
| jrosser | no worries - pick what works :) I have a ton invested already in my PXEboot/provisioning stuff that uses ifupdown so i'm a bit biassed here anyway | 15:48 |
| CeeMac | haha | 15:48 |
| CeeMac | yeah, i bet | 15:48 |
| CeeMac | i still need to work on PXEboot/provisioning, but thats way down the line once I've got POC acceptable in the RC build | 15:49 |
| CeeMac | I'm close (hopefully) | 15:49 |
| jamesdenton | CeeMac Might be worth looking at that systemd unit file example i posted, so you can be sure the interface comes up on reboot. If you want to leave Netplan in place for now | 15:49 |
| CeeMac | still having some issues with journal-remote which i need to both cloudnull about | 15:49 |
| CeeMac | some weird errors with telemetry i've not looked at yet, then add in a couple 3rd party pieces | 15:50 |
| CeeMac | jamesdenton, yeah I'm going to take a look at that shortly | 15:50 |
| cloudnull | CeeMac what you seeing ? | 15:56 |
| CeeMac | hey | 15:57 |
| CeeMac | so, i applied that patch we discussed and it was working | 15:57 |
| CeeMac | then I did some other stuff and now its not | 15:58 |
| cloudnull | ah, the old did some stuff problem | 15:58 |
| cloudnull | :) | 15:58 |
| CeeMac | so, all of the hosts are getting unable to connect errors on port 19532 | 15:59 |
| CeeMac | looking on the log host, journal-remote isn't listening on that port | 15:59 |
| CeeMac | thats about as far as i got | 15:59 |
| cloudnull | ah is the process running ? | 15:59 |
| CeeMac | service is up | 15:59 |
| cloudnull | journal-upload and journal-remote | 15:59 |
| CeeMac | 1 sec, just logging back in to node | 16:00 |
| cloudnull | maybe the service died unexpectedly? | 16:00 |
| CeeMac | journal-upload runs on the log node too? | 16:00 |
| mnaser | cloudnull, DimGR, d34dh0r53, hughsaunders, b3rnard0, palendae, odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory, alextricity25, jasondotstar, admin0, michaelgugino, ametts, bgmccollum, darrenc, JRobinson__, colinmcnamara, thorst, adreznec, eil397, | 16:01 |
| mnaser | qwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen, ullbeking, mnaser, nicolasbock, jrosser, cjloader, antonym, dcdamien, jamesdenton | 16:01 |
| cloudnull | systemd-journal-upload runs on the logging node | 16:01 |
| mnaser | meeting time! | 16:01 |
| cloudnull | o . | 16:01 |
| mnaser | #startmeeting openstack_ansible_meeting | 16:01 |
| openstack | Meeting started Tue Feb 5 16:01:58 2019 UTC and is due to finish in 60 minutes. The chair is mnaser. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:02 |
| mnaser | #topic rollcall | 16:02 |
| *** openstack changes topic to " (Meeting topic: openstack_ansible_meeting)" | 16:02 | |
| cloudnull | o/ | 16:02 |
| openstack | The meeting name has been set to 'openstack_ansible_meeting' | 16:02 |
| *** openstack changes topic to "rollcall (Meeting topic: openstack_ansible_meeting)" | 16:02 | |
| cloudnull | presente! | 16:02 |
| jamesdenton | o/ | 16:02 |
| *** gyee has joined #openstack-ansible | 16:02 | |
| jrosser | o/ | 16:02 |
| prometheanfire | o/ | 16:03 |
| mnaser | #topic last week highlgihts | 16:05 |
| *** openstack changes topic to "last week highlgihts (Meeting topic: openstack_ansible_meeting)" | 16:05 | |
| mnaser | #topic last week highlights | 16:05 |
| *** openstack changes topic to "last week highlights (Meeting topic: openstack_ansible_meeting)" | 16:05 | |
| mnaser | https://review.openstack.org/#/q/projects:openstack/openstack-ansible+is:merged | 16:05 |
| mnaser | looking over merged code over the past week, looks like mostly master branch merges, a bunch of cool os_tempest work | 16:06 |
| mnaser | and some other stuff around nspawn | 16:06 |
| mnaser | anyone else want to highlight anything in specific this week? | 16:06 |
| jrosser | we are a little closer to aarch64, we got fixed packages for haproxy landed in bionic -proposed repo this week | 16:07 |
| jamesdenton | I'll prob be working on getting other deployment scenarios for MNAIO, esp OVS | 16:08 |
| mnaser | nice | 16:08 |
| mnaser | arm gating would be really cool, and thats' awesome, i should hack around mnaio at some point | 16:08 |
| jrosser | we need aarch64 wheels on the mirror - thats the next bigge | 16:08 |
| mnaser | yeah, i saw that convo over -infra | 16:08 |
| cloudnull | I recently just updated the plugins to pull in changes for config_template into our main plugins repo. is there a plan in place to switch over to that ? | 16:08 |
| cloudnull | https://review.openstack.org/#/c/634857/ | 16:09 |
| cloudnull | i remember evrardjp working on that a while back | 16:09 |
| jrosser | mnaser: should i prod ianw about that, is that the correct approach to move that aarch64 on? | 16:09 |
| cloudnull | however IDK what the state is currently ? | 16:09 |
| *** TxGirlGeek has joined #openstack-ansible | 16:09 | |
| mnaser | jrosser: i think he is the infra arm resident expert | 16:09 |
| jrosser | ok, cool, i will follow that up | 16:10 |
| mnaser | cloudnull: yes, i think we should get rid of that and use the one that's on it's own | 16:10 |
| cloudnull | ++ agreed | 16:10 |
| mnaser | it's a significant amount of work involved i think, updating all ansible-role-requirements across all roles before ripping it out | 16:11 |
| jrosser | iirc evrardjp and odyssey4me were cooking up a plan for how to do that withoug breaking everything | 16:11 |
| cloudnull | ^ ++ | 16:11 |
| mnaser | unless we actually jump to using integrated repo for testing, then it might be easier | 16:11 |
| mnaser | i think both aren't present at the moment so we may keep this to next week | 16:11 |
| cloudnull | I'm not sure what the best plan of attack is | 16:11 |
| openstackgerrit | James Denton proposed openstack/openstack-ansible-os_neutron stable/rocky: Fix the subprocess invocation of neutron-l3-agent https://review.openstack.org/634968 | 16:12 |
| mnaser | let's wait to hear from them | 16:13 |
| mnaser | also, another topic, i sent out an eamil about a bug squash slash hackday with zero response on the ML | 16:13 |
| mnaser | i'm wondering if folks just don't follow the ML as much (totally ok, there is a lot of email) or if it's more of a "not possible for us" | 16:13 |
| jamesdenton | I filter the ML and have a habit of not checking the folders | 16:14 |
| cloudnull | ^ | 16:14 |
| *** archklikk has joined #openstack-ansible | 16:14 | |
| jamesdenton | I am open to a bug squash and pretty flexible on date/time. | 16:14 |
| jrosser | i have some new people on my team and would be able to carve out a day for this | 16:14 |
| cloudnull | ++ | 16:15 |
| mnaser | no problem, is there any specific timeframe that works better for some folks in terms of a day of week? | 16:15 |
| mnaser | someone proposed a '24 hour' thing which seemed cool given how we're pretty distributed | 16:16 |
| jamesdenton | sounds good | 16:16 |
| jrosser | fridays are not so good for me | 16:16 |
| jamesdenton | Wednesday(s)? | 16:16 |
| archklikk | Hello guys, im trying to deploy an openstack build with ansible but i'm getting an error while running the "setup-hosts" playbook. At the "lxc_container_Create : Create container (dir)" task some of the containers are failing to start. One of them says "instantiate_veth: 172 Operation not permitted - Failed to attach "be909e25_eth1" to bridge "br-mgmt". What can be the problem? | 16:17 |
| mnaser | archklikk: we're in an openstack ansible meeting, we can look into this shortly after :) | 16:17 |
| mnaser | how does wednesday sound jrosser (and cloudnull maybe?) | 16:17 |
| * cloudnull checks cal | 16:18 | |
| jrosser | weds in general is fine | 16:18 |
| jrosser | tomorrow, specifically not so good :) | 16:18 |
| cloudnull | ^ | 16:18 |
| cloudnull | but wednessday/thursday is generally good | 16:18 |
| *** Emine has joined #openstack-ansible | 16:19 | |
| *** markvoelker has quit IRC | 16:19 | |
| mnaser | yeah that's already last minute | 16:20 |
| mnaser | okay, so i'll aim for wednesday next week for a bug squash day | 16:20 |
| mnaser | we can even discuss options of high bandwidth methods of communication | 16:20 |
| cloudnull | excellent | 16:21 |
| cloudnull | I have zoom available and can open it up to folks | 16:21 |
| mnaser | done and announced | 16:22 |
| mnaser | samesies | 16:22 |
| mnaser | cools | 16:23 |
| mnaser | next up, making meeting more like office hours, we brought it up last week and it feels like we're getting a bit more traction in our meetings with it that way | 16:23 |
| cloudnull | just informed my team that we're doing bug smash things next Wednesday :) | 16:24 |
| mnaser | yays | 16:24 |
| mnaser | any thoughts about the office hours rather than meeting | 16:25 |
| jamesdenton | not familiar with the office hours.... | 16:27 |
| jrosser | ^ same, not sure really what the choice is between | 16:27 |
| jrosser | or if we just need to iterate the format of the meeting in general | 16:27 |
| mnaser | office hours would be more like | 16:28 |
| mnaser | no specific agenda | 16:28 |
| mnaser | just a time where we all open in our calendars to be available at | 16:28 |
| mnaser | so we can discuss all sorts of stuff like this in a more sync manner | 16:28 |
| mnaser | office hours can be open for an entire hour, even if we're not really discussing much rather than a meeting that's "ok we're done" | 16:29 |
| jamesdenton | sure, i guess? The format doesn't make much difference to me as long as we get done what needs to get done | 16:30 |
| mnaser | alright, fair enough, we can discuss more in depth later.. | 16:31 |
| mnaser | (skipping bug triage because next week we'll clear all this stuff out) | 16:31 |
| mnaser | #topic open discussion | 16:31 |
| *** openstack changes topic to "open discussion (Meeting topic: openstack_ansible_meeting)" | 16:31 | |
| mnaser | anyone has anything to bring up? | 16:31 |
| prometheanfire | not really | 16:31 |
| prometheanfire | gentoo stuff will need another dib release to get the infra image fixed for the systemd bug, but that's it (and I'll bug them for it) | 16:32 |
| mnaser | woot | 16:32 |
| jrosser | how are we doing on the goals for Stein? | 16:32 |
| mnaser | then we can create our OSA distribution | 16:32 |
| mnaser | :) | 16:32 |
| mnaser | jrosser: are you talking about community goals or the things we aimed to accomplish discussed at the ptg | 16:32 |
| mnaser | ptg discussions are https://etherpad.openstack.org/p/osa-stein-ptg | 16:33 |
| jrosser | perhaps a bit of both, seems like there is probably more to do than we have effort for right now? | 16:33 |
| mnaser | i agree. so there's a few things | 16:33 |
| mnaser | Simplifing release: evrardjp picked this up and put up a few reviews but i think it's a wip that he's working on | 16:33 |
| mnaser | Moving towards nspawn: what's needed? -- i have been putting efforts but i have been blocked, i'm hoping for help from cloudnull on this, right now, ./run_tests.sh aio_nspawn will fail in tempest because the flat network doesnt work | 16:34 |
| mnaser | bridging a macvlan interface seems to work but it doesn't seem to be functional (aka, data doesnt land on the interface) | 16:34 |
| mnaser | so we either revert to using a normal bridge or use a dummy interface for the 'vlan' network (aka eth12 in old world) | 16:35 |
| mnaser | or i'd need a bit of help in getting the last bits of nspawn working in the gate | 16:35 |
| cloudnull | Im around today if we want to hack on that some ? | 16:35 |
| prometheanfire | mnaser: I've been targeting nspawn, after cloudnull's patch merged it seems to be working for me | 16:35 |
| *** openstackgerrit has quit IRC | 16:35 | |
| prometheanfire | but my test is just keystone | 16:36 |
| mnaser | yeah i'm working with the integrated repo | 16:36 |
| mnaser | it's more of a neutron/macvlan/bridge thing | 16:36 |
| mnaser | cloudnull: if you have time, i can show you my findings and leave the vm with you because i've exhausted my knowledge domain at least :< | 16:36 |
| mnaser | and we're very very very close to working upgrade jobs, https://review.openstack.org/#/c/627782/ fails in the second tempest in some workspace related thing that guilhermesp has been iterating on recently | 16:37 |
| prometheanfire | nice | 16:37 |
| mnaser | the upgrade does complete, just the tempest stuff seems to be acting weird, but yeah, it is great progress | 16:38 |
| *** alvinstarr has joined #openstack-ansible | 16:38 | |
| mnaser | i think those were the big 3 things for that we mainly talked about, unless i'm missing something | 16:38 |
| jrosser | mnaser: what would you like to land for S for nspawn, feels like we are behind what that describes | 16:39 |
| jrosser | ^what the etherpad describes | 16:39 |
| prometheanfire | mnaser: question about py3 switch, where is OSA there? | 16:39 |
| mnaser | jrosser: yes, we are, i've put in efforts a few times to get nspawn going but i keep getting blocked | 16:39 |
| prometheanfire | and ya, is nspawn default in S? | 16:39 |
| mnaser | i'd love to hit that goal if we can our gating working | 16:40 |
| mnaser | it looks like it's more of a gating issue rather than an nspawn issue given folks have been using it | 16:40 |
| mnaser | i'm hoping i can get some hacktime with cloudnull (or if anyone else can help, just spin up a vm and `./run_tests.sh aio_nspawn`) | 16:41 |
| cloudnull | I also have test VMs I can donate to the cause | 16:43 |
| cloudnull | and would be happy to join a hangout, zoom, tmux, to work through the issues | 16:43 |
| prometheanfire | always a helper | 16:43 |
| mnaser | i'll take you up on that later today if you're free cloudnull -- i think i'm really like 1 config option away from it i feel | 16:43 |
| cloudnull | ++ | 16:44 |
| * cloudnull gotta run to a meeting. bb in an hour | 16:44 | |
| mnaser | cool, i'll try to jump into figuring out the upgrade jobs more in depth as well and follow up with evrardjp about the release stuff | 16:44 |
| mnaser | anyone else have anything for discussion? | 16:44 |
| prometheanfire | the py3 question? | 16:45 |
| prometheanfire | 10:41 < prometheanfire > mnaser: question about py3 switch, where is OSA there? | 16:45 |
| mnaser | da heck | 16:45 |
| mnaser | i swear i wrote a reply to that | 16:45 |
| mnaser | prometheanfire: all projects were lobbed in there, we're likely not going to be doing that until our deployment targets supposed python3 well and natively | 16:46 |
| prometheanfire | kk | 16:46 |
| mnaser | we've switched most of our non-functional jobs to py3 like docs and so on, but eventually hopefully once rhel8 is out we can start doing all the python3 stuff | 16:46 |
| mnaser | so hopefully that works out for the T cycle cleanly | 16:46 |
| prometheanfire | ya, with py2 dying | 16:47 |
| mnaser | yep, kill it with fire | 16:47 |
| mnaser | also, having projects with py3 out for a cycle will help us make sure that everything is already working | 16:47 |
| mnaser | and with the work odyssey4me has put in recently it puts us in a good place with python_venv_build | 16:47 |
| prometheanfire | cool | 16:48 |
| mnaser | thanks for bringing that up prometheanfire | 16:48 |
| mnaser | anything else? :> | 16:48 |
| prometheanfire | non | 16:49 |
| jamesdenton | i'm good | 16:49 |
| guilhermesp | for the upgrade job there is a way suggested by odyssey4me for this https://review.openstack.org/#/c/627782/6 | 16:50 |
| guilhermesp | that is deploy an aio and run the gate-check script | 16:50 |
| guilhermesp | but I think I'd need someone of tempest close to help with this | 16:50 |
| mnaser | cool, we can chat over this as well in a bit with some of teh tempest experts :) | 16:51 |
| mnaser | thanks everyone | 16:51 |
| mnaser | #endmeeting | 16:51 |
| *** openstack changes topic to "Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: http://bit.ly/2xA1eZC" | 16:51 | |
| openstack | Meeting ended Tue Feb 5 16:51:14 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:51 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2019/openstack_ansible_meeting.2019-02-05-16.01.html | 16:51 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2019/openstack_ansible_meeting.2019-02-05-16.01.txt | 16:51 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2019/openstack_ansible_meeting.2019-02-05-16.01.log.html | 16:51 |
| *** hamzy has quit IRC | 16:51 | |
| archklikk | sooo, im trying to deploy an openstack build with ansible but i'm getting an error while running the "setup-hosts" playbook. At the "lxc_container_Create : Create container (dir)" task some of the containers are failing to start. One of them says "instantiate_veth: 172 Operation not permitted - Failed to attach "be909e25_eth1" to bridge "br-mgmt". What can be the problem? | 17:02 |
| *** nurdie has quit IRC | 17:02 | |
| *** sohny has joined #openstack-ansible | 17:04 | |
| *** nurdie has joined #openstack-ansible | 17:05 | |
| *** kopecmartin is now known as kopecmartin|off | 17:12 | |
| *** markvoelker has joined #openstack-ansible | 17:16 | |
| *** openstackgerrit has joined #openstack-ansible | 17:24 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_mistral master: Add initial Mistral role https://review.openstack.org/634991 | 17:24 |
| *** slaweq has quit IRC | 17:28 | |
| *** nurdie has quit IRC | 17:36 | |
| *** nurdie has joined #openstack-ansible | 17:38 | |
| *** nurdie has quit IRC | 17:47 | |
| *** nurdie has joined #openstack-ansible | 17:48 | |
| *** markvoelker has quit IRC | 17:48 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_mistral master: Add initial Mistral role https://review.openstack.org/634991 | 18:01 |
| *** sdake has quit IRC | 18:11 | |
| *** hamzy has joined #openstack-ansible | 18:12 | |
| *** gillesMo has quit IRC | 18:23 | |
| *** sohny has quit IRC | 18:26 | |
| *** hamzaachi has joined #openstack-ansible | 18:33 | |
| *** DanyC has quit IRC | 18:36 | |
| *** DanyC has joined #openstack-ansible | 18:36 | |
| *** DanyC has quit IRC | 18:41 | |
| *** Bico_Fino has joined #openstack-ansible | 18:43 | |
| *** markvoelker has joined #openstack-ansible | 18:46 | |
| *** cmart has quit IRC | 18:47 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-tests master: Add mistral_install_method for distro installations https://review.openstack.org/635007 | 18:52 |
| *** sdake has joined #openstack-ansible | 18:53 | |
| *** macza has joined #openstack-ansible | 18:55 | |
| *** DanyC has joined #openstack-ansible | 19:00 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_mistral master: Add initial Mistral role https://review.openstack.org/634991 | 19:02 |
| *** shardy has quit IRC | 19:03 | |
| *** DanyC has quit IRC | 19:04 | |
| *** vnogin has joined #openstack-ansible | 19:19 | |
| *** markvoelker has quit IRC | 19:19 | |
| *** vnogin has quit IRC | 19:23 | |
| *** schwicht has quit IRC | 19:26 | |
| *** pcaruana has quit IRC | 19:27 | |
| openstackgerrit | James Denton proposed openstack/openstack-ansible master: [docs] Apply provider network config on per-group basis https://review.openstack.org/635013 | 19:30 |
| *** cmart has joined #openstack-ansible | 19:45 | |
| *** asettle has quit IRC | 19:51 | |
| mnaser | the mistral ansible role seems to be passing everything.. except centos-7 | 19:58 |
| mnaser | http://logs.openstack.org/91/634991/3/check/openstack-ansible-functional-centos-7/7586479/logs/ara-report/ | 19:58 |
| mnaser | for SOME reason.. the tables seem to have been already created, even though you can see the task before that the db was created | 19:58 |
| openstackgerrit | James Denton proposed openstack/openstack-ansible master: [docs] Apply provider network config on per-group basis https://review.openstack.org/635013 | 20:01 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_mistral master: Add initial Mistral role https://review.openstack.org/634991 | 20:14 |
| *** markvoelker has joined #openstack-ansible | 20:16 | |
| *** markvoelker has quit IRC | 20:48 | |
| *** KenCrandall has joined #openstack-ansible | 20:53 | |
| *** priteau has joined #openstack-ansible | 20:54 | |
| *** KenCrandall has quit IRC | 20:59 | |
| *** KenCrandall has joined #openstack-ansible | 21:00 | |
| *** KenCrandall has left #openstack-ansible | 21:04 | |
| *** KenCrandall has joined #openstack-ansible | 21:04 | |
| *** cmart has quit IRC | 21:08 | |
| *** cmart has joined #openstack-ansible | 21:08 | |
| *** priteau has quit IRC | 21:12 | |
| *** priteau has joined #openstack-ansible | 21:14 | |
| cloudnull | mnaser nice ! | 21:15 |
| * cloudnull had to go look up what mistral does | 21:15 | |
| cloudnull | archklikk did you make it go ? | 21:19 |
| cloudnull | I'd check if br-mgmt exists | 21:19 |
| archklikk | cloudnull: yes i could, but i got an other issue after when it wanted to start systemd-networkd. I don't know why but I don't have the right interfaces. Afaik, ansible should make them, am I right? | 21:24 |
| archklikk | I mean, it wanted to restart systemd-networkd | 21:25 |
| jamesdenton | OSA makes them for an AIO, but not a "prod" deployment | 21:25 |
| jamesdenton | you would need to configure those bridges by hand | 21:25 |
| mnaser | cloudnull: for some SUPER weird reason, it seems to fail with a table already eixsting when it runs the db sync | 21:25 |
| archklikk | Oh, I see. I only used the AIO version before :D | 21:26 |
| mnaser | i just ran it in my own centos vm and it didnt fail :\ | 21:26 |
| mnaser | (and the funniest part is that i developed all this against centos, all the other operating systems worked in ci right away.. except centos which i tested working lol) | 21:26 |
| cloudnull | archklikk yea. in the AIO we create all of the needed interfaces | 21:28 |
| jrosser | mnaser: i've been having all sorts of similar centos grief | 21:28 |
| cloudnull | when you go prod you need to build those for yoru environment. | 21:28 |
| jrosser | trying to debug some gate fails and i just can't get it to break the same way in a local vm | 21:28 |
| mnaser | jrosser: you can ask infra to hold a failed vm if you want | 21:29 |
| mnaser | they can setup a hold so instead of deleting the vmw hen the job fails, it gets held | 21:29 |
| mnaser | give them your keys and you can ssh into it :) | 21:29 |
| mnaser | (usually you need some sort of "hey there's a reason why i'd like to do this") -- aka see #openstack-infra now -- that might be helpful | 21:29 |
| jrosser | sure | 21:30 |
| mnaser | cloudnull: got a few secs to show off how things are stuck right now in nspawn? | 21:30 |
| cloudnull | sure | 21:32 |
| mnaser | cloudnull: mind hopping on https://zoom.us/j/909856631 ? | 21:35 |
| mnaser | anyone else who wants to see how/why nspawn isnt working is welcome | 21:36 |
| mnaser | :> | 21:36 |
| *** ansmith_ has quit IRC | 21:39 | |
| *** sdake has quit IRC | 21:39 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_mistral master: Add initial Mistral role https://review.openstack.org/634991 | 21:42 |
| *** markvoelker has joined #openstack-ansible | 21:46 | |
| *** hamzy has quit IRC | 21:46 | |
| *** nurdie_ has joined #openstack-ansible | 21:50 | |
| prometheanfire | setup-infra failing in rocky on 'Apply resolver sorting' VERY early https://pasted.tech/pastes/fa3b35736c446d00318b8545e1c47c9d82a1d651.raw | 21:53 |
| prometheanfire | not sure if others have seen it | 21:53 |
| *** nurdie has quit IRC | 21:54 | |
| prometheanfire | https://www.youtube.com/watch?v=k4vIVY-6O00 | 21:54 |
| jamesdenton | Looks like you have an extra \ in there | 21:54 |
| prometheanfire | heh, oops | 21:54 |
| prometheanfire | jamesdenton: extra \ ? | 21:54 |
| jamesdenton | j/k. gobblygook termcap stuffs | 21:55 |
| *** nurdie_ has quit IRC | 21:55 | |
| prometheanfire | yep | 21:55 |
| *** cmart has quit IRC | 21:56 | |
| prometheanfire | https://paste.pound-python.org/show/EfVfZEGFgV9LisqsFH1R/ is a bit cleaner | 21:59 |
| jrosser | is that in your environment vars? | 21:59 |
| prometheanfire | that? | 21:59 |
| jrosser | the termcap entry | 22:00 |
| prometheanfire | doesn't look like it | 22:00 |
| jrosser | we've seen this a few times - i've never encountered it to dig though | 22:00 |
| prometheanfire | hmm it is a fact though | 22:01 |
| jrosser | but somewhere the termcap contents ends up in a var which is then templated | 22:01 |
| jrosser | and all sorts of brok then occurs | 22:01 |
| prometheanfire | https://gist.github.com/prometheanfire/f1b64b50d1dbbb3223ade836dcc24615 | 22:01 |
| * prometheanfire needs to see how many paste services he can use in one sitting | 22:01 | |
| *** hwoarang has joined #openstack-ansible | 22:02 | |
| * prometheanfire is going to just kill the cached facts and see what happens | 22:02 | |
| jrosser | interestingly my ansible_facts/localhost has "TERM": "xterm-256color" | 22:03 |
| jrosser | but not TERMCAP, i wonder if it is an artefact of screen? | 22:03 |
| prometheanfire | I'm guessing | 22:03 |
| prometheanfire | our instructions state to use screen | 22:04 |
| prometheanfire | TERMCAP=SC|screen.xterm-256color|VT 100/ANSI X3.64 virtual terminal:\ | 22:04 |
| prometheanfire | that's what I have on a fresh screen | 22:04 |
| prometheanfire | the install unbound stuff happens before gathering facts | 22:06 |
| prometheanfire | removed facts as well, wfm for now | 22:08 |
| prometheanfire | cjloader: seen the pastes above? ^ | 22:08 |
| prometheanfire | last 15 min | 22:08 |
| prometheanfire | also, screen > tmux | 22:11 |
| mnaser | ive seen that ahppen before with screen | 22:12 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: nspawn: use correct network range for tempest https://review.openstack.org/635059 | 22:14 |
| * prometheanfire watches | 22:14 | |
| *** markvoelker has quit IRC | 22:20 | |
| *** cmart has joined #openstack-ansible | 22:20 | |
| *** DanyC has joined #openstack-ansible | 22:22 | |
| prometheanfire | heh, had it happen again | 22:25 |
| *** cmart has quit IRC | 22:27 | |
| *** sdake has joined #openstack-ansible | 22:27 | |
| *** sdake has quit IRC | 22:34 | |
| openstackgerrit | Merged openstack/ansible-config_template master: Correct "changed" to work with the latest ansible https://review.openstack.org/634856 | 22:36 |
| *** sdake has joined #openstack-ansible | 22:36 | |
| *** DanyC has quit IRC | 22:41 | |
| *** DanyC has joined #openstack-ansible | 22:42 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: nspawn: setup basic networking structure https://review.openstack.org/635067 | 22:47 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: nspawn: enable jobs https://review.openstack.org/635068 | 22:47 |
| *** cmart has joined #openstack-ansible | 22:49 | |
| *** priteau has quit IRC | 22:52 | |
| *** cmart has quit IRC | 23:07 | |
| *** markvoelker has joined #openstack-ansible | 23:16 | |
| archklikk | Now it's crashing at "Update package cache (apt)" :D | 23:21 |
| archklikk | can it be because of bad network config? | 23:21 |
| *** ansmith_ has joined #openstack-ansible | 23:31 | |
| redkrieg | Hello all, I've been having trouble with setting up ml2.ovs. the neutron-openvswitch-agent.log on the compute node indicates that my bridge does not exist, but the bridge does show up under brctl show. If anyone has time, I've put together what I think are the relevant config lines and logs: http://paste.openstack.org/show/744597/ | 23:32 |
| *** cmart has joined #openstack-ansible | 23:36 | |
| *** markvoelker has quit IRC | 23:50 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!