| *** spatel has quit IRC | 00:03 | |
| *** aedc has joined #openstack-ansible | 00:11 | |
| *** tosky has quit IRC | 00:12 | |
| *** hamzy has joined #openstack-ansible | 00:44 | |
| *** nowster has quit IRC | 00:50 | |
| *** cshen has joined #openstack-ansible | 00:51 | |
| *** cshen has quit IRC | 00:55 | |
| *** hw_wutianwei_ has joined #openstack-ansible | 01:11 | |
| *** klamath has quit IRC | 01:38 | |
| *** klamath has joined #openstack-ansible | 01:41 | |
| *** mino_ has joined #openstack-ansible | 03:10 | |
| *** maddtux has joined #openstack-ansible | 03:29 | |
| *** udesale has joined #openstack-ansible | 04:17 | |
| *** mmercer has quit IRC | 04:27 | |
| *** cshen has joined #openstack-ansible | 04:45 | |
| *** cshen has quit IRC | 04:50 | |
| *** chandankumar has joined #openstack-ansible | 05:36 | |
| *** chandankumar is now known as chkumar|ruck | 05:37 | |
| openstackgerrit | Chandan Kumar proposed openstack/openstack-ansible-os_tempest master: Added task to list tempest tests https://review.openstack.org/619024 | 06:41 |
|---|---|---|
| *** ahosam has joined #openstack-ansible | 07:14 | |
| *** cshen has joined #openstack-ansible | 07:18 | |
| *** gkadam has joined #openstack-ansible | 07:25 | |
| *** DanyC has joined #openstack-ansible | 07:30 | |
| *** DanyC has joined #openstack-ansible | 07:31 | |
| *** DanyC has quit IRC | 07:44 | |
| *** DanyC_ has joined #openstack-ansible | 07:44 | |
| *** DanyC_ has quit IRC | 07:49 | |
| *** hamzaachi has joined #openstack-ansible | 08:06 | |
| *** rpittau has joined #openstack-ansible | 08:19 | |
| cshen | I need some review for my patch https://review.openstack.org/#/c/581248/ , thanks. | 08:27 |
| *** shardy has joined #openstack-ansible | 08:35 | |
| *** shardy has quit IRC | 08:39 | |
| *** eumel8 has joined #openstack-ansible | 08:44 | |
| *** mino_ has quit IRC | 08:50 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-ops master: MNAIO: Add ability to deploy Ceph https://review.openstack.org/607338 | 08:50 |
| *** tosky has joined #openstack-ansible | 08:57 | |
| *** hamzaachi has quit IRC | 09:00 | |
| Miouge | I’m geting errors like “fatal: repository 'https://git.openstack.org/cgit/openstack/neutron-vpnaas-dashboard/' not found” in CI, it sounds completely unrelated to my patch, so I wonder what is going on: https://review.openstack.org/619092 | 09:02 |
| *** dcdamien has joined #openstack-ansible | 09:07 | |
| odyssey4me | Miouge oops, wrong URL in https://review.openstack.org/#/c/617847/ | 09:07 |
| odyssey4me | cgit should not be in the URL :/ | 09:08 |
| odyssey4me | would you like to push up a fix, or shall I? | 09:08 |
| Miouge | odyssey4me: cool! Let me give it a shot | 09:08 |
| odyssey4me | thanks :) | 09:09 |
| dcdamien | odyssey4me: my bad - I'll fix it | 09:09 |
| odyssey4me | dcdamien Perhaps give Miouge a shot at it, then you can review the patch. :) | 09:10 |
| dcdamien | kk | 09:11 |
| Miouge | I wonder why it was not caught by the CI the the openstack-ansible-os_horizon repo? | 09:12 |
| openstackgerrit | Maxime Guyot proposed openstack/openstack-ansible-os_horizon master: Fix the vpnaas-dashboard git repo URL https://review.openstack.org/619179 | 09:13 |
| Miouge | here you go dcdamien and odyssey4me that URL works with cloning locally ^ | 09:16 |
| *** chkumar|ruck has quit IRC | 09:18 | |
| odyssey4me | Miouge yep, thanks! :) | 09:18 |
| *** chkumar|ruck has joined #openstack-ansible | 09:18 | |
| *** hamzaachi has joined #openstack-ansible | 09:25 | |
| *** DanyC has joined #openstack-ansible | 09:26 | |
| *** DanyC has quit IRC | 09:28 | |
| *** DanyC has joined #openstack-ansible | 09:29 | |
| *** cshen has quit IRC | 09:35 | |
| *** ahosam has quit IRC | 09:36 | |
| *** ahosam has joined #openstack-ansible | 09:36 | |
| *** ahosam has quit IRC | 09:40 | |
| openstackgerrit | Maxime Guyot proposed openstack/openstack-ansible master: Add a check that keepalived interfaces have IPs https://review.openstack.org/619092 | 09:42 |
| *** hamzaachi_ has joined #openstack-ansible | 09:45 | |
| *** hamzaachi has quit IRC | 09:46 | |
| *** pcaruana has joined #openstack-ansible | 09:48 | |
| *** hamzaachi_ has quit IRC | 09:50 | |
| *** shardy has joined #openstack-ansible | 09:51 | |
| *** hamzaachi has joined #openstack-ansible | 09:53 | |
| *** hamzaachi has quit IRC | 09:54 | |
| evrardjp | hola | 09:54 |
| *** hamzaachi has joined #openstack-ansible | 09:55 | |
| *** hamzaachi_ has joined #openstack-ansible | 09:57 | |
| *** hamzaachi has quit IRC | 09:57 | |
| *** hamzaachi_ has quit IRC | 09:58 | |
| *** hamzaachi has joined #openstack-ansible | 10:00 | |
| *** cshen has joined #openstack-ansible | 10:01 | |
| dcdamien | +1 Miouge I think we should figure out why such as fail as setting incorrect git URL was not caught in CI run related to change that instroduced it. | 10:03 |
| *** ahosam has joined #openstack-ansible | 10:03 | |
| Miouge | dcdamien: I can’t find any “vpnass” in http://logs.openstack.org/47/617847/6/check/openstack-ansible-functional-ubuntu-bionic/6273c37/job-output.txt.gz so I suppose that it didn’t get tested | 10:04 |
| *** hamzaachi has quit IRC | 10:05 | |
| dcdamien | shouldn't we enable this dashboard in AIO to get it tested in CI? | 10:06 |
| odyssey4me | dcdamien none of those are used in the CI test - only the basic dashboard is setup | 10:06 |
| *** cshen has quit IRC | 10:07 | |
| dcdamien | is there any reason to not test all of them? | 10:07 |
| odyssey4me | dcdamien I don't think anyone's tried. :) | 10:07 |
| Miouge | I think that openstack-ansible CI tried to clone it but openstack-ansible-os_horizon CI didn’t | 10:08 |
| odyssey4me | With the basic horizon setup in the role, we had to add a nova service catalog entry for it to work - adding the extra dashboards may also require similar things to get them to allow horizon to start properly. It'd be nice to have them tested in CI. :) | 10:09 |
| odyssey4me | dcdamien also note that for every _git_repo entry in the role, there should be a corresponding SHA pin in the integrated repo- otherwise the py_pkgs lookup will always grab master and the integrated build will break | 10:09 |
| *** Emine has joined #openstack-ansible | 10:10 | |
| dcdamien | odyssey4me: isn't nova service catalog entry just required to use (access) dashboards and not required to deploy them? | 10:15 |
| *** nowster has joined #openstack-ansible | 10:19 | |
| odyssey4me | dcdamien yes, but the CI test actually runs tempest against horizon - I think all that does is verify that it loads and can be logged into | 10:20 |
| dcdamien | ok, now I got the point | 10:25 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-ops master: MNAIO: Add ability to deploy Ceph https://review.openstack.org/607338 | 10:28 |
| *** Emine has quit IRC | 10:31 | |
| *** priteau has joined #openstack-ansible | 10:37 | |
| *** hamzaachi has joined #openstack-ansible | 10:38 | |
| nowster | Hmm... | 10:43 |
| *** ahosam has quit IRC | 10:43 | |
| nowster | $ git clone https://git.openstack.org/cgit/openstack/neutron-vpnaas-dashboard/ | 10:43 |
| nowster | Cloning into 'neutron-vpnaas-dashboard'... | 10:43 |
| nowster | fatal: repository 'https://git.openstack.org/cgit/openstack/neutron-vpnaas-dashboard/' not found | 10:43 |
| odyssey4me | nowster https://review.openstack.org/619179 will fix that | 10:45 |
| nowster | thanks | 10:45 |
| odyssey4me | jrosser hwoarang evrardjp you around? we need another review for https://review.openstack.org/619179 to unblock the integrated gate | 10:46 |
| nowster | jrosser is in a meeting at the moment | 10:47 |
| *** ahosam has joined #openstack-ansible | 10:51 | |
| *** shardy has quit IRC | 10:52 | |
| *** shardy has joined #openstack-ansible | 10:53 | |
| evrardjp | odyssey4me: I missed it :( | 10:53 |
| evrardjp | my bad | 10:54 |
| *** Emine has joined #openstack-ansible | 10:54 | |
| odyssey4me | evrardjp yeah, me too :/ | 10:54 |
| *** ahosam has quit IRC | 10:55 | |
| *** fghaas has joined #openstack-ansible | 10:58 | |
| *** cshen has joined #openstack-ansible | 11:02 | |
| mnaser | bonjour | 11:03 |
| jrosser | o/ | 11:05 |
| mnaser | odyssey4me: do we want to start adding nspawn jobs inside our gates? | 11:07 |
| odyssey4me | mnaser I dunno if the existing integrated build test is passing yet on master - I'd think that's an initial target. | 11:07 |
| odyssey4me | You were working on that some time ago? | 11:07 |
| odyssey4me | My backports to rocky were only to get the nspawn_hosts/nspawn_container_create roles up to speed with master. | 11:08 |
| mnaser | odyssey4me: https://review.openstack.org/#/c/608433/ i have this but i can respin if someone has done some iterations on that since | 11:08 |
| odyssey4me | mnaser I think you can abandon that and start fresh again, because I think much of that has been done and merged. | 11:11 |
| odyssey4me | I guess I could push a patch up to bring it into the check queue and see what happens. | 11:12 |
| odyssey4me | lemme do that | 11:12 |
| fghaas | odyssey4me, another follow-up from last week. You'll recall we spoke briefly about our default haproxy connection timeouts being too conservative for things like openstack stack event list --follow, where the client keeps a connection open essentially indefinitely (until interrupted by SIGINT or, rather rudely, by an upstream haproxy as in our case). I am beginning to wonder whether or not that might affect other services as w | 11:13 |
| fghaas | ell. In particular, I'm not sure if heat-engine doesn't try to pipeline many (all?) of its requests to a particular API endpoint over a single TCP connection (I haven't looked at the code, because I don't even know where I'd start looking). If that's the case, then the 50s timeout is _definitely_ too conservative. Have you run into any of that before? | 11:13 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: Enable passing nspawn jobs https://review.openstack.org/608433 | 11:13 |
| mnaser | odyssey4me: oops i was alt tabbed ^ | 11:13 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: [WIP] Switch ubuntu/nspawn/source jobs to voting https://review.openstack.org/619211 | 11:15 |
| odyssey4me | mnaser ah ok, well - keep going with that one, let's see what happens | 11:17 |
| odyssey4me | I'll abandon mine. | 11:17 |
| mnaser | odyssey4me: cool, i was thinking that if we get somewhat functional nspawn jobs, we can actually look into using integrated role for our test jobs using nspawn | 11:19 |
| mnaser | so all role testing uses integrated using nspawn | 11:20 |
| mnaser | which will exercise a lot more 'testing' on our nspawn components and we keep lxc in integrated | 11:20 |
| odyssey4me | mnaser yeah, it might be good to do that when we switch - I need to find some time to get back to getting the changes to the aio bootstrap done to allow us to switch to using the integrated build for everything | 11:21 |
| odyssey4me | I think I can squeeze in some time this week. | 11:21 |
| mnaser | odyssey4me: i've been a bit swamped personally but i'm happy with the good progress we're doing in os_placement which will keep us up to date. | 11:22 |
| mnaser | on a totally seperate note i've been looking to add new people who would be working on OSA so hopefully that'll give us more help here too | 11:22 |
| mnaser | but that takes forever :) | 11:22 |
| mnaser | (but hopefully takes a bit of load off of everyone) | 11:22 |
| odyssey4me | mnaser thanks - the more climb in to help, the easier it is for everyone | 11:23 |
| odyssey4me | and no contribution is too small to spread the load | 11:23 |
| *** hamzaachi has quit IRC | 11:23 | |
| mnaser | yup, i think right now we have a lot of big heavy lifting ahead of us | 11:24 |
| odyssey4me | we may have to pin our nova role once the switch is made in upstream nova, so that our role tests can continue until we have our placement role done | 11:24 |
| mnaser | esp with the change to nspawn and figuring out that whole thing | 11:24 |
| mnaser | odyssey4me: placement is not being removed until one deployment tool adds it | 11:24 |
| odyssey4me | ah ok, heh, it'd be good to be that tool ;) | 11:25 |
| mnaser | tripleo is working on it and so are we, so hopefully we can have it done. let me share a small doc | 11:25 |
| mnaser | https://etherpad.openstack.org/p/BER-placement-extract | 11:25 |
| mnaser | both existing in deployment tool *with* upgrade jobs.. so yeah | 11:25 |
| odyssey4me | ah, yes - well - with the role in place and working, and integrated, we can focus on doing the upgrade thing | 11:26 |
| mnaser | part of the placement upgrade is like | 11:26 |
| mnaser | shutting down old placement service | 11:26 |
| mnaser | export db into a new one.. | 11:27 |
| mnaser | and start it back up | 11:27 |
| * mnaser is terrified already | 11:27 | |
| mnaser | the placement team has given us a bash script which is noop if it's already done | 11:27 |
| odyssey4me | yep, but let's get greenfield done - then we can work out the best way to migrate | 11:27 |
| mnaser | yep | 11:27 |
| mnaser | https://github.com/openstack/placement/blob/master/tools/mysql-migrate-db.sh | 11:27 |
| mnaser | fwiw | 11:27 |
| odyssey4me | ok, the bash script will be a good reference | 11:27 |
| mnaser | yup :) | 11:28 |
| openstackgerrit | Merged openstack/openstack-ansible-nspawn_container_create stable/rocky: Add missing user defined interface support https://review.openstack.org/616164 | 11:28 |
| *** udesale has quit IRC | 11:28 | |
| noonedeadpunk | Morning everyone. | 11:30 |
| noonedeadpunk | I'd appreciate if someone can take a look into https://review.openstack.org/#/c/619058/ | 11:30 |
| mnaser | noonedeadpunk: small nit | 11:31 |
| noonedeadpunk | mnaser: To be honest, I've placed patch just after https://review.openstack.org/#/c/617847/ being merged, so mentioned patch wasn't placed at that time | 11:33 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [DNM] Test arm64 https://review.openstack.org/618305 | 11:33 |
| mnaser | noonedeadpunk: no worries :D if you can just drop that part and add depends-on or things will be in merge conflict | 11:33 |
| noonedeadpunk | I just thought, that the first placed patch should be more valuable, but ok, I'll remove this part from mine | 11:35 |
| openstackgerrit | Merged openstack/openstack-ansible-nspawn_container_create stable/rocky: Add a guard so we don't allow for duplicate config https://review.openstack.org/616165 | 11:35 |
| openstackgerrit | Dmitriy Rabotjagov (noonedeadpunk) proposed openstack/openstack-ansible-os_horizon master: Implements masakari-dashboard to be installed in horizon https://review.openstack.org/619058 | 11:36 |
| *** cshen has quit IRC | 11:36 | |
| *** cshen has joined #openstack-ansible | 11:39 | |
| evrardjp | fghaas: haproxy? | 11:41 |
| evrardjp | you can have sticky tables if you like | 11:41 |
| evrardjp | in haproxy | 11:41 |
| evrardjp | not sure if the role is ready for that though | 11:41 |
| odyssey4me | fghaas last I ran into trouble with that was a very long time ago in essex, but I don't operate a cloud any more - so perhaps better to ask those who do on a day to day basis, like mnaser logan- jrosser | 11:43 |
| *** cshen has quit IRC | 11:44 | |
| mnaser | heat-api talks to heat-engine via rpc | 11:44 |
| mnaser | i have noticed the issues mentioned about stack event indeed | 11:44 |
| openstackgerrit | Merged openstack/openstack-ansible-nspawn_container_create stable/rocky: Add test for host_only networks and ensure network ordering https://review.openstack.org/616163 | 11:45 |
| jamesdenton | good morning | 11:45 |
| *** hamzaachi has joined #openstack-ansible | 11:46 | |
| *** mugsie has joined #openstack-ansible | 11:48 | |
| *** cshen has joined #openstack-ansible | 11:50 | |
| openstackgerrit | Dmitriy Rabotjagov (noonedeadpunk) proposed openstack/openstack-ansible-os_horizon master: Implements masakari-dashboard to be installed in horizon https://review.openstack.org/619058 | 11:51 |
| openstackgerrit | Kaio Kassiano Moura Oliveira proposed openstack/openstack-ansible-os_monasca-ui master: Retire openstack-ansible-os_monasca-ui https://review.openstack.org/617204 | 11:52 |
| fghaas | evrardjp: I don't follow. How do sticky tables help with enforced timeouts? | 11:53 |
| openstackgerrit | Merged openstack/openstack-ansible-nspawn_hosts stable/rocky: Install the timezone package by default https://review.openstack.org/616150 | 11:53 |
| fghaas | mnaser: yes I know that heat-api talks to heat-engine via RPC, but the question is whether heat-engine pipelines its calls to *other* APIs | 11:54 |
| evrardjp | fghaas: it would not help on the timeouts | 11:54 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Remove unnecessary octavia scenario AIO bootstrap https://review.openstack.org/619218 | 11:54 |
| jrosser | odyssey4me: should we add that tz package ^^ to my stripped down lxc_hosts packages patch? | 11:54 |
| evrardjp | fghaas: but the client would always target the same backend nodes, so I suppose it's not really a problem anymore, as connections would be reused? | 11:54 |
| evrardjp | fghaas: I am just trying to understand the root cause :) | 11:54 |
| fghaas | evrardjp: no, the problem is haproxy _actively killing_ the connection | 11:55 |
| evrardjp | oh I see | 11:55 |
| mnaser | fghaas: even if heat pipelined calls, no calls to openstack actually take 50s with nothing happening | 11:55 |
| mnaser | except for glance in weird scenarios | 11:55 |
| evrardjp | And it's from heat-engine, which you mean internally, right? | 11:55 |
| mnaser | where it buffers all the upload stuff then pushes it out after | 11:55 |
| guilhermesp | morning | 11:55 |
| mnaser | o/ guilhermesp | 11:56 |
| evrardjp | mnaser: I am not sure we need to be agressive in the killing of connections between internal endpoints | 11:56 |
| mnaser | we dont have to be aggressive, but i think 50s is a very reasonable timeout for api services not responding | 11:57 |
| evrardjp | but that 50 seconds thing is interesting indeed, why does the call take so long | 11:57 |
| evrardjp | mnaser: agreed | 11:57 |
| evrardjp | so maybe there is an exceptional case? | 11:57 |
| mnaser | heat stack event is this unique "streaming" like user facing api | 11:57 |
| mnaser | glance has always been an exception in wanting big timeouts | 11:57 |
| mnaser | maybe heat is another one now | 11:57 |
| odyssey4me | jrosser you mean remove it in your strip-down patch? I'm not sure I understand the question - although I think the timezone package is likely one of those which should be done in the base, because it's required for the copy of the timezone config from the host to the container to work | 11:58 |
| evrardjp | good morning jamesdenton | 12:00 |
| jrosser | odyssey4me: well oddly it looks like it was never there in the lxc_hosts package list, so i figured that if it was necessary in the nspawn base packages then it probably should also be necessary in the lxc_hosts list | 12:00 |
| odyssey4me | jrosser yep, makes sense to me | 12:00 |
| jrosser | ok, i'll add it to make things match up with nspawn | 12:00 |
| evrardjp | kaiokmo: thanks for the work on retiring :) | 12:00 |
| evrardjp | kaiokmo: want to retire more things, like freezer? :D | 12:01 |
| evrardjp | you're now used to it, so it's good :) | 12:01 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Remove unnecessary octavia scenario AIO bootstrap https://review.openstack.org/619218 | 12:01 |
| kaiokmo | evrardjp: no problem. I thought freezer was already retired | 12:02 |
| evrardjp | kaiokmo: let me double check :) | 12:02 |
| evrardjp | I think there are a few ones not fully retired yet | 12:03 |
| evrardjp | https://github.com/openstack/governance/blob/master/reference/projects.yaml#L2605-L2607 and https://github.com/openstack/governance/blob/master/reference/projects.yaml#L2614-L2616 could probably be retired. https://github.com/openstack/governance/blob/master/reference/projects.yaml#L2617-L2619 is a different story, but we can try, as I have changed releases to make that work | 12:04 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Minimise distro packages installed into the lxc image https://review.openstack.org/613078 | 12:04 |
| evrardjp | if you want I can do that | 12:04 |
| evrardjp | but I thought it's good to exercice your newly acquired skills :) | 12:04 |
| fghaas | evrardjp: the way you see this is, for example, on an `openstack stack <operation> --wait` call, which will just die with "Connection aborted.', BadStatusLine("''",))" | 12:05 |
| evrardjp | oh yeah the wait bit! | 12:06 |
| evrardjp | mmm | 12:06 |
| evrardjp | I didn't get that recently though, so I am curious | 12:06 |
| evrardjp | which version is that? | 12:06 |
| fghaas | But the problem is, it'll do this far sooner than 50s after the latest thing it sent | 12:06 |
| evrardjp | Rocky? | 12:06 |
| fghaas | Yes | 12:06 |
| evrardjp | that might explain why I haven't seen it | 12:06 |
| evrardjp | I am still lagging behind there | 12:07 |
| fghaas | I have an output in front of me where I see the prior event entry at 12:04:13Z, and then at 12:04:30Z I see the drop. | 12:07 |
| fghaas | No, this has been the case in Queens as well. Probably Pike too. | 12:07 |
| fghaas | So while I agree with mnaser that no OpenStack API call should take 50s with nothing happening, there clearly _are_ things happening here but the connection still gets killed. | 12:08 |
| evrardjp | fghaas: I have done a few times this kind of stacks with wait in Q and didn't get the issue. | 12:08 |
| fghaas | I see this in all our OSA deployments, whereas our legacy RDO stuff (which uses nginx, not haproxy) doesn't have this issue. | 12:08 |
| evrardjp | more interesting : ) | 12:08 |
| fghaas | How complex were your stacks though? | 12:09 |
| *** hamzaachi has quit IRC | 12:10 | |
| evrardjp | I recently did a deploy of 4 nodes k8s cluster with auto enrollment of nodes | 12:10 |
| *** hamzaachi_ has joined #openstack-ansible | 12:10 | |
| evrardjp | so kinda complex | 12:10 |
| evrardjp | (yes I am not using magnum :p) | 12:11 |
| evrardjp | but still | 12:11 |
| evrardjp | I have weird environment though, it's not really haproxy you generate from the OSA plays | 12:12 |
| evrardjp | so that might be it | 12:12 |
| jamesdenton | odyssey4me was another bug or patch opened to fix the URL here? https://review.openstack.org/#/c/617847/6/defaults/main.yml | 12:19 |
| odyssey4me | jamesdenton yep: https://review.openstack.org/619179 | 12:20 |
| jonher | gate is running as we speak | 12:20 |
| jamesdenton | great, thanks! | 12:21 |
| mnaser | jrosser: odyssey4me evrardjp can i get https://review.openstack.org/#/c/615895/ and https://review.openstack.org/#/c/615896/ some love? it's been around for 2 sad weeks :( | 12:22 |
| mnaser | and i need to do backport to rocky and queens sooOo | 12:22 |
| odyssey4me | mnaser done | 12:25 |
| mnaser | jrosser: mind looking at the second patch in there? :> | 12:28 |
| Miouge | fghaas: would something like that http://paste.openstack.org/show/735888/ reproduce the problem? | 12:29 |
| dcdamien | folks, you know that famous change that brings back VPNaaS dashboard back using wrong git URL | 12:30 |
| dcdamien | can you please let me know if can it be backported? what policy states about? | 12:30 |
| jrosser | mnaser: so about that second patch, what is it that connects to the public endpoint there? | 12:30 |
| odyssey4me | dcdamien you are welcome to propose backports - although you should do a backport with the correct content this time :) | 12:30 |
| odyssey4me | dcdamien also note that there needs to be a corresponding patch to the integrated build first to ensure that the SHA is pinned | 12:30 |
| *** pvradu has joined #openstack-ansible | 12:31 | |
| *** hamzaachi_ has quit IRC | 12:31 | |
| mnaser | jrosser: if you're using the heat agent like os-collect-config or so | 12:31 |
| odyssey4me | dcdamien similar to https://github.com/openstack/openstack-ansible/blob/master/playbooks/defaults/repo_packages/openstack_services.yml#L105-L109 | 12:31 |
| mnaser | and also in magnum is pushes it out to the container to do things like auth to get keys | 12:31 |
| jrosser | mnaser: the reason I hesitate is that (in my case) the internals of the openstack services can't connect to the public endpoints | 12:32 |
| *** pvradu has quit IRC | 12:32 | |
| jrosser | and i don't know enough it to know if that is/isnt a problem | 12:32 |
| *** pvradu has joined #openstack-ansible | 12:33 | |
| mnaser | jrosser: it should still work, that url is the one exposed to the user | 12:33 |
| jrosser | righto, thats fine | 12:33 |
| mnaser | it's just to kinda say "here is where you should go to authenticate" | 12:33 |
| jrosser | gotcha | 12:33 |
| odyssey4me | mnaser ok, what determines the endpoint the magnum service itself uses? | 12:34 |
| odyssey4me | I've posed another Q in review. | 12:34 |
| mnaser | odyssey4me: an awful scary workaround with a big TODO in magnum | 12:34 |
| mnaser | let me find it | 12:34 |
| jrosser | imho this is something we should really improve in our testing, making sure there isnt an accidental short circuit internal/external | 12:35 |
| *** dcdamien has quit IRC | 12:35 | |
| odyssey4me | jrosser yeah, best way to achieve that would be to actually fire up a heat stack in our integrated build tests | 12:35 |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 12:35 |
| odyssey4me | the healthcheck-openstack playbook actually does that | 12:35 |
| jrosser | in an AIO internal/external isnt really very distinct though is it? | 12:36 |
| *** maddtux has quit IRC | 12:36 | |
| odyssey4me | jrosser yeah, but due to the self signed cert - anything that tries to use the public endpoint will fail | 12:37 |
| odyssey4me | any internal service I mean | 12:37 |
| jrosser | ah sneaky | 12:37 |
| mnaser | odyssey4me: voila | 12:37 |
| odyssey4me | mnaser oh dear, that is horrible | 12:38 |
| mnaser | yep. you can imagine how much fun that was to find lol | 12:39 |
| cshen | hello, is it true that each entry in the when block must be wrapper with double quotes? | 12:39 |
| cshen | wrapped | 12:40 |
| mnaser | not always cshen .. examples? | 12:40 |
| odyssey4me | cshen it is best to do so, but not required | 12:40 |
| cshen | odyssey4me Ok! | 12:40 |
| odyssey4me | cshen doing so ensures that ansible interprets the line as jinja, which sometimes doesn't happen correctly if unquoted | 12:41 |
| evrardjp | cshen: at least that's what I have been told by ansible people. | 12:41 |
| odyssey4me | I really wish the syntax requirements were more consistent - using {{}} in some places but not others really drives me nuts. | 12:41 |
| mnaser | i've found that it mostly is necessary when you do things like when: "foo" in bar | 12:41 |
| evrardjp | odyssey4me: haha | 12:41 |
| mnaser | the quotes there confuse yaml | 12:41 |
| evrardjp | mnaser: ? | 12:41 |
| mnaser | if you do: when: "foo" in bar | 12:42 |
| evrardjp | if you're doing " " it is just fine for me | 12:42 |
| evrardjp | when: \n - "'foo' in bar" | 12:42 |
| mnaser | yaml goes like "why is there a " in bar" after the quoted string | 12:42 |
| mnaser | so you need to do somehtling like | 12:42 |
| mnaser | yeah, you have to quote it in those scenarios | 12:42 |
| mnaser | but for example if you do | 12:42 |
| mnaser | when: foo is bar | 12:42 |
| mnaser | the yaml evaluates fine there | 12:42 |
| evrardjp | yeah but foo is a var | 12:42 |
| mnaser | so in my experience usually its just a yaml thing rather than an ansible thing | 12:42 |
| evrardjp | so foo needs to be a string | 12:43 |
| evrardjp | so '' | 12:43 |
| mnaser | when: 'foo is bar' vs when: foo is bar | 12:43 |
| mnaser | is the same thing in yaml | 12:43 |
| evrardjp | no I meant | 12:43 |
| evrardjp | 'foo' is bar | 12:43 |
| evrardjp | which is easier if we at the end write | 12:43 |
| mnaser | afia if you did | 12:43 |
| evrardjp | "'foo' is bar" | 12:43 |
| mnaser | when: bar == 'foo' | 12:43 |
| mnaser | that doesnt need quotes | 12:43 |
| mnaser | the fact we discuss this that much | 12:44 |
| mnaser | is a problem | 12:44 |
| evrardjp | hahah | 12:44 |
| mnaser | lolol | 12:44 |
| evrardjp | fair | 12:44 |
| evrardjp | I'd rather stick with the convention | 12:44 |
| evrardjp | " 'string' is whatevervar" | 12:44 |
| *** dcdamien has joined #openstack-ansible | 12:44 | |
| evrardjp | or the other way around | 12:44 |
| evrardjp | whatever is readable :p | 12:44 |
| evrardjp | we should have a linter to force all the when conditions to be wrapped with double quotes :p | 12:45 |
| evrardjp | #hatersaregonnahate | 12:45 |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 12:45 |
| guilhermesp | ppl ^ I was wondering how to add ansible-lint tests in os_placement | 12:46 |
| cshen | since it's best practice, I will wrap my patch with double quotes. | 12:47 |
| mnaser | guilhermesp: you'll have to add things to zuul.d | 12:48 |
| mnaser | let me see | 12:48 |
| guilhermesp | mnaser: didn't find explicitly in roles like os_glance | 12:49 |
| mnaser | guilhermesp: create a projects.yaml similar to this one https://github.com/openstack/openstack-ansible-os_glance/blob/master/zuul.d/project.yaml | 12:49 |
| mnaser | openstack-ansible-role-jobs is a template that includes linters | 12:49 |
| guilhermesp | nice | 12:49 |
| mnaser | you pretty much want to copy that exact file and drop the check: and gate: sections | 12:49 |
| mnaser | because those are custom glance jobs | 12:49 |
| mnaser | and then zuul will do its magical thing :) | 12:49 |
| guilhermesp | lovely | 12:50 |
| fghaas | Miouge: yes it does for me! | 12:50 |
| mnaser | jrosser: odyssey4me here is another gem of auth_uri having a private url (which it should not): https://review.openstack.org/#/c/615916/ | 12:50 |
| fghaas | Miouge: http://paste.openstack.org/show/735892/ | 12:50 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_heat stable/queens: Deprecate auth_uri option https://review.openstack.org/619231 | 12:52 |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 12:53 |
| Miouge | fghaas: this is what I get: http://paste.openstack.org/show/735893/ on a Pike OSA deploy with the following (default?) HAProxy conf: http://paste.openstack.org/show/735894/ | 12:56 |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 12:56 |
| openstackgerrit | Chenjun Shen proposed openstack/openstack-ansible master: Fix "configure container (other services)" when statement https://review.openstack.org/581248 | 12:58 |
| openstackgerrit | Arx Cruz proposed openstack/openstack-ansible-os_tempest master: [WIP] - Add centos-7 job with support to python-tempestconf https://review.openstack.org/619021 | 12:58 |
| mnaser | guilhermesp: http://zuul.openstack.org/status you can see that the linters and jobs are running :) | 12:59 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Ensure AIO bootstrap container tech var is namespaced https://review.openstack.org/619233 | 13:00 |
| guilhermesp | mnaser: sweet | 13:02 |
| guilhermesp | didn't notice that zuul interface has changed | 13:03 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Ensure AIO bootstrap container tech var is namespaced https://review.openstack.org/619233 | 13:03 |
| guilhermesp | ok so all functional tests are there and are going to fail of course | 13:03 |
| kaiokmo | guilhermesp: what if all functional pass at first? :) | 13:05 |
| mnaser | that theory has already failed kaiokmo :P | 13:05 |
| mnaser | http://logs.openstack.org/20/618820/13/check/openstack-ansible-functional-centos-7/2f14b4f/ | 13:05 |
| guilhermesp | kaiokmo: they're already failing :) | 13:05 |
| kaiokmo | hahah | 13:05 |
| mnaser | odyssey4me: was the run_tests.sh file sync'd with our magic tools? | 13:06 |
| mnaser | or did we manually add that | 13:06 |
| guilhermesp | I'm just writing the role. That's why I'd like linters to check the syntax. But I do need to create tests/ and run_tests.sh and start fixing stuff | 13:06 |
| odyssey4me | mnaser it will sync from the tests repo if a file by that name exists | 13:06 |
| mnaser | ok so we need to bootstrap that file | 13:07 |
| odyssey4me | so yes, you need to initially add it in a patch | 13:07 |
| mnaser | okay so guilhermesp can just copy and paste it from another repo like glance and that'll get us past that point, cool | 13:07 |
| guilhermesp | will do that mnaser | 13:07 |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 13:09 |
| openstackgerrit | Merged openstack/openstack-ansible-os_horizon master: Fix the vpnaas-dashboard git repo URL https://review.openstack.org/619179 | 13:10 |
| *** fghaas has quit IRC | 13:13 | |
| nowster | yay! | 13:19 |
| *** fghaas has joined #openstack-ansible | 13:31 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_heat master: Switch to using public endpoint for URIs https://review.openstack.org/615916 | 13:34 |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_placement master: [WIP] Create base files to install placement https://review.openstack.org/618820 | 13:38 |
| *** hamzaachi has joined #openstack-ansible | 13:47 | |
| *** strattao has joined #openstack-ansible | 13:52 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum master: Drop identity_uri usage in keystone_authtoken https://review.openstack.org/615895 | 13:58 |
| *** pvradu_ has joined #openstack-ansible | 14:03 | |
| *** pvradu has quit IRC | 14:07 | |
| *** pvradu_ has quit IRC | 14:07 | |
| *** pvradu has joined #openstack-ansible | 14:08 | |
| *** chkumar|ruck has quit IRC | 14:12 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_horizon master: Implements masakari-dashboard to be installed in horizon https://review.openstack.org/619058 | 14:13 |
| openstackgerrit | Florian Haas proposed openstack/openstack-ansible master: Include Swift AUTH_%(tenant_id)s suffix in rgw Keystone endpoint https://review.openstack.org/614194 | 14:23 |
| openstackgerrit | Florian Haas proposed openstack/openstack-ansible master: Include Swift AUTH_%(tenant_id)s suffix in rgw Keystone endpoint https://review.openstack.org/614194 | 14:30 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: [WIP] Implement changes to allow test usage by roles https://review.openstack.org/551791 | 14:32 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Ensure AIO bootstrap container tech var is namespaced https://review.openstack.org/619233 | 14:36 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: [WIP] Implement changes to allow test usage by roles https://review.openstack.org/551791 | 14:36 |
| *** udesale has joined #openstack-ansible | 14:39 | |
| openstackgerrit | Luis Filipe Domingues proposed openstack/openstack-ansible-os_cinder master: Add possibility to have private cinder volume types https://review.openstack.org/619264 | 14:40 |
| arxcruz | guys, i need help with https://review.openstack.org/#/c/619021/ | 14:40 |
| arxcruz | the tempest_use_tempestconf variable isn't being set to true for some reason | 14:41 |
| *** hamzaachi has quit IRC | 14:41 | |
| odyssey4me | arxcruz uh, nope - that's not going to work with our role tests | 14:42 |
| odyssey4me | we don't use zuul's ansible to run the test | 14:42 |
| arxcruz | (╯°□°）╯︵ ┻━┻ | 14:42 |
| arxcruz | odyssey4me: so need to create entry on tox | 14:43 |
| arxcruz | got it | 14:43 |
| odyssey4me | arxcruz you'll want to do something like https://github.com/openstack/openstack-ansible-os_glance/blob/master/tox.ini#L115 | 14:43 |
| arxcruz | yup, got it, thanks odyssey4me | 14:44 |
| odyssey4me | and then in the job: https://github.com/openstack/openstack-ansible-os_glance/blob/master/zuul.d/jobs.yaml#L23-L28 | 14:44 |
| odyssey4me | one day when there's a zuul runner, and we can specify a version of ansible to use to run the play, we can switch to using a simpler mechanism | 14:45 |
| openstackgerrit | Luis Filipe Domingues proposed openstack/openstack-ansible-os_cinder master: Add possibility to have private cinder volume types https://review.openstack.org/619264 | 14:46 |
| *** hamzaachi has joined #openstack-ansible | 14:47 | |
| openstackgerrit | Arx Cruz proposed openstack/openstack-ansible-os_tempest master: [WIP] - Add centos-7 job with support to python-tempestconf https://review.openstack.org/619021 | 14:48 |
| arxcruz | odyssey4me: ok, thanks! | 14:48 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Implement AIO changes to allow test usage by roles https://review.openstack.org/551791 | 14:55 |
| *** hamzaachi has quit IRC | 15:01 | |
| *** fghaas has left #openstack-ansible | 15:06 | |
| *** chandankumar has joined #openstack-ansible | 15:11 | |
| Miouge | I'm running into this error "Selective block device migration is not supported with tunnelled block migrations." when trying to live migrate VMs with local storage AND cinder volumes (RBD). It looks like OSA sets live_migration_tunnelled=True. Is there any way out other than detaching the volumes before live migrating? | 15:17 |
| *** Emine has quit IRC | 15:17 | |
| *** Emine has joined #openstack-ansible | 15:18 | |
| mnaser | Miouge: i guess you might have to disable that but im not sure of the consueqnces | 15:19 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_magnum stable/rocky: Drop identity_uri usage in keystone_authtoken https://review.openstack.org/619270 | 15:19 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_magnum stable/queens: Drop identity_uri usage in keystone_authtoken https://review.openstack.org/619271 | 15:20 |
| *** chandankumar has quit IRC | 15:22 | |
| mnaser | that gate fix cant merge fast enough | 15:22 |
| mnaser | oh it did merge | 15:23 |
| odyssey4me | mnaser which one? | 15:23 |
| odyssey4me | yep | 15:23 |
| mnaser | odyssey4me: the one that makes clones fail | 15:23 |
| Miouge | mnaser: the vpnaas dashboard fix got merged | 15:23 |
| mnaser | odyssey4me: https://review.openstack.org/#/c/608433/ im going to be tracking this on my little time i have, iterating slowly to make things happen | 15:23 |
| mnaser | coool | 15:24 |
| *** chandankumar has joined #openstack-ansible | 15:24 | |
| mnaser | my super red patch is gonna be a bit less red | 15:24 |
| mnaser | centos failed on clones for nspawn, expected.. | 15:24 |
| odyssey4me | mnaser I think that https://review.openstack.org/551791 will enable the use of the integrated build for roles, assuming it passes -along with the dep patches below it. We'll see how the gate responds. | 15:24 |
| mnaser | odyssey4me: when we have some gate results i will invest in some time really reading through it | 15:25 |
| odyssey4me | yep | 15:25 |
| odyssey4me | basically the scenario name can now be named whatever you like, but it will assume keystone & haproxy as a base | 15:26 |
| odyssey4me | if you add 'aio' to the scenario name, it'll add the compute kit | 15:26 |
| odyssey4me | if you add 'nspawn' to the name, it'll use nspawn instead of lxc | 15:26 |
| odyssey4me | if you add 'metal' to the name, it'll skip haproxy | 15:26 |
| mnaser | will we have to maintain all of the scenarios in integrated repo or can they live in-role? | 15:27 |
| odyssey4me | then just add whatever other services you like for extra stuff | 15:27 |
| odyssey4me | we'll need to add conf.d files, plays, group_vars, etc to the integrated build for now | 15:27 |
| openstackgerrit | Merged openstack/openstack-ansible-os_monasca-ui master: Retire openstack-ansible-os_monasca-ui https://review.openstack.org/617204 | 15:28 |
| *** hamzaachi has joined #openstack-ansible | 15:29 | |
| odyssey4me | we can see whether we want to change it up to allow in-repo other things - that would be an extension on top of that work... for now I'm focusing on the stuff already integrated | 15:29 |
| mnaser | odyssey4me: yeah lets not have too much scope creep, we'd already be years ahead just getting integrated up in roles | 15:29 |
| odyssey4me | thanks to evrardjp for the idea to simply use the scenario name and extrapolate from there | 15:29 |
| odyssey4me | to enable testing of the infrastructure roles, and different affinities, I'll do some other bits on top to enable that in a follow on patch once this is working | 15:30 |
| mnaser | i'm trying to debug nspawn failures | 15:31 |
| mnaser | centos was just git clone fail (re: vpn)) | 15:31 |
| mnaser | opensuse seems to try to access a folder that's missing in our mirrors? | 15:31 |
| mnaser | http://mirror.iad.rax.openstack.org/opensuse/distribution/leap/42.3/repo/oss/repodata/repomd.xml | 15:31 |
| mnaser | however i noticed this | 15:31 |
| mnaser | http://mirror.iad.rax.openstack.org/opensuse/distribution/leap/15.0/repo/oss/repodata/repomd.xml | 15:31 |
| mnaser | i should ahve checked another mirror | 15:32 |
| odyssey4me | maybe something's gone into the lxc hosts/container create roles which needs to be replicated to the nspawn roles that correspond | 15:32 |
| mnaser | yeah repodata is missingin upstream mirrors | 15:32 |
| mnaser | http://ftp.opensuse.org/pub/opensuse/distribution/leap/42.3/repo/oss/ here too | 15:32 |
| mnaser | so that means that we're doing something weird | 15:32 |
| mnaser | hwoarang: evrardjp any ideas? | 15:33 |
| mnaser | i wonder if its including the wrong file or if its creating like | 15:34 |
| mnaser | 15.0 containers | 15:34 |
| mnaser | cloudnull: any ideas too? | 15:36 |
| odyssey4me | mnaser it's perhaps likely that https://github.com/openstack/openstack-ansible-lxc_hosts/commit/5f0f01c355972a3c4ef87abf2aa5302ef33ceb1c or something like it needs to happen in the nspawn_hosts role | 15:37 |
| odyssey4me | mnaser perhaps it'd be best to just pick one distro and get it to working, then go from there | 15:38 |
| *** chandankumar has quit IRC | 15:39 | |
| mnaser | odyssey4me: fair enough, centos it is :P | 15:39 |
| mnaser | that's the easiest i can iterate on at least anyways | 15:39 |
| mnaser | dont mind pitching in for the rest but this is the fastest path to success anyways :P | 15:39 |
| odyssey4me | sure, once that's alright then myself/cloudnull can have a go at bionic | 15:40 |
| mnaser | awesome, im excited | 15:41 |
| openstackgerrit | Chenjun Shen proposed openstack/openstack-ansible master: Fix "configure container (other services)" when statement https://review.openstack.org/581248 | 15:42 |
| mnaser | odyssey4me: i think i will repush that patch with all jobs removed except for the centos nspawn one, just to get it to run faster and waste less ci resources | 15:42 |
| mnaser | thoughts? | 15:42 |
| odyssey4me | mnaser yep, go for it | 15:43 |
| openstackgerrit | James Denton proposed openstack/openstack-ansible master: Adds haproxy endpoints for networking-ovn support https://review.openstack.org/584069 | 15:44 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: Enable passing nspawn jobs https://review.openstack.org/608433 | 15:44 |
| mnaser | there, that'll get us through quicker | 15:45 |
| *** devx has quit IRC | 15:48 | |
| Miouge | mnaser: might need to replace live_migration_tunnelled by live_migration_inbound_addr & co. Is it that most people just run with nova_libvirt_images_rbd_pool instead? | 15:48 |
| mnaser | Miouge: i mean personally speaking we run everything on rbd so i cant really like | 15:49 |
| mnaser | know what to do in that scenario | 15:49 |
| *** hamzaachi has quit IRC | 15:49 | |
| *** ThiagoCMC has quit IRC | 15:50 | |
| openstackgerrit | James Denton proposed openstack/openstack-ansible-os_neutron master: Adds DPDK support for Open vSwitch https://review.openstack.org/590810 | 15:51 |
| *** cshen has quit IRC | 15:52 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Ensure AIO bootstrap container tech var is namespaced https://review.openstack.org/619233 | 15:53 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Implement AIO changes to allow test usage by roles https://review.openstack.org/551791 | 15:53 |
| *** udesale has quit IRC | 15:58 | |
| *** gkadam has quit IRC | 16:40 | |
| *** ThiagoCMC has joined #openstack-ansible | 16:42 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Ensure AIO bootstrap container tech var is namespaced https://review.openstack.org/619233 | 16:46 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Ensure AIO container_tech/install_method vars are namespaced https://review.openstack.org/619233 | 16:48 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Implement AIO changes to allow test usage by roles https://review.openstack.org/551791 | 16:49 |
| *** pcaruana has quit IRC | 16:50 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-ops master: MNAIO: Add ability to deploy Ceph https://review.openstack.org/607338 | 16:52 |
| *** ThiagoCMC has quit IRC | 16:53 | |
| *** shardy has quit IRC | 17:01 | |
| *** Emine has quit IRC | 17:05 | |
| *** shardy has joined #openstack-ansible | 17:07 | |
| *** pvradu has quit IRC | 17:08 | |
| openstackgerrit | Arx Cruz proposed openstack/openstack-ansible-os_tempest master: Add centos-7 job with support to python-tempestconf https://review.openstack.org/619021 | 17:17 |
| *** dcdamien has quit IRC | 17:17 | |
| arxcruz | odyssey4me: evrardjp https://review.openstack.org/#/c/619021/ ready for review :) | 17:18 |
| *** arbrandes has quit IRC | 17:21 | |
| jrosser | odyssey4me: would we expect the "install tempest plugin requirements" to be all available from the repo server, or does the utility container need to be able to build those? | 17:36 |
| cloudnull | afternoons all | 17:41 |
| jrosser | o/ | 17:41 |
| cloudnull | mnaser odyssey4me - what do we need to give a spin in Bionic ? | 17:43 |
| odyssey4me | cloudnull well, mnaser is working on getting the nspawn gate on centos to pass, then we can build on top of that for bionic | 17:46 |
| cloudnull | cool! | 17:46 |
| * cloudnull was just looking at https://review.openstack.org/#/c/608433 | 17:47 | |
| odyssey4me | cloudnull yep, obviously if you see any results and can suggest changes - go ahead and do so in review | 17:48 |
| cloudnull | ++ | 17:48 |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Correct clustered gating job https://review.openstack.org/619033 | 17:53 |
| *** rpittau has quit IRC | 17:57 | |
| *** hamzaachi has joined #openstack-ansible | 18:04 | |
| *** arbrandes has joined #openstack-ansible | 18:10 | |
| mnaser | Hey cloudnull quick question | 18:21 |
| mnaser | Are bind mounts done differently with nspawn? | 18:22 |
| mnaser | http://logs.openstack.org/33/608433/8/check/openstack-ansible-deploy-aio_nspawn-centos-7/7e7b76b/logs/openstack/aio1_glance_container-0f1deaf7/glance/glance-api.log.txt.gz | 18:22 |
| mnaser | Actually nevermind. This is not /var/log but /var/lib | 18:22 |
| mnaser | I wonder why it’s an issue in nspawn but not lxc | 18:23 |
| cloudnull | whats the issue? | 18:24 |
| mnaser | https://www.irccloud.com/pastebin/dSMqSC04 | 18:25 |
| mnaser | cloudnull: nspawn fails in tempest upload image because glance can’t go | 18:25 |
| mnaser | Up | 18:25 |
| mnaser | Because it somehow can’t write to /var/lib/glance/cache | 18:25 |
| mnaser | Somehow that folder already exists?! | 18:26 |
| mnaser | http://logs.openstack.org/33/608433/8/check/openstack-ansible-deploy-aio_nspawn-centos-7/7e7b76b/logs/ara-report/result/837db9b8-3534-4265-9505-0a771ae84a7e/ | 18:26 |
| mnaser | But that’s the first task of glance so I’m struggling to understand how that folder exists | 18:26 |
| mnaser | Unless somehow when we create the container it’s getting installed? | 18:27 |
| cloudnull | that's peculiar | 18:28 |
| * cloudnull reading the logs | 18:28 | |
| cloudnull | I have nspawn running on my local lab here however the glance store is on NFS | 18:29 |
| cloudnull | are there tasks in the glance role / playbook that reload the container ? | 18:29 |
| cloudnull | maybe its not getting the correct config initially ? | 18:29 |
| mnaser | cloudnull: well digging deeper it seems that /var/lib/glance somehow already exists (and with different permissions I assume) | 18:30 |
| mnaser | And so that’s why things don’t work properly because when Ansible creates the user it notices it’s already there | 18:31 |
| *** DanyC has quit IRC | 18:32 | |
| *** DanyC has joined #openstack-ansible | 18:33 | |
| mnaser | Something at some point creates it before. I’m checking this on mobile so I’ll try to look deeepr when I have my laptop ahead of me | 18:33 |
| cloudnull | if we could pull in the /etc/systemd/nspawn and /etc/systemd/system/$container_name.service files we could inspect whats going on there. | 18:33 |
| *** shardy has quit IRC | 18:34 | |
| *** ThiagoCMC has joined #openstack-ansible | 18:35 | |
| cloudnull | and yes, ansible is saying that those dirs already exist | 18:35 |
| cloudnull | http://logs.openstack.org/33/608433/8/check/openstack-ansible-deploy-aio_nspawn-centos-7/7e7b76b/logs/openstack/ansible-logging/setup-openstack.log.txt.gz#_2018-11-21_17_13_17_264 | 18:35 |
| cloudnull | so maybe something in the DVSM image? | 18:36 |
| *** shardy has joined #openstack-ansible | 18:36 | |
| *** DanyC has quit IRC | 18:37 | |
| cloudnull | this is interesting though http://logs.openstack.org/33/608433/8/check/openstack-ansible-deploy-aio_nspawn-centos-7/7e7b76b/logs/openstack/ansible-logging/setup-openstack.log.txt.gz#_2018-11-21_17_15_42_966 | 18:38 |
| cloudnull | https://pasted.tech/pastes/fe5e91ac58d13d2e7a8474f275c154f1c9045eaa.raw | 18:39 |
| *** strattao has quit IRC | 18:42 | |
| cloudnull | https://pasted.tech/pastes/7cd117eb8e73828cc85eea6007669851935fe76b - this is all very suspect. | 18:45 |
| * cloudnull goes to make an AIO | 18:45 | |
| cloudnull | also looks like we need to change default_store to default_backend | 18:47 |
| *** strattao has joined #openstack-ansible | 18:50 | |
| ThiagoCMC | This isn't an OpenStack Ansible question but, I'm curious, currently, there is "Nova Compute with LXD", but, what about "Nova Compute with nspawn", will it come someday? | 18:52 |
| *** ahosam has joined #openstack-ansible | 18:53 | |
| cloudnull | ThiagoCMC it could come one day, though idk anyone working on that | 18:53 |
| ThiagoCMC | Ok! =) | 18:53 |
| cloudnull | I think the closest thing to that is lxd or zun | 18:53 |
| cloudnull | we have nova-lxd already, and we have a new role for zun | 18:53 |
| cloudnull | though it needs folks to use and figure out the rough parts | 18:53 |
| ThiagoCMC | Nice! I started to use nspawn a few weeks ago and I really like it! It's so simple. ^_^ | 18:54 |
| * cloudnull made the zun role during the vancouver summit | 18:54 | |
| ThiagoCMC | :-D | 18:54 |
| ThiagoCMC | I mean, I'm using nspawn not with OSA, just to replace my own Ubuntu KVM guests in my Ubuntu desktop PC, it's awesome! | 18:54 |
| ThiagoCMC | =) | 18:54 |
| cloudnull | ++ I agree. Im a fan of the nspawn methods | 18:54 |
| * cloudnull uses nspawn for all my workstation tools and services | 18:56 | |
| *** shardy has quit IRC | 18:56 | |
| *** DanyC has joined #openstack-ansible | 18:57 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum master: Switch to www_authenticate_uri instead of auth_uri https://review.openstack.org/615896 | 18:59 |
| *** ThiagoCMC has quit IRC | 19:00 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-nspawn_hosts master: Change ssh variable in Ubuntu and remove masks https://review.openstack.org/619339 | 19:01 |
| *** DanyC has quit IRC | 19:02 | |
| cloudnull | any cores around want to give this a review https://review.openstack.org/#/c/607811/ | 19:07 |
| *** mmercer has joined #openstack-ansible | 19:07 | |
| *** dcdamien has joined #openstack-ansible | 19:08 | |
| jrosser | cloudnull: remember there’s still a bit of a mess in the swift service config..... | 19:09 |
| cloudnull | oh yes... | 19:10 |
| jrosser | That patch is cool but it think it’s doing a load of duplicate either way | 19:11 |
| cloudnull | I have my local lab w/ swift here . let me update to the latest master and see where things less than perfect | 19:11 |
| cloudnull | do you by chance remember the specific units that were busted? | 19:12 |
| jrosser | Unless I’m missing something there are a bunch of duplicate replicator services | 19:12 |
| jrosser | With oh-so-similar but different service names | 19:12 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_cinder master: remove loop control when running include_role https://review.openstack.org/607814 | 19:12 |
| *** klamath has quit IRC | 19:13 | |
| jrosser | cloudnull: and then in the journals for half of those services they are all busy because the sockets are already in use iirc | 19:13 |
| jrosser | *bust | 19:14 |
| mnaser | cloudnull: ok so when we're running the db migration, it's creating the folder | 19:16 |
| mnaser | cloudnull: aw i bet you it's this -- http://logs.openstack.org/33/608433/8/check/openstack-ansible-deploy-aio_nspawn-centos-7/7e7b76b/job-output.txt.gz#_2018-11-21_17_13_18_014087 | 19:16 |
| cloudnull | jrosser https://pasted.tech/pastes/97021b92c88df4dac3a540e03c50aaf00680ff85 | 19:18 |
| cloudnull | this is what i have currently | 19:18 |
| cloudnull | which is osa master from a few weeks ago | 19:19 |
| mnaser | HA | 19:20 |
| mnaser | found the bug | 19:20 |
| mnaser | i think. | 19:20 |
| mnaser | ok no its just a misnamed task | 19:21 |
| mnaser | ok so whats happening here is that `/var/lib/glance/cache` is being created by the task which does the bind mounts to the host | 19:23 |
| mnaser | in lxc, we don't create it inside the container, i guess the lxc config creates it automagically | 19:23 |
| cloudnull | jrosser the proxy server only has the one unit - https://pasted.tech/pastes/fc6492c07f19dfff2da778181ad927f4857c4f44 | 19:23 |
| cloudnull | so with that Im going to go upgrade all the things and see if it goes a bit haywire | 19:24 |
| cloudnull | I also have an AIO spinning so i should have a greenfield env to compare with | 19:24 |
| mnaser | cloudnull: do any of the nspawn roles take care of doing bind mounts? | 19:25 |
| cloudnull | mnaser so var/lib/glance/cache is being created by the service ? | 19:25 |
| mnaser | cloudnull: nope its being created by a task | 19:25 |
| mnaser | http://logs.openstack.org/33/608433/8/check/openstack-ansible-deploy-aio_nspawn-centos-7/7e7b76b/job-output.txt.gz#_2018-11-21_17_13_18_014087 | 19:25 |
| mnaser | that task is not the right name, it should say (container) and not (physical host) but it creates a folder there | 19:25 |
| mnaser | i assume it creates it under root:root | 19:25 |
| cloudnull | the bind-mounts in nspawn are created by the playbook in the integrated repo, same as lxc | 19:25 |
| mnaser | it doesnt look like we actually create folders in lxc | 19:26 |
| mnaser | but we do in nspawn | 19:26 |
| mnaser | (inside the container) | 19:26 |
| mnaser | i.e. this task doesnt exist in lxc world: https://github.com/openstack/openstack-ansible/blob/master/playbooks/common-tasks/os-nspawn-container-setup.yml#L50-L57 | 19:26 |
| cloudnull | ah yes, in lxc the interior directories are created here https://github.com/openstack/openstack-ansible/blob/master/playbooks/common-tasks/os-lxc-container-setup.yml#L52 | 19:28 |
| cloudnull | theres no specific task in lxc that does it , its handled by the mount option | 19:29 |
| mnaser | i mean regardless it's still going to be an issue because we're creating things as root:root | 19:29 |
| mnaser | so maybe what we need to do is add /var/lib/glance to the task that creates folders? | 19:29 |
| cloudnull | so maybe the interious nspawn dirs need to be created more permissively ? | 19:29 |
| cloudnull | ** interior | 19:30 |
| mnaser | either that or the roles should enforce permissions | 19:30 |
| mnaser | aka do chown glance:glance /var/lib/glance | 19:30 |
| mnaser | which feels more right imho | 19:30 |
| mnaser | rather than most probably making it world writeable | 19:31 |
| cloudnull | ++ however I think its doing the the chown in the role already ? | 19:31 |
| cloudnull | or it should be | 19:31 |
| *** hamzaachi has quit IRC | 19:31 | |
| mnaser | let me double check | 19:31 |
| mnaser | in terms of roles that do bind monuts | 19:31 |
| *** hamzaachi has joined #openstack-ansible | 19:31 | |
| mnaser | we got ceph, galera, glance and gnocchi | 19:31 |
| cloudnull | here https://github.com/openstack/openstack-ansible-os_glance/blob/master/tasks/glance_install.yml#L53-L78 | 19:32 |
| mnaser | cloudnull: i guess /cache is not in that list | 19:32 |
| mnaser | so /cache is root:root but /cache/{api,registry} is glance:glance | 19:32 |
| cloudnull | ah... | 19:32 |
| cloudnull | this would make some sense . | 19:32 |
| mnaser | i think adding "{{ glance_system_user_home }}/cache" to that list probably will get us our result | 19:33 |
| cloudnull | +1 | 19:33 |
| * cloudnull happy to push the button if your still afk | 19:33 | |
| * mnaser push fix | 19:33 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_glance master: Fix permissions on /var/lib/glance/cache https://review.openstack.org/619343 | 19:36 |
| mnaser | cloudnull: ^ that should do | 19:36 |
| * cloudnull does not have an lxc cloud i can go check atm | 19:36 | |
| cloudnull | however i would suspect the cache is 0777 ? | 19:36 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: WIP: Enable passing nspawn jobs https://review.openstack.org/608433 | 19:36 |
| mnaser | well if its glance:glance that'll be good enough | 19:37 |
| mnaser | that's a depends on and we'll see how it goes | 19:37 |
| cloudnull | ++ | 19:37 |
| mnaser | thanks for help cloudnull | 19:37 |
| * mnaser goes back to breaking thingz | 19:37 | |
| * cloudnull is curious what the perms are for that dir in a filebacked lxc environment | 19:37 | |
| *** klamath has joined #openstack-ansible | 19:38 | |
| cloudnull | mnaser nspawn will likely still fail in the gate until we correct - https://review.openstack.org/#/c/619339/ - for ubuntu that is | 19:39 |
| jrosser | cloudnull: so this https://github.com/openstack/openstack-ansible-os_swift/blob/master/defaults/main.yml#L395 vs. https://github.com/openstack/openstack-ansible-os_swift/blob/master/defaults/main.yml#L441 | 19:39 |
| jrosser | and a few similar for the other replicators | 19:39 |
| cloudnull | ah . looks like we also need to deal with the journal change over there too ... | 19:41 |
| * cloudnull is looking | 19:41 | |
| mnaser | cloudnull: want me to repush the change with bionic in there too to get some results if you have time to iterate? | 19:43 |
| mnaser | i just put it to centos cause i was just gonna focus on getting that up and running as it's the one im most comfortalbe with | 19:44 |
| mnaser | but i did notice that o | 19:44 |
| cloudnull | nah. I'll build on top your patch | 19:44 |
| cloudnull | i'll make a dependent commit | 19:44 |
| mnaser | ok cools | 19:44 |
| cloudnull | as to not impede cent progress | 19:44 |
| mnaser | thanks, im gonna be iterating over and over that one, it seemed to actually make really good progress | 19:45 |
| mnaser | like i really dont think there's much more work to be done | 19:45 |
| mnaser | (famous last words) | 19:45 |
| mnaser | everything seemed to be running ok in terms of other services | 19:45 |
| mnaser | uhhh we're deploying rsyslog container in aio? | 19:46 |
| mnaser | http://logs.openstack.org/33/608433/8/check/openstack-ansible-deploy-aio_nspawn-centos-7/7e7b76b/logs/openstack/aio1_rsyslog_container-d3e30614/ | 19:46 |
| *** klamath has quit IRC | 19:46 | |
| *** mmercer has quit IRC | 19:48 | |
| *** klamath has joined #openstack-ansible | 19:49 | |
| cloudnull | yes we are, though its not doing much these days | 20:03 |
| *** cshen has joined #openstack-ansible | 20:05 | |
| *** mmercer has joined #openstack-ansible | 20:11 | |
| cshen | is there any problem with zuul now? zuul check for my patch had timeout. | 20:12 |
| *** ahosam has quit IRC | 20:24 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/queens: Use loop_var name in when clause https://review.openstack.org/616596 | 20:25 |
| cloudnull | cshen there could be | 20:39 |
| cloudnull | just comment `recheck` and it should retry, however, if it continues we might need to look into why | 20:39 |
| *** d3n14l has joined #openstack-ansible | 20:42 | |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible stable/rocky: Add openstackclient bash completion https://review.openstack.org/619354 | 20:45 |
| d3n14l | @cloudnull @evrardjp @odyssey4me . We just migrated from pike to queens - neutron agents update and migrated to baremetal, but no snat namespaces get created, router can not communicate with outside world… any help appreciated. | 20:45 |
| cloudnull | d3n14l upgraded with the "run-upgrade.sh" script ? | 20:46 |
| cloudnull | are the agents on the infra nodes starting? | 20:46 |
| cloudnull | anything in the journal/logs indicating specific errors ? | 20:46 |
| d3n14l | can not see errors in neutron logs. will check journl | 20:47 |
| cloudnull | also the status of the neutron-agents, are they running ? | 20:49 |
| cshen | cloudnull thanks, I'll try. | 20:49 |
| *** hamzaachi has quit IRC | 20:50 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_heat stable/rocky: Switch to using public endpoint for URIs https://review.openstack.org/619355 | 20:50 |
| d3n14l | @cloudnull agents are running on baremetal lxc-containers are stopped. | 20:51 |
| d3n14l | looks good… suspect some neutron config regarding the br-provider … but can not put the finger on it. | 20:52 |
| jrosser | in this case do you have to make sure that eth12 (or equivalent) is present on the infra host in the same way as it would have been in the container? | 20:52 |
| cloudnull | I'd tail the logs / journal in one window and restart the agents in another. | 20:52 |
| cloudnull | maybe something failing to initialize ? | 20:52 |
| cloudnull | also what jrosser said :) | 20:53 |
| jrosser | becasue the OSA stuff would have set up a bunch of particularly named interfaces on the old container eth11/12 etc and the config might be expecting to find those now on the host | 20:53 |
| d3n14l | neutron-openvswitch service log shows errors on restart… i will check what it is. | 21:00 |
| cshen | cloudnull https://review.openstack.org/#/c/581248/10 did I comment 'recheck' correctly? | 21:01 |
| cshen | zull check is not restarted. | 21:01 |
| jrosser | cshen: have a look here http://zuul.openstack.org/status | 21:02 |
| jrosser | you can see the job running and even watch the console if you need to | 21:02 |
| d3n14l | Erros were only due to restart :-/ | 21:03 |
| cloudnull | cshen yes. | 21:04 |
| cshen | I got it, thanks. | 21:04 |
| cloudnull | d3n14l are your network hosts able to get back to the rabbitmq nodes and are they running ? | 21:05 |
| cloudnull | I've seen issues where the service looks like it starts but does nothing, which was caused by not being able to communicate with rabbit | 21:05 |
| d3n14l | @cloudnull no errors regarding rabbit also everything else seems to be running except snat namespaces. | 21:06 |
| d3n14l | @jrosser which config will expect the eth11 eth12 names exactly? | 21:06 |
| jamesdenton | anything interesting in the neutron l3-agent log file? | 21:07 |
| d3n14l | nope | 21:10 |
| jamesdenton | can you hop into the utility container and perform an 'openstack network agent list'? | 21:11 |
| jamesdenton | it may be a matter of shuffling routers around | 21:11 |
| jrosser | d3n14l: well in the example linucbridge config eth12 is the interface given to neutron, but as you're doing OVS i'll leave you in the capable hands of jamesdenton :) | 21:13 |
| jamesdenton | eek | 21:15 |
| d3n14l | @jamesdenton @jrosser @cloudnull - our container_interface still is "eth12" and type is veth - but that matched the lxc container setup - what we need would be br-vlan with type bridge - I guess. | 21:15 |
| d3n14l | https://pastebin.com/Xvvbr8z7 | 21:16 |
| d3n14l | We did not change that config when moving from lxc to baremetal. | 21:17 |
| d3n14l | What we did was: manually plugging the br-vlan into br-provider in the openvswitch. | 21:17 |
| jamesdenton | does that mirror how the computes are configured? | 21:18 |
| d3n14l | yes | 21:20 |
| jamesdenton | do you also have an override configured for 'neutron_provider_networks' in say, user_variables.yml? | 21:21 |
| d3n14l | https://pastebin.com/66pFvt8A | 21:22 |
| *** radeks_ has quit IRC | 21:22 | |
| d3n14l | yes | 21:23 |
| d3n14l | neutron_provider_network override: https://pastebin.com/S58bHSLm | 21:24 |
| jamesdenton | so at this point, do you have any working vm traffic? Does it work out thru a distributed router on a compute? And only the snat namespace seems to be an issue? Are dhcp namespaces OK? | 21:24 |
| d3n14l | Yes - despite: dhcp namespaces are there and dnsmasqs are listening - but they can not get data from their upstream because their gateway (the router) is missing its external ip (in the snat). | 21:25 |
| d3n14l | communication through floating ips on compute nodes is possible. | 21:26 |
| d3n14l | with dvr | 21:26 |
| jamesdenton | k | 21:26 |
| jamesdenton | can you post the output of 'openstack network agent list --router <router>' | 21:28 |
| jamesdenton | as well as ''openstack network agent list' | 21:31 |
| d3n14l | https://pastebin.com/BxWhugLH | 21:31 |
| mnaser | cloudnull: we got beyond the glance stuff | 21:31 |
| mnaser | but now stuck on tempest flavors.. #progress regardless :) | 21:31 |
| d3n14l | there it is... | 21:31 |
| jamesdenton | d3n14l yeah, it's still scheduled to the old agents | 21:31 |
| jamesdenton | so you should be able to do a 'openstack network agent remove router <agent> <router>' to unschedule, then 'add router' to schedule to a new baremetal agent | 21:32 |
| mnaser | btw, can we get some love on https://review.openstack.org/#/c/613285/ | 21:32 |
| jamesdenton | you can prob do one at a time | 21:32 |
| d3n14l | @jamesdenton thx we will try it with one router and see what happens. | 21:34 |
| cloudnull | mnaser nice! | 21:34 |
| jamesdenton | cool. It may be possible to disable one of the old agents via the 'network agent set' command, unschedule, and see if neutron auto reschedules to a baremetal agent | 21:35 |
| jamesdenton | i can't recall | 21:35 |
| d3n14l | the container agents are already disabled - no auto rescheduling. | 21:36 |
| jamesdenton | k | 21:37 |
| d3n14l | while a colleague tries this… how would I reschedule a lbaas :-) | 21:37 |
| d3n14l | ?? | 21:37 |
| d3n14l | We still got one of them on the lxc-containers. | 21:38 |
| d3n14l | the only one. | 21:38 |
| jamesdenton | hmm | 21:38 |
| jamesdenton | this is lbaasv2 i assume? | 21:39 |
| jamesdenton | i don't think there is an API hook for that. There is an auto failover mechanism, though. http://www.nirmagnezi.com/neutron-lbaas-ha-part-i-auto-reschedule-loadbalancers/ | 21:41 |
| *** ahosam has joined #openstack-ansible | 21:42 | |
| d3n14l | @jamesdenton - we added one of the baremetal agents to the router and restartet the baremetal agent - snat namespace comes up after a while and vms get dns and outbound traffic back. | 21:44 |
| d3n14l | thank you very much! | 21:44 |
| d3n14l | We will try to work it up from here. | 21:44 |
| d3n14l | regarding the lbaas we will hav a look tomorrow | 21:44 |
| jamesdenton | that's very good news | 21:44 |
| jamesdenton | we'll want to update the upgrade docs to reflect that behavior | 21:45 |
| d3n14l | that would be … ahmm… awesome :-D | 21:45 |
| jamesdenton | do you have the URL you followed? | 21:46 |
| d3n14l | @cshen will then also have this info when his upgrade will finally take place :-) Good evening cshen ;-) | 21:46 |
| d3n14l | @jamesdenton need to look it up in our own docs - colleague planned and executed the upgrade - I just joined for firefighting. | 21:47 |
| cshen | d3n14l good luck :-) | 21:48 |
| d3n14l | thanks! | 21:48 |
| jamesdenton | fair enough | 21:49 |
| jamesdenton | keep us posted! | 21:49 |
| d3n14l | @jamesdenton we followed https://docs.openstack.org/openstack-ansible/queens/admin/upgrades/major-upgrades.html and the upgrade notes of the queens release notes | 21:52 |
| d3n14l | Was that what you meant? | 21:52 |
| jamesdenton | it is | 21:53 |
| jamesdenton | I'll see about updating those release notes to reflect an OVS deploy. a little trickier, as you saw | 21:54 |
| jamesdenton | Do you mind filing a bug? | 21:55 |
| d3n14l | Sure - if it can wait until tomorrow :-D it is 10:55 p.m. here. | 21:55 |
| jamesdenton | oh of course. i probably won't look at it until Monday :D | 21:56 |
| d3n14l | No problem. | 21:56 |
| *** eumel8 has quit IRC | 21:59 | |
| jrosser | oh man all these setup.py are unhelpful | 22:15 |
| jrosser | i just replaced my make binary with a shell script that calls the original with --jobs=<lots> and it speeds up all the compiling hugely | 22:15 |
| jrosser | theres no standard way to pass those args | 22:16 |
| *** priteau has quit IRC | 22:16 | |
| mnaser | jrosser: i assume this is a huge speed up for arm | 22:16 |
| jrosser | it was just sat there with one core stuck at 100% | 22:17 |
| mnaser | jrosser: !! | 22:17 |
| mnaser | https://stackoverflow.com/a/32598533 | 22:17 |
| mnaser | ignore the first part | 22:17 |
| mnaser | the second part | 22:17 |
| jrosser | theres no wheels for a load of stuff so it has to do really loads of work just to get through boostrap_ansible | 22:17 |
| jrosser | hahahah | 22:17 |
| jrosser | mnaser: this also shows that with that arm image we need to add build-essentials to the packages the boostrap installs, as you need a toolchain in this situation | 22:19 |
| d3n14l | @jamesdenton as we did not had auto rescheduling of lbaas turned on (a point that goes on our todo list - nudge @cshen) we fell back to changing the hosting agent in the database and restartet the agents. Worked perfectly. | 22:43 |
| d3n14l | Expect the bug report tomorrow and again - we owe you a beer! Thanks a lot! | 22:43 |
| *** d3n14l has left #openstack-ansible | 22:48 | |
| jamesdenton | i work for beer. cheers! | 23:07 |
| *** ThiagoCMC has joined #openstack-ansible | 23:09 | |
| *** ahosam has quit IRC | 23:19 | |
| *** strattao has quit IRC | 23:39 | |
| *** lbragstad is now known as lbragstad_turkey | 23:46 | |
| *** lbragstad_turkey has quit IRC | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!