| *** cshen has joined #openstack-ansible | 00:15 | |
| *** cshen has quit IRC | 00:20 | |
| *** bgmccollum has joined #openstack-ansible | 00:26 | |
| *** cmart has quit IRC | 00:46 | |
| *** markvoelker has joined #openstack-ansible | 00:50 | |
| *** aedc has quit IRC | 00:59 | |
| *** aedc has joined #openstack-ansible | 01:00 | |
| *** gyee has quit IRC | 01:15 | |
| *** markvoelker has quit IRC | 01:23 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: improve deployments on 14.04 https://review.openstack.org/641901 | 02:05 |
|---|---|---|
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: improve deployments on 14.04 https://review.openstack.org/641901 | 02:06 |
| *** cshen has joined #openstack-ansible | 02:15 | |
| *** cshen has quit IRC | 02:20 | |
| *** markvoelker has joined #openstack-ansible | 02:20 | |
| *** hwoarang has quit IRC | 02:28 | |
| *** hwoarang has joined #openstack-ansible | 02:30 | |
| *** jhesketh has quit IRC | 02:45 | |
| *** jhesketh has joined #openstack-ansible | 02:47 | |
| *** markvoelker has quit IRC | 02:53 | |
| mnaser | jrosser, odyssey4me, guilhermesp: https://review.openstack.org/#/c/641907/ | 03:15 |
| mnaser | i am 99% confident that is the cause of our gate breakages. | 03:15 |
| mnaser | in the meantime, i will try and see why we're reloading the service.. | 03:15 |
| mnaser | (we still need to fix the service) | 03:15 |
| *** sep has quit IRC | 03:20 | |
| *** sep has joined #openstack-ansible | 03:21 | |
| *** mhayden has quit IRC | 03:24 | |
| *** jbadiapa has quit IRC | 03:24 | |
| *** jbadiapa has joined #openstack-ansible | 03:25 | |
| *** mgariepy has quit IRC | 03:25 | |
| *** mhayden has joined #openstack-ansible | 03:26 | |
| *** mgariepy has joined #openstack-ansible | 03:26 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: nova: restart instead of reloading services https://review.openstack.org/641911 | 03:28 |
| mnaser | my god was that the _hardest_ thing to dig | 03:28 |
| mnaser | ever. | 03:28 |
| mnaser | probably easily poured some 3-4 hours into that | 03:28 |
| mnaser | but we also figured out that we were breaking deployments and they were working _by luck_ | 03:29 |
| *** markvoelker has joined #openstack-ansible | 03:50 | |
| logan- | wow mnaser read the -nova backscroll, quite a find | 03:56 |
| *** BjoernT has joined #openstack-ansible | 03:58 | |
| mnaser | :D | 04:00 |
| *** markvoelker has quit IRC | 04:23 | |
| *** udesale has joined #openstack-ansible | 04:24 | |
| *** hwoarang has quit IRC | 05:02 | |
| *** hwoarang has joined #openstack-ansible | 05:08 | |
| *** markvoelker has joined #openstack-ansible | 05:20 | |
| mnaser | yay. looks like tempest is passing | 05:29 |
| *** markvoelker has quit IRC | 05:53 | |
| *** marst has joined #openstack-ansible | 06:12 | |
| mnaser | failed in proxy and opensuse | 06:42 |
| mnaser | I think we might just pull out the support it’s consistently and constantly broken without commitment behind supporting it :( | 06:43 |
| *** ppetit has joined #openstack-ansible | 06:46 | |
| *** DanyC has joined #openstack-ansible | 06:47 | |
| *** ivve has joined #openstack-ansible | 06:49 | |
| *** markvoelker has joined #openstack-ansible | 06:51 | |
| *** Bhujay has joined #openstack-ansible | 06:52 | |
| *** luksky has joined #openstack-ansible | 07:17 | |
| *** cjloader has quit IRC | 07:22 | |
| *** markvoelker has quit IRC | 07:23 | |
| *** hwoarang has quit IRC | 07:25 | |
| *** pcaruana has joined #openstack-ansible | 07:26 | |
| *** cshen has joined #openstack-ansible | 07:29 | |
| *** hwoarang has joined #openstack-ansible | 07:30 | |
| *** DanyC has quit IRC | 07:31 | |
| *** miloa has joined #openstack-ansible | 07:37 | |
| miloa | Morning | 07:37 |
| *** cjloader has joined #openstack-ansible | 07:39 | |
| *** phasespace has quit IRC | 07:44 | |
| fnpanic | morning | 07:48 |
| *** gkadam__ has joined #openstack-ansible | 07:50 | |
| *** Bhujay has quit IRC | 07:50 | |
| *** Bhujay has joined #openstack-ansible | 07:57 | |
| *** BjoernT has quit IRC | 07:57 | |
| *** DanyC has joined #openstack-ansible | 08:16 | |
| *** markvoelker has joined #openstack-ansible | 08:20 | |
| *** DanyC has quit IRC | 08:22 | |
| *** Bhujay has quit IRC | 08:31 | |
| *** admin0 has joined #openstack-ansible | 08:34 | |
| *** phasespace has joined #openstack-ansible | 08:42 | |
| *** Nhey has quit IRC | 08:42 | |
| *** marst has quit IRC | 08:51 | |
| *** markvoelker has quit IRC | 08:53 | |
| *** pcaruana has quit IRC | 09:12 | |
| *** DanyC has joined #openstack-ansible | 09:19 | |
| *** hamzaachi has joined #openstack-ansible | 09:23 | |
| *** cshen has quit IRC | 09:23 | |
| *** DanyC has quit IRC | 09:24 | |
| *** DanyC has joined #openstack-ansible | 09:25 | |
| ioni | i was wondering why openstack ansible is creating a new member role called _member_ | 09:26 |
| ioni | rocky has already a member role | 09:26 |
| *** pcaruana has joined #openstack-ansible | 09:27 | |
| odyssey4me | ioni some time ago, the default in openstack was _member_, so OSA creates that for compatibility | 09:34 |
| odyssey4me | mnaser we use reload, because that's what the rolling upgrade docs say we should be doing | 09:35 |
| *** waverider has joined #openstack-ansible | 09:36 | |
| odyssey4me | mnaser in theory it ensures that when the last reload is done, any in-flight transactions do not get lost or fail - it's all part of trying to ensure a rolling upgrade works | 09:48 |
| *** markvoelker has joined #openstack-ansible | 09:51 | |
| *** DanyC_ has joined #openstack-ansible | 10:01 | |
| *** DanyC has quit IRC | 10:04 | |
| *** DanyC_ has quit IRC | 10:08 | |
| *** mkuf has quit IRC | 10:09 | |
| *** mkuf has joined #openstack-ansible | 10:13 | |
| *** markvoelker has quit IRC | 10:23 | |
| *** hamzaachi_ has joined #openstack-ansible | 10:32 | |
| *** hamzaachi has quit IRC | 10:33 | |
| *** electrofelix has joined #openstack-ansible | 10:36 | |
| ostackz | Hi odyssey4me mnaser can you confirm that after fresh Rocky 18.1.3/4 deployment heat stacks are working? Just upgraded my 18.1.3 to 18.1.4 but heat still does not work https://pastebin.com/raw/jVXb2kAz Just want to know if I am trying in vain and it does not work out of the box | 10:49 |
| *** cshen has joined #openstack-ansible | 10:51 | |
| *** udesale has quit IRC | 11:03 | |
| *** udesale has joined #openstack-ansible | 11:04 | |
| *** ppetit has quit IRC | 11:04 | |
| *** ppetit has joined #openstack-ansible | 11:06 | |
| *** luksky has quit IRC | 11:06 | |
| ostackz | has anyone successfully deployed Rocky and heat stacks are working? | 11:06 |
| odyssey4me | ostackz there has been quite a bit of discussion about that recently, and it happens when a self-signed cert is used for the public endpoints | 11:09 |
| odyssey4me | assuming it's the same issue, I mean | 11:10 |
| odyssey4me | I know jrosser, mnaser and guilhermesp have done work on it - but am not sure exactly where it's at | 11:10 |
| jrosser | It is a mess :/ I am working on it but out of office again today | 11:12 |
| ostackz | odyssey4me jrosser thanks for update, but for me to understand - is there any workaround I can try, like use non-self signed certs or something? | 11:14 |
| ostackz | or it is just sit & wait :) | 11:14 |
| *** markvoelker has joined #openstack-ansible | 11:20 | |
| ostackz | hmm, wondering if heat issue could be related to host part in URL - Im using IP, but probably ssl stuff wants fqdn? | 11:26 |
| *** cshen has quit IRC | 11:35 | |
| *** luksky has joined #openstack-ansible | 11:36 | |
| *** sep has quit IRC | 11:38 | |
| *** sep has joined #openstack-ansible | 11:39 | |
| odyssey4me | ostackz using a real SSL cert works, or not using SSL... but that's not a great solution | 11:39 |
| odyssey4me | I don't know what other workarounds are an option. It might be possible to turn off cert validation for the thing that's causing the error, although that's not a great solution from a security standpoint. | 11:39 |
| *** cshen has joined #openstack-ansible | 11:40 | |
| *** hamzaachi_ has quit IRC | 11:42 | |
| ostackz | odyssey4me thanks, on my way to deploy non-self-signed certs, let's see | 11:43 |
| *** markvoelker has quit IRC | 11:54 | |
| *** gkadam__ has quit IRC | 12:02 | |
| *** gkadam has joined #openstack-ansible | 12:04 | |
| *** tosky has joined #openstack-ansible | 12:12 | |
| mnaser | odyssey4me: unfortunately — the reload in Oslo service actually stops and restarts the service so I’m working on a fix upstream for the | 12:40 |
| mnaser | That | 12:40 |
| mnaser | Even the nova team was surprised by it | 12:41 |
| jrosser | ostackz: if you can use a real cert then the heat stuff will probably work | 12:55 |
| jrosser | odyssey4me: I started a patch for openstack_hosts to install custom CA certs - is there a place for role tests we can add extra packages on the host? It fails because of no pyopenssl | 12:57 |
| *** ivve has quit IRC | 13:08 | |
| *** DanyC has joined #openstack-ansible | 13:15 | |
| *** fghaas has left #openstack-ansible | 13:16 | |
| mnaser | jrosser: do you know why `openstack-ansible-deploy-aio_proxy-ubuntu-bionic` might be failing? | 13:20 |
| mnaser | right now, https://review.openstack.org/#/c/641911/ fixes gate isssues but opensuse still failing and aio_proxy | 13:21 |
| *** strattao has joined #openstack-ansible | 13:24 | |
| jrosser | mnaser: I fired off an AIO in a vm earlier today, it should be done by now so I will look | 13:24 |
| jrosser | AIO with the proxy scenario, even | 13:25 |
| *** hamzaachi has joined #openstack-ansible | 13:28 | |
| mnaser | jrosser: ok great, we might have a squashed fix to put in | 13:29 |
| mnaser | i don't know what to do about opensuse jobs | 13:29 |
| jrosser | mnaser: sadly my vm worked - i cherry picked your patch in the top and it's run ok | 13:29 |
| *** pcaruana has quit IRC | 13:30 | |
| mnaser | lol | 13:30 |
| mnaser | in our world | 13:30 |
| mnaser | we wish things would fail sometimes | 13:30 |
| mnaser | </3 | 13:30 |
| mnaser | ok, lets assume that was a random failure | 13:30 |
| mnaser | opensuse is still broken | 13:30 |
| jrosser | perhaps we recheck and see again, but i've seen the proxy test fail for a couple of days now | 13:30 |
| mnaser | so that's still gonna fail | 13:30 |
| jrosser | i think so | 13:31 |
| logan- | the proxy test failed because the metadata service returned 500... and that happened because keystone was not working correctly: http://logs.openstack.org/11/641911/1/check/openstack-ansible-deploy-aio_proxy-ubuntu-bionic/404aa8a/logs/openstack/aio1_nova_api_container-3021aa15/nova/nova-metadata-wsgi.log.txt.gz#_2019-03-08_05_30_13_750 | 13:32 |
| logan- | im thinking unrelated to proxying | 13:32 |
| logan- | since it failed during tempest | 13:32 |
| logan- | and that's the end of the trail since it seems we don't store keystone logs http://logs.openstack.org/11/641911/1/check/openstack-ansible-deploy-aio_proxy-ubuntu-bionic/404aa8a/logs/openstack/aio1_keystone_container-7fd93f2f/ | 13:33 |
| mnaser | shucks | 13:35 |
| mnaser | hwoarang, evrardjp -- is anyone able to look into the suse stuff that's blocking our gate? we're taking/wasting a lot of resources and if there isn't anyone driving or committing to that, we might not be able to keep it :( | 13:36 |
| hwoarang | mnaser: i am not as i am off for 2 weeks. are you talking about hte container problem? | 13:38 |
| mnaser | hwoarang: yeah, it looks like opensuse-150 has no images i think? | 13:39 |
| fnpanic | hi | 13:40 |
| hwoarang | i would have to bribe cloudnull to have a quick look about the veth network thing | 13:40 |
| hwoarang | i am not aware of any leap15 problems | 13:40 |
| fnpanic | has anyone integrated an external radogw as swift replacement? | 13:40 |
| cloudnull | o/ | 13:40 |
| cloudnull | hwoarang what's up? | 13:41 |
| fnpanic | i looked at the stein documentation but it looks like it cannot fine the /swift/v1 endpoint :-( | 13:41 |
| hwoarang | cloudnull: odyssey4me mentioned yesterday this http://logs.openstack.org/98/638198/2/check/openstack-ansible-functional-opensuse-423/d860e2d/logs/ara-report/result/1a65f714-597b-407c-ab5e-4f27a968f5a4/ which happens with the new opensuse 42.3 containers but i have no time to look at it and you know this part of the code more than i do | 13:41 |
| odyssey4me | jrosser extra packages on the host? hmm, well you could just add it to the playbook that does the role test? ie tests/test.yml ? | 13:42 |
| cloudnull | ok. | 13:42 |
| cloudnull | I should be able to test that with an AIO on a new 42.3 build? | 13:42 |
| cloudnull | I guess I can just run tests on git fetch https://git.openstack.org/openstack/openstack-ansible-os_cinder refs/changes/98/638198/2 && git checkout FETCH_HEAD | 13:43 |
| cloudnull | which is what is failing | 13:43 |
| cloudnull | in that ara output | 13:43 |
| mnaser | also opensuse-150 is failing in the integrated gate due to missing images | 13:43 |
| hwoarang | mnaser: do you have a log handy? | 13:43 |
| mnaser | https://review.openstack.org/#/c/641911/ | 13:43 |
| mnaser | http://logs.openstack.org/11/641911/1/check/openstack-ansible-deploy-aio_lxc-opensuse-150/983497c/ | 13:44 |
| hwoarang | the image is there | 13:44 |
| hwoarang | just the name has changed | 13:44 |
| hwoarang | let me confirm this change is permanent | 13:45 |
| odyssey4me | mnaser yeah, I saw your oslo patch - it sucks a bit that this has been around for so long! | 13:46 |
| odyssey4me | well found though - damn! | 13:46 |
| mnaser | odyssey4me: yeah i was exposed when we dropped vif_plugging_is_fatal = false | 13:48 |
| mnaser | which was non default | 13:48 |
| mnaser | but yeah, that's pretty major. | 13:48 |
| jrosser | logan-: are there keystone logs that we don't collect, or is it stuff now buried in journals? | 13:48 |
| cloudnull | hwoarang is there any specific image I should build opensuse 423 images from? or would a `zypper dup` be good enough to update my base image and run the test? | 13:49 |
| odyssey4me | mnaser hmm, I wonder why we had that in the first place | 13:49 |
| mnaser | odyssey4me: git blame shows it being done pre-osa existing | 13:50 |
| logan- | jrosser: maybe buried in journals, yeah | 13:50 |
| mnaser | but it masks a lot of issues obviously | 13:50 |
| odyssey4me | I have vague memories of that relating to either something to do with some linuxbridge driver bug, or something to do with the LXD driver. | 13:50 |
| odyssey4me | It may just be some sort of legacy. | 13:50 |
| mnaser | odyssey4me: i don't have much history because i can see it being there since we the roles were split | 13:51 |
| mnaser | odyssey4me: https://github.com/openstack/openstack-ansible/blame/cfe01cb2a6144be747dc89155f624f7a80ab6398/rpc_deployment/roles/nova_common/templates/nova.conf that's as far back as i can go | 13:54 |
| mnaser | and i don't see why :) | 13:54 |
| odyssey4me | mnaser well, I just dug all the way back to juno and it's there from 'initial commit'... I even went to the original rcbops git repo, and same deal | 13:55 |
| hwoarang | cloudnull: the 42.3 images are fetched from https://us.images.linuxcontainers.org/meta/1.0/index-system as far as I can see | 13:56 |
| mnaser | haha | 13:56 |
| mnaser | odyssey4me: forever a mystery :) | 13:56 |
| *** hamzaachi has quit IRC | 13:57 | |
| *** hamzaachi has joined #openstack-ansible | 13:57 | |
| * mnaser sees cloudnull push up a patch "improve deployments on 14.04" | 13:58 | |
| * mnaser hugs cloudnull | 13:58 | |
| mnaser | it's gonna b eok. | 13:58 |
| *** hamzaachi has quit IRC | 13:58 | |
| cloudnull | thank you :) | 13:58 |
| hwoarang | cloudnull: sorry the networking issue only affects queens. latest releases are fetching images from elsewhere | 13:58 |
| * cloudnull goes back to my hidey-hole | 13:58 | |
| *** hamzaachi has joined #openstack-ansible | 14:00 | |
| *** hamzaachi has quit IRC | 14:01 | |
| cloudnull | hwoarang cool. im all setup and will give it a bash | 14:01 |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: improve deployments on 14.04 https://review.openstack.org/641901 | 14:01 |
| *** marst has joined #openstack-ansible | 14:03 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_hosts master: SUSE: Update image name for Leap 15 https://review.openstack.org/642025 | 14:06 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_hosts master: SUSE: Update image names for Leap 42.3 and 15.0 https://review.openstack.org/642025 | 14:06 |
| hwoarang | mnaser: ^ will keep an eye on it but i am on holidays next two weeks so either I make sure it gets green today or i would rely to be dear colleage evrardjp to follow up | 14:07 |
| * hwoarang gives evrardjp some belgian chocolate | 14:07 | |
| *** BjoernT has joined #openstack-ansible | 14:08 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_hosts master: SUSE: Update image names for Leap 42.3 and 15.0 https://review.openstack.org/642025 | 14:10 |
| *** BjoernT_ has joined #openstack-ansible | 14:11 | |
| *** BjoernT has quit IRC | 14:13 | |
| guilhermesp | chkumar|pto: hey man, do you have a clue why these tempest tests are failing? http://logs.openstack.org/45/641645/3/check/openstack-ansible-functional-centos-7/24362f8/logs/ara-report/result/95b06402-396e-4f84-9104-423cf0cabaa8/ It was passing some weeks ago here https://review.openstack.org/#/c/634991/ | 14:16 |
| guilhermesp | asked in ppl on openstack-mistral but I haven't had any response from them yet | 14:16 |
| *** markvoelker has joined #openstack-ansible | 14:20 | |
| *** hamzaachi has joined #openstack-ansible | 14:21 | |
| *** hamzaachi has quit IRC | 14:27 | |
| *** hamzaachi has joined #openstack-ansible | 14:27 | |
| *** irclogbot_0 has joined #openstack-ansible | 14:33 | |
| *** pcaruana has joined #openstack-ansible | 14:37 | |
| *** phasespace has quit IRC | 14:41 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_hosts master: SUSE: Update image names for Leap 42.3 and 15.0 https://review.openstack.org/642025 | 14:54 |
| *** markvoelker has quit IRC | 14:54 | |
| openstackgerrit | Luigi Toscano proposed openstack/openstack-ansible-os_sahara master: Install also the Sahara plugins https://review.openstack.org/642041 | 14:55 |
| tosky | let's see if I've got it right... | 14:55 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Convert git.openstack.org to opendev.org https://review.openstack.org/642049 | 15:09 |
| *** waverider has quit IRC | 15:13 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Convert git.openstack.org to opendev.org https://review.openstack.org/642049 | 15:20 |
| *** udesale has quit IRC | 15:21 | |
| *** udesale has joined #openstack-ansible | 15:21 | |
| cloudnull | hwoarang odyssey4me sadly on my local test instance - http://logs.openstack.org/98/638198/2/check/openstack-ansible-functional-opensuse-423/d860e2d/logs/ara-report/result/1a65f714-597b-407c-ab5e-4f27a968f5a4/ - is not happening ... | 15:23 |
| cloudnull | running the change - git fetch https://git.openstack.org/openstack/openstack-ansible-os_cinder refs/changes/98/638198/2 && git checkout FETCH_HEAD | 15:23 |
| cloudnull | which should be testing the cinder commit thats failing in queens | 15:24 |
| cloudnull | currently on "TASK [os_cinder : Unarchive pre-built venv]" | 15:24 |
| *** cmart has joined #openstack-ansible | 15:24 | |
| cloudnull | https://pasted.tech/pastes/e528c167339dfcbc880175684c3561cfa532d7a1 | 15:25 |
| odyssey4me | cloudnull hmm, I wonder if the issue is something to do with the reverse proxy that we use in openstack-infra then | 15:25 |
| odyssey4me | that's quite possible, given that I think the config for them has restricted paths... and the paths possibly changed | 15:26 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-lxc_hosts master: SUSE: Update image names for Leap 42.3 and 15.0 https://review.openstack.org/642025 | 15:29 |
| cloudnull | odyssey4me could be? | 15:44 |
| cloudnull | going to try again to get it to fail | 15:45 |
| cloudnull | this time without first updating my base image | 15:45 |
| jrosser | is it the case that everything that we run as a systemd service, which doesn’t create its own independent log file we don’t collect any log in CI? | 15:47 |
| *** hamzaachi_ has joined #openstack-ansible | 15:47 | |
| *** hamzaachi has quit IRC | 15:48 | |
| cloudnull | we should get a log file dump of the journal in ci | 15:49 |
| *** marst has quit IRC | 15:50 | |
| jrosser | But for a service like keystone, for example? | 15:50 |
| cloudnull | like so http://logs.openstack.org/91/634991/8/check/openstack-ansible-functional-opensuse-423/683963d/logs/openstack/openstack1/keystone.slice.journal.log.txt.gz | 15:50 |
| jrosser | Oh - hmmm | 15:50 |
| *** markvoelker has joined #openstack-ansible | 15:51 | |
| *** luksky has quit IRC | 15:51 | |
| cloudnull | so it should be there, however if its not then i'd say its a bug | 15:51 |
| jrosser | Oh but that’s a role test | 15:52 |
| cloudnull | yes, maybe that wasn't implemented in the integrated gate? | 15:52 |
| jrosser | Unless I’ve missed something obvious, could well be | 15:53 |
| cloudnull | https://github.com/openstack/openstack-ansible-tests/blob/master/test-log-collect.sh this is what we do in the role tests | 15:56 |
| cloudnull | i remember we had something similar in the integrated test | 15:56 |
| cloudnull | maybe that just needs an update | 15:56 |
| mnaser | #killtheroletests | 16:04 |
| jrosser | cloudnull: i think the role tests are much smarter about collecting the service journals | 16:17 |
| jrosser | we need to port that across | 16:17 |
| cloudnull | looking at that now | 16:18 |
| cloudnull | looks like the integrated tests are just running the old log_instance_info function | 16:19 |
| cloudnull | seems like we can port over the role test script, almost as is. | 16:20 |
| jrosser | that would be great, would help a bunch | 16:20 |
| *** cshen has quit IRC | 16:22 | |
| *** markvoelker has quit IRC | 16:23 | |
| *** miloa has quit IRC | 16:25 | |
| *** gkadam has quit IRC | 16:33 | |
| hwoarang | cloudnull: do you know what this means http://logs.openstack.org/25/642025/5/check/openstack-ansible-functional-opensuse-150/22a798a/job-output.txt.gz#_2019-03-08_15_51_50_217962 ? | 16:37 |
| cloudnull | hwoarang looks like opt doesnt exist in that container image | 16:49 |
| cloudnull | the prep script is placed there here : https://github.com/openstack/openstack-ansible-lxc_hosts/blob/468cffb585ab0534aa0e90a7dfd53dfefc5f1e19/tasks/lxc_cache_preparation.yml#L107 | 16:49 |
| hwoarang | yeah very strange | 16:49 |
| hwoarang | maybe we should ensure that the dir is created :) | 16:49 |
| cloudnull | ++ | 16:49 |
| cloudnull | we could change it to tmp ? | 16:49 |
| cloudnull | or something else if that's at all helpful | 16:50 |
| hwoarang | yeah possibly whatever works | 16:50 |
| cloudnull | maybe just a task to ensure the opt dir exists is the easiest path forward there. | 16:51 |
| jrosser | mnaser: proxy test passed this time :/ https://review.openstack.org/#/c/641911/ | 16:59 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_hosts master: SUSE: Update image names for Leap 42.3 and 15.0 https://review.openstack.org/642025 | 16:59 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_hosts master: tasks: lxc_cache_preparation: Ensure the /opt directory exists in the container https://review.openstack.org/642079 | 16:59 |
| jrosser | we've got something else lurking - though if that service restart bug is affecting more than just nova ....... | 17:00 |
| jrosser | restart/reload | 17:00 |
| *** udesale has quit IRC | 17:04 | |
| *** cmart has quit IRC | 17:06 | |
| *** aedc has quit IRC | 17:08 | |
| *** aedc has joined #openstack-ansible | 17:09 | |
| *** pcaruana has quit IRC | 17:12 | |
| *** jeh has joined #openstack-ansible | 17:13 | |
| *** ppetit has quit IRC | 17:15 | |
| *** markvoelker has joined #openstack-ansible | 17:20 | |
| mnaser | jrosser: it does, it breaks neutron too, there's a reported bug for that | 17:21 |
| *** gyee has joined #openstack-ansible | 17:22 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Port role test log collection into the integrated gate https://review.openstack.org/642083 | 17:23 |
| cloudnull | jrosser mnaser ^ | 17:23 |
| mnaser | neat, we won't be able to get it to merge because of currently broken gate, but hopefully we'll get it that cleared out soon | 17:24 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Port role test log collection into the integrated gate https://review.openstack.org/642083 | 17:26 |
| cloudnull | forgot to add back ara. | 17:26 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_hosts master: tasks: lxc_cache_preparation: Ensure the /opt directory exists in the container https://review.openstack.org/642079 | 17:26 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_hosts master: SUSE: Update image names for Leap 42.3 and 15.0 https://review.openstack.org/642025 | 17:26 |
| *** hamzaachi_ has quit IRC | 17:28 | |
| *** asettle is now known as away | 17:46 | |
| *** away is now known as asettle | 17:46 | |
| *** markvoelker has quit IRC | 17:54 | |
| *** cmart has joined #openstack-ansible | 17:56 | |
| *** electrofelix has quit IRC | 18:04 | |
| *** aedc has quit IRC | 18:05 | |
| *** aedc has joined #openstack-ansible | 18:05 | |
| *** ppetit has joined #openstack-ansible | 18:10 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-tests master: zuul: move non-voting jobs to experimental https://review.openstack.org/642104 | 18:21 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: nova: restart instead of reloading services https://review.openstack.org/641911 | 18:28 |
| mnaser | jrosser, cloudnull, hwoarang: ^ fyi | 18:28 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible master: nova: restart instead of reloading services https://review.openstack.org/641911 | 18:30 |
| *** hamzaachi has joined #openstack-ansible | 18:41 | |
| *** markvoelker has joined #openstack-ansible | 18:51 | |
| mnaser | cloudnull: https://review.openstack.org/#/c/642104/ would be nice to push through | 19:09 |
| cloudnull | done | 19:10 |
| mnaser | cloudnull: neat, thanks, i hope to clean up all of this gate stuff soon | 19:10 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Port role test log collection into the integrated gate https://review.openstack.org/642083 | 19:12 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Port role test log collection into the integrated gate https://review.openstack.org/642083 | 19:13 |
| *** markvoelker has quit IRC | 19:24 | |
| cloudnull | mnaser logan- jrosser evrardjp odyssey4me d34dh0r53 guilhermesp , and everyone else https://review.openstack.org/#/q/topic:fix/include/deprecation+(status:open+OR+status:merged) - I wanted to bring these up at the meeting but life was happening and I wasnt able to attend | 19:29 |
| cloudnull | essentially those are the last of the deprecation warnings in the integrated repo | 19:29 |
| cloudnull | however the impact is that the serialization for the core services that we supported it on is gone. | 19:30 |
| cloudnull | not to say we cant readd it a different way | 19:30 |
| mnaser | hmm | 19:30 |
| cloudnull | but the current implementation wont work | 19:30 |
| mnaser | the biggest impact is potentially on upgrades, right? | 19:30 |
| cloudnull | yes, potentially | 19:30 |
| mnaser | the good thing is we actually have working upgrade jobs | 19:31 |
| cloudnull | not the roles dont currenty test with any of that serialization | 19:31 |
| cloudnull | **now the | 19:31 |
| cloudnull | but yes all that is gone . | 19:31 |
| mnaser | https://review.openstack.org/#/c/627782/ | 19:31 |
| cloudnull | we'd need to readd it differently | 19:31 |
| mnaser | IMHO | 19:31 |
| mnaser | i feel like the serialization should happen in the role | 19:31 |
| cloudnull | I share that opinion, and for the most part it does | 19:32 |
| mnaser | and not in the playbooks? | 19:32 |
| cloudnull | however this is the way we've done it for now, which may just be overkill | 19:32 |
| mnaser | i mean if we deploy, run tempest, upgrade, run tempest again.. and it's working | 19:32 |
| cloudnull | so I agree with odyssey4me, its something we need to collectively agree on ' | 19:32 |
| *** ppetit has quit IRC | 19:33 | |
| cloudnull | if any of those patches are accecpted as is | 19:33 |
| mnaser | cloudnull: do you think we can maybe bring this to the ML given we're all a bit distributed and real life happening? | 19:33 |
| cloudnull | sure | 19:33 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_manila master: Basic working os_manila role https://review.openstack.org/611930 | 19:41 |
| *** waverider has joined #openstack-ansible | 19:45 | |
| *** hamzaachi_ has joined #openstack-ansible | 19:50 | |
| *** hamzaachi_ has quit IRC | 19:51 | |
| *** hamzaachi has quit IRC | 19:52 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: zuul: move non-voting jobs to experimental https://review.openstack.org/642104 | 19:53 |
| *** hamzaachi has joined #openstack-ansible | 19:55 | |
| *** hamzaachi_ has joined #openstack-ansible | 19:57 | |
| *** jbadiapa has quit IRC | 19:59 | |
| *** hamzaachi has quit IRC | 20:00 | |
| guilhermesp | cloudnull: taking a look at all the patches | 20:02 |
| logan- | cloudnull: i don't understand the patches.. the whole layout should not need to be changed right? the syntax just need to be modernized by changing include: <playbook> to import_playbook: <playbook> and the function remains the same. | 20:09 |
| cloudnull | sadly import_playbook cant accept variables | 20:09 |
| *** jeh has quit IRC | 20:10 | |
| cloudnull | if they had an include_playbook it could but that's not a valid option | 20:10 |
| logan- | https://pasted.tech/pastes/a39d1a3c44e942160a9c5b2d2f8d3c8ac71c2da4 ? | 20:10 |
| guilhermesp | also cloudnull we need to figure out how to deal using tags with the new layout, I remember some issues regarding include_* about it | 20:11 |
| cloudnull | logan- mind trying "nova_serial: "{{ nova_conductor_serial | default(['1', '100%']) }}"" as a variable | 20:12 |
| cloudnull | maybe thats just an undocumented function ? | 20:13 |
| logan- | yep, one sec | 20:13 |
| * cloudnull on moble | 20:13 | |
| *** irclogbot_0 has quit IRC | 20:13 | |
| guilhermesp | just to document https://bugs.launchpad.net/openstack-ansible/+bug/1815043 in bug squash day we discussed about the behavior of include_*/import_* and the use of tags. Maybe is not strictly related or crucial to be aware now, but may be considered in the future | 20:14 |
| openstack | Launchpad bug 1815043 in openstack-ansible "running os-nova-install.yml --tags nova-config does note generate config" [Undecided,New] | 20:14 |
| logan- | cloudnull: seems to work https://pasted.tech/pastes/85ffda843d4af3cd247914677917473a2ce230a9 | 20:15 |
| *** cmart has quit IRC | 20:15 | |
| *** cshen has joined #openstack-ansible | 20:16 | |
| cloudnull | so maybe that all works after all. | 20:17 |
| cloudnull | so I could just do the quick sed | 20:17 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Port role test log collection into the integrated gate https://review.openstack.org/642083 | 20:19 |
| *** cshen has quit IRC | 20:20 | |
| *** markvoelker has joined #openstack-ansible | 20:21 | |
| cloudnull | logan- can you pass that variable into the sub playbook ? | 20:23 |
| cloudnull | ** into the groups of the sub playbook | 20:24 |
| logan- | checking | 20:25 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Convert the nova playbook to use newer ansible syntax https://review.openstack.org/642143 | 20:25 |
| cloudnull | ^ that's the simpler version if all that works | 20:26 |
| * cloudnull editing things on a tablet is hard. | 20:26 | |
| logan- | https://pasted.tech/pastes/6c60480f3afa202eb4064c97c1777df78d11e750.raw | 20:27 |
| logan- | looks good ^ | 20:28 |
| cloudnull | nice! | 20:28 |
| guilhermesp | hum so with logan's test all of these https://review.openstack.org/#/q/topic:fix/include/deprecation+(status:open+OR+status:merged) could be converted to https://review.openstack.org/642143 ? | 20:33 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Convert the nova playbook to use newer ansible syntax https://review.openstack.org/642143 | 20:37 |
| cloudnull | guilhermesp yes, it looks like based on logan- 's test we can convert https://review.openstack.org/#/q/topic:fix/include/deprecation+(status:open) to just import_playbook | 20:38 |
| cloudnull | and we can deal with the serialization, if we want to later. | 20:39 |
| guilhermesp | seems cooler! | 20:41 |
| *** phasespace has joined #openstack-ansible | 20:44 | |
| fnpanic | hi | 20:48 |
| *** fdz is now known as francois | 20:50 | |
| *** aedc has quit IRC | 20:50 | |
| *** aedc has joined #openstack-ansible | 20:51 | |
| *** markvoelker has quit IRC | 20:53 | |
| *** DanyC has quit IRC | 21:09 | |
| *** ansmith has quit IRC | 21:11 | |
| *** hamzaachi_ has quit IRC | 21:19 | |
| *** DanyC has joined #openstack-ansible | 21:23 | |
| *** cmart has joined #openstack-ansible | 21:25 | |
| *** strattao has quit IRC | 21:26 | |
| openstackgerrit | Guilherme Steinmuller Pimentel proposed openstack/openstack-ansible-os_mistral master: Remove deprecated auth_uri https://review.openstack.org/642161 | 21:35 |
| *** luksky has joined #openstack-ansible | 21:35 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Port role test log collection into the integrated gate https://review.openstack.org/642083 | 21:50 |
| *** markvoelker has joined #openstack-ansible | 21:51 | |
| *** cmart has quit IRC | 21:53 | |
| *** grabes_ has joined #openstack-ansible | 21:57 | |
| *** cmart has joined #openstack-ansible | 22:15 | |
| *** cshen has joined #openstack-ansible | 22:16 | |
| *** cshen has quit IRC | 22:21 | |
| guilhermesp | Appreciate reviews to unblock os_mistral https://review.openstack.org/642161 :) | 22:23 |
| *** markvoelker has quit IRC | 22:24 | |
| cloudnull | guilhermesp you'll need one more +2 | 22:25 |
| cloudnull | but LGTM | 22:25 |
| guilhermesp | Thanks cloudnull ! | 22:29 |
| guilhermesp | mnaser :) | 22:29 |
| guilhermesp | Thanks logan- ! | 22:42 |
| logan- | np | 22:43 |
| *** grabes_ has quit IRC | 22:49 | |
| *** waverider has quit IRC | 22:58 | |
| *** irclogbot_0 has joined #openstack-ansible | 23:01 | |
| openstackgerrit | Merged openstack/openstack-ansible master: nova: restart instead of reloading services https://review.openstack.org/641911 | 23:04 |
| cloudnull | jrosser mnaser looks like https://review.openstack.org/#/c/642083/ does what we want with regard to log collection - http://logs.openstack.org/83/642083/6/check/openstack-ansible-deploy-aio_metal-ubuntu-bionic/1b85be6/logs/ | 23:11 |
| mnaser | no keystone logs still but i guess thats gone in the jorunals | 23:12 |
| mnaser | im not sure how we can see those logs | 23:12 |
| cloudnull | hum good call. I saw the journals on the host, but the container journals are not there. | 23:13 |
| cloudnull | looking now | 23:13 |
| logan- | cinder also | 23:13 |
| *** BjoernT_ has quit IRC | 23:14 | |
| NobodyCam | has anyone been able to use the HAProxy self generated to deploy a cluser | 23:18 |
| NobodyCam | cluster even | 23:18 |
| NobodyCam | I'm hitting `controllermanager.go:169] error building controller context: cloud provider could not be initialized: could not init cloud provider "openstack": Post https://10.21.100.198:5000/v3/auth/tokens: x509: cannot validate certificate for 10.21.100.198 because it doesn't contain any IP SANs` | 23:19 |
| NobodyCam | if I dump the cert I do see the san ip | 23:19 |
| NobodyCam | `Issuer: C = US, ST = Texas, L = San Antonio, O = IT, CN = 10.21.100.198, subjectAltName = IP.1=10.21.100.198` | 23:20 |
| NobodyCam | I added the cert to ../certifi/cacerts.pem | 23:21 |
| NobodyCam | this works for all the Openstack services | 23:21 |
| NobodyCam | but failed with kubernetes | 23:21 |
| *** markvoelker has joined #openstack-ansible | 23:21 | |
| cloudnull | logan- cinder was there, strange that keystone was not. but maybe there was no journal to pull from ? | 23:29 |
| openstackgerrit | Merged openstack/openstack-ansible-os_mistral master: Remove deprecated auth_uri https://review.openstack.org/642161 | 23:34 |
| *** KenCrandall has quit IRC | 23:37 | |
| *** markvoelker has quit IRC | 23:54 | |
| *** DanyC has quit IRC | 23:58 | |
| *** DanyC has joined #openstack-ansible | 23:58 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Port role test log collection into the integrated gate https://review.openstack.org/642083 | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!