| *** TSCHAK has quit IRC | 00:13 | |
| *** fpan has joined #tripleo | 00:17 | |
| *** fpan has quit IRC | 00:18 | |
| *** fpan has joined #tripleo | 00:18 | |
| *** saneax is now known as saneax-_-|AFK | 00:22 | |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci: [DO NOT MERGE] Test a fake periodic job, newton 3 https://review.openstack.org/365802 | 00:36 |
|---|---|---|
| *** limao has joined #tripleo | 00:41 | |
| *** tremble has quit IRC | 00:52 | |
| *** tremble has joined #tripleo | 00:53 | |
| *** tremble has quit IRC | 00:53 | |
| *** tremble has joined #tripleo | 00:53 | |
| *** yamahata has joined #tripleo | 01:21 | |
| *** akshai has joined #tripleo | 01:56 | |
| *** akshai_ has joined #tripleo | 02:29 | |
| *** akshai has quit IRC | 02:32 | |
| *** rhallisey has quit IRC | 02:33 | |
| stevebaker | flaper87: hey, ramishra and I would like to chat about the docker-cmd hook when you have the chance | 02:36 |
| *** limao has quit IRC | 02:41 | |
| *** limao has joined #tripleo | 02:43 | |
| *** limao has quit IRC | 02:46 | |
| *** limao has joined #tripleo | 02:46 | |
| *** bkopilov has quit IRC | 02:50 | |
| *** akshai_ has quit IRC | 02:52 | |
| *** tzumainn has joined #tripleo | 02:59 | |
| *** tzumainn has quit IRC | 03:11 | |
| *** bana_k has joined #tripleo | 03:41 | |
| *** coolsvap_ has joined #tripleo | 03:52 | |
| *** masco has joined #tripleo | 04:11 | |
| *** pgadiya has joined #tripleo | 04:17 | |
| *** bkopilov has joined #tripleo | 04:24 | |
| *** abregman has joined #tripleo | 04:37 | |
| *** links has joined #tripleo | 04:39 | |
| *** bana_k has quit IRC | 04:39 | |
| *** bana_k has joined #tripleo | 04:58 | |
| *** dsariel has joined #tripleo | 05:10 | |
| *** skramaja has joined #tripleo | 05:11 | |
| *** jaosorior has joined #tripleo | 05:21 | |
| *** cwolferh has joined #tripleo | 05:22 | |
| *** jprovazn has joined #tripleo | 05:22 | |
| *** florianf has joined #tripleo | 05:30 | |
| *** liverpooler has quit IRC | 05:42 | |
| *** oshvartz has joined #tripleo | 05:46 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: WIP -- clean full upgrade review https://review.openstack.org/364859 | 05:48 |
| *** saneax-_-|AFK is now known as saneax | 05:48 | |
| bandini | morning | 05:49 |
| jaosorior | bandini: sup dude | 05:51 |
| *** atom_ has quit IRC | 05:53 | |
| *** mcornea has joined #tripleo | 05:54 | |
| bandini | jaosorior: hola, que tal? | 05:55 |
| *** abregman has quit IRC | 05:56 | |
| *** abregman has joined #tripleo | 05:57 | |
| jaosorior | bandini: nada mal, aquí tratando empezando la mañana... me hace falta café. Tu que tal? | 05:57 |
| bandini | jaosorior: lol a mi me falta cafe tambien :D | 05:57 |
| bandini | me voy a tomar uno ;) | 05:58 |
| jaosorior | bandini: que tal vas de trabajo? | 05:59 |
| bkero | quiero un cafe ahora :( | 06:00 |
| jaosorior | bkero: jaja no sabía que tu también hablas español | 06:01 |
| openstackgerrit | Saravanan KR proposed openstack/python-tripleoclient: Tripleoclient leaks temporary files https://review.openstack.org/330638 | 06:02 |
| bkero | jaosorior: perdon, es un lengua segunda pero es la primera lengua por Barcelona | 06:06 |
| bkero | leguna seguna para mi :) | 06:06 |
| bkero | lengua* | 06:06 |
| jaosorior | bkero: bien ahí! | 06:11 |
| bkero | jaosorior: vives en espana? | 06:12 |
| bkero | perdon para los accentos, mi teclado no tengo | 06:14 |
| jaosorior | bkero: no pasa nada | 06:14 |
| jaosorior | bkero: vivo en Finlandia, pero soy mexicano | 06:15 |
| bkero | oooo cool | 06:15 |
| jaosorior | bkero: tu donde vives? | 06:16 |
| bkero | jaosorior: estados unidos, y tambien | 06:17 |
| *** anshul has joined #tripleo | 06:17 | |
| jaosorior | y tambien? | 06:17 |
| bkero | soy uno americano | 06:18 |
| bkero | un* | 06:18 |
| *** hjensas has quit IRC | 06:18 | |
| bkero | -_- | 06:18 |
| jaosorior | bkero: cuanto llevas estudiando español? | 06:19 |
| bkero | jaosorior: si, en un universidad comunitario y Duolingo.com. Mi profesora en universidad era Colombina, no habla Ingles | 06:22 |
| bkero | excepto Madonna + Evita :) | 06:23 |
| *** nyechiel_ has joined #tripleo | 06:24 | |
| bkero | person, es tarde en la noche aqui. buenos noches! | 06:26 |
| jaosorior | bkero: va, saludos | 06:27 |
| *** pcaruana has joined #tripleo | 06:48 | |
| *** tremble has quit IRC | 06:50 | |
| jaosorior | shadower: you around yet? | 06:53 |
| *** tremble has joined #tripleo | 06:54 | |
| shadower | jaosorior: just logged in :-) | 06:54 |
| shadower | what's up | 06:54 |
| jaosorior | shadower: haha was gonna poke you about reviews :P | 06:54 |
| jaosorior | shadower: got time? | 06:55 |
| shadower | jaosorior: sure | 06:55 |
| shadower | the fqdn stuff? | 06:56 |
| jaosorior | shadower: that indeed | 06:56 |
| shadower | will do | 06:56 |
| jaosorior | shadower: and this one https://review.openstack.org/#/c/365522/5 | 06:56 |
| shadower | yeah | 06:57 |
| *** zoli_gone-proxy is now known as zoliXXL | 06:57 | |
| *** dsariel has quit IRC | 06:57 | |
| *** mhenkel has joined #tripleo | 07:00 | |
| *** tesseract- has joined #tripleo | 07:00 | |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-quickstart: WIP gate upgrade https://review.openstack.org/342161 | 07:04 |
| *** jpena|off is now known as jpena | 07:08 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts https://review.openstack.org/357765 | 07:09 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add VIP names to allNodesConfig https://review.openstack.org/365895 | 07:09 |
| *** liverpooler has joined #tripleo | 07:10 | |
| *** hjensas has joined #tripleo | 07:11 | |
| *** hjensas has quit IRC | 07:11 | |
| *** hjensas has joined #tripleo | 07:11 | |
| openstackgerrit | Tomas Sedovic proposed openstack/tripleo-validations: Check for differences between introspected HW https://review.openstack.org/353978 | 07:16 |
| *** ifarkas_afk is now known as ifarkas | 07:16 | |
| *** leanderthal|afk is now known as leanderthal | 07:19 | |
| *** ebarrera has joined #tripleo | 07:20 | |
| shadower | jaosorior: oh man that's some heavy yaqs/heat templating | 07:21 |
| jaosorior | shadower: which commit? | 07:22 |
| *** fzdarsky has joined #tripleo | 07:22 | |
| shadower | Move VIP-related data to AllNodesConfig https://review.openstack.org/#/c/365522/ | 07:22 |
| jaosorior | shadower: haha well, pretty much got it from shardy's work :P http://hardysteven.blogspot.fi/2016/09/complex-data-transformations-with.html | 07:23 |
| shadower | jaosorior: I haven't read that but plan to | 07:23 |
| jaosorior | shadower: aaaah, the VIP stuff is pretty much a copy paste of what was already in vip-config, so actually nothing really changed there | 07:24 |
| shadower | jaosorior: oh! I didn't see the deleted file yet. | 07:24 |
| jaosorior | shadower: so actually I didn't change much, except that it's now deployed in all the nodes instead of just the controllers | 07:25 |
| shadower | cool | 07:25 |
| jaosorior | shadower: the stuff that changes is the last bits of the vip-config | 07:25 |
| jaosorior | shadower: the stuff that sets the virtual ip related stuff (keystone_public_api_vip, admin_vip and that stuff | 07:25 |
| jaosorior | shadower: so instead of coming from an input, it's now calculated in-place | 07:25 |
| shadower | right, I see | 07:26 |
| *** zoliXXL is now known as zoli|brb | 07:28 | |
| *** jpich has joined #tripleo | 07:30 | |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Fix error when identity file is missing https://review.openstack.org/365906 | 07:31 |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Handle missing mistral environment https://review.openstack.org/365907 | 07:31 |
| *** jbadiapa has joined #tripleo | 07:32 | |
| jaosorior | jistr: hey dude, so I ended up changing the submission that set the cloudname entries for /etc/hosts. Now I don't even change the endpoint map (which is what people where doubting the most), I just get everything via hiera | 07:33 |
| mandre | shadower: ^ a couple of bug fixes for the run validations workflow when it is called before we create the SSH keys | 07:33 |
| shadower | mandre: I'll have a look, thanks | 07:34 |
| *** aufi has joined #tripleo | 07:34 | |
| *** abregman has quit IRC | 07:35 | |
| *** zoli|brb is now known as zoliXXL | 07:37 | |
| *** jlinkes has joined #tripleo | 07:39 | |
| openstackgerrit | Merged openstack/tripleo-ui: Add app.conf to .gitignore https://review.openstack.org/365618 | 07:42 |
| openstackgerrit | Christopher Brown proposed openstack/tripleo-docs: Change version to fix wwn extensions support https://review.openstack.org/365918 | 07:48 |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: WIP -- clean full upgrade review https://review.openstack.org/364859 | 07:49 |
| *** bana_k has quit IRC | 07:49 | |
| *** dbecker has joined #tripleo | 07:49 | |
| *** abregman has joined #tripleo | 07:51 | |
| jaosorior | marios, shadower, jistr: Do you guys mind if I merge this? https://review.openstack.org/#/c/365522/ | 07:52 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Implement Validation Detail modal https://review.openstack.org/365921 | 07:56 |
| shadower | jaosorior: fine by me, but I should not be merging non-validation stuff yet (which is why I only +2) | 07:56 |
| jaosorior | shadower: no biggie | 07:56 |
| shadower | I'm trying to do more non-validation reviews to get familiar & be able to +A again | 07:57 |
| jaosorior | shadower: makes sense man | 07:57 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Customize undercloud and overcloud with virt-customize https://review.openstack.org/361112 | 07:59 |
| marios | jaosorior: no objection from me | 08:00 |
| jtomasek | shadower: this is how the validation detail looks in GUI now http://imgur.com/a/PGchd | 08:00 |
| *** bvandenh has joined #tripleo | 08:02 | |
| *** dbecker has quit IRC | 08:02 | |
| shadower | jtomasek: right, so it's just the raw output from Ansible. Can you run the "discovery diff" one? It should show a bunch of warnings | 08:03 |
| * shadower wants to be sure those are displayed, too | 08:04 | |
| *** abregman has quit IRC | 08:04 | |
| jtomasek | shadower: which one is it? is that one in tripleo-validations already? | 08:05 |
| *** abregman has joined #tripleo | 08:05 | |
| shadower | jtomasek: ah right. It's not :-( | 08:05 |
| shadower | jtomasek: if I sent you a patch, could you add it in easily? | 08:06 |
| shadower | (don't bother for now if you can't) | 08:06 |
| jtomasek | shadower: where is that patch? | 08:07 |
| shadower | jtomasek: https://review.openstack.org/#/c/353978/ | 08:07 |
| *** jistr is now known as jistr|dentist | 08:07 | |
| shadower | jtomasek: hm, it's failing for me now | 08:09 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add HAProxy TLS handled by certmonger as composable service https://review.openstack.org/356430 | 08:09 |
| *** ohamada has joined #tripleo | 08:11 | |
| jtomasek | shadower: ok, tested it, and fails too. I can check it again whenever you want | 08:12 |
| *** skramaja has quit IRC | 08:13 | |
| shadower | jtomasek: yeah, thanks. It's not the validation itself but seems we have a bug there somewhere. Investigating now | 08:13 |
| *** skramaja has joined #tripleo | 08:13 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: IGNORE - testing manila-generic deploy after tidyup https://review.openstack.org/362866 | 08:14 |
| *** derekh has joined #tripleo | 08:15 | |
| *** lucas-dinner is now known as lucasagomes | 08:16 | |
| *** b00tcat has joined #tripleo | 08:21 | |
| b00tcat | Hi, I'm following the tripleo docs and am now in the "Basic Deployment (CLI)" section - there it says that I should install the trunk delorean repos inside the instack-undercloud machine | 08:22 |
| b00tcat | however in the previous step I installed the `delorean-mitaka` ones, is this step mandatory? | 08:22 |
| b00tcat | forget that, I read below and this is already covered for stable-mitaka :D | 08:25 |
| flepied | periodic job succeeded to promote last night or was it promoted by hand? | 08:30 |
| *** nyechiel_ has quit IRC | 08:31 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts https://review.openstack.org/357765 | 08:33 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add HAProxy TLS handled by certmonger as composable service https://review.openstack.org/356430 | 08:33 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add parameters for internal TLS https://review.openstack.org/365942 | 08:33 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add parameters for internal TLS https://review.openstack.org/365942 | 08:35 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add HAProxy TLS handled by certmonger as composable service https://review.openstack.org/356430 | 08:35 |
| *** bvandenh has quit IRC | 08:37 | |
| shadower | mandre: just discovered this: https://review.openstack.org/#/c/353978/ (see my last comment) | 08:42 |
| shadower | mandre: would appreciate your feedback | 08:42 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Move VIP-related data to AllNodesConfig https://review.openstack.org/365522 | 08:44 |
| openstackgerrit | Merged openstack/tripleo-common: Prevent replacement of Port, Net and Subnet https://review.openstack.org/353411 | 08:45 |
| *** nyechiel_ has joined #tripleo | 08:45 | |
| *** athomas has joined #tripleo | 08:46 | |
| *** abregman has quit IRC | 08:46 | |
| shadower | jtomasek: so a quick fix is to edit /etc/ansible/ansible.cfg, find "sudo_flags" and set it to "-H -S -n -E" | 08:48 |
| mandre | shadower: have you tried the solution #3 you proposed? | 08:48 |
| shadower | mandre: yeah, and it works fine | 08:48 |
| shadower | mandre: actually, I haven't tried #1 but can't see why it wouldn't work. #2 works fine, too | 08:49 |
| mandre | shadower: rather than edit /etc/ansible/ansible.cfg, I prefer you export a variable in run-validations script | 08:49 |
| mandre | shadower: that's the general pattern I adopted to configure ansible | 08:49 |
| shadower | mandre: ah, good point. Not sure that's possible here (but I think so) | 08:50 |
| shadower | mandre: one problem with that is we can no longer use Ansible directly | 08:50 |
| shadower | hm but we can just use the run-validation script so w/e | 08:50 |
| shadower | I'll give it a go | 08:50 |
| mandre | shadower: either you'll use the run-validation wrapper or you call ansible-playbook with the the corresponding CLI option | 08:51 |
| shadower | mandre: yeah I think that's fair (though we should document that) | 08:51 |
| *** dsariel has joined #tripleo | 08:54 | |
| openstackgerrit | Derek Higgins proposed openstack/tripleo-common: Add port registration to IPMI power driver https://review.openstack.org/365953 | 08:57 |
| mandre | shadower: supposing we go with #2, can we imagine a small helper module that export the environment variables after becoming root? | 08:58 |
| mandre | shadower: this way the validation authors decide when to become root and it's not too painful to have the proper environment variables | 08:59 |
| *** abregman has joined #tripleo | 08:59 | |
| shadower | mandre: yea I was thinking something similar. Though you need to call it before you become root | 09:00 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Remove the ha2 JOBTYPE https://review.openstack.org/363411 | 09:00 |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: WIP -- clean full upgrade review https://review.openstack.org/364859 | 09:01 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/tripleo-common: Test baremetal: Do not mock at module level https://review.openstack.org/365958 | 09:01 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/tripleo-common: Ironic Client: Bump the max_retries and retry_interval https://review.openstack.org/365694 | 09:02 |
| *** florianf has quit IRC | 09:02 | |
| mandre | shadower: right... or we configure ansible to pass the environment variables with sudo like we discussed, and we stop defaulting to 'become: true' in the inventory script | 09:02 |
| mandre | shadower: the 'become: true' will need to be explicit in the validations tasks | 09:03 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/tripleo-common: Test baremetal: Do not mock at module level https://review.openstack.org/365958 | 09:03 |
| *** florianf has joined #tripleo | 09:04 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/tripleo-common: Ironic Client: Bump the max_retries and retry_interval https://review.openstack.org/365694 | 09:04 |
| shadower | mandre: right that's an option, too. My proposals were to keep the same functionality, but make the env vars available | 09:04 |
| shadower | mandre: so, setting sudo flags in run-validation works fine | 09:05 |
| shadower | mandre: I'm happy to create that patch if you're fine with that | 09:06 |
| mandre | shadower: let's do it in 2 steps, first set ansible to maintain environment variables with sudo, then in a separate commit get rid of the 'become:true' in the inventory script and be explicit in the validations files that require it | 09:06 |
| shadower | mandre: sounds got to me. Expect patches soon :-) | 09:07 |
| mandre | shadower: great | 09:07 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add service networks to hieradata https://review.openstack.org/365475 | 09:09 |
| *** nyechiel_ has quit IRC | 09:10 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/361501 | 09:13 |
| *** limao has quit IRC | 09:19 | |
| openstackgerrit | Tomas Sedovic proposed openstack/tripleo-common: Preserve the shell environment for run-validation https://review.openstack.org/365983 | 09:20 |
| shadower | mandre: that's the first one ^ | 09:20 |
| *** r-mibu has quit IRC | 09:23 | |
| *** jistr|dentist is now known as jistr | 09:23 | |
| shadower | mandre: should we remove become from the overcloud nodes, too? | 09:26 |
| *** nyechiel_ has joined #tripleo | 09:28 | |
| openstackgerrit | Tomas Sedovic proposed openstack/tripleo-validations: Remove `become: true` from the inventory script https://review.openstack.org/366010 | 09:30 |
| shadower | mandre: and that's the other one ^ | 09:30 |
| jtomasek | shadower: I am getting this now: "No password entered, or found via --os-password or OS_PASSWORD" | 09:30 |
| shadower | jtomasek: hm. That's exactly what this was supposed to fix | 09:31 |
| shadower | jtomasek: can you pastebin /etc/ansible/ansible.cfg? | 09:31 |
| jtomasek | shadower: do I need to restart anything after changing that config? | 09:31 |
| shadower | jtomasek: nope | 09:32 |
| shadower | at least I don't think. I didn't need to (but I'm not going through Mistral) | 09:32 |
| shadower | jtomasek: anyway, this should be fixed soon so don't worry about it now | 09:33 |
| jtomasek | ok | 09:33 |
| *** r-mibu has joined #tripleo | 09:33 | |
| *** nyechiel_ has quit IRC | 09:33 | |
| b00tcat | if my undercloud is a VM inside a baremetal server (which I installed using instack-undercloud), can I tell the undercloud to provision the overcloud controller as a VM on this baremetal server? | 09:36 |
| *** mcornea is now known as mcornea|afk | 09:41 | |
| *** shardy_afk has joined #tripleo | 09:43 | |
| *** saneax is now known as saneax-_-|AFK | 09:43 | |
| lucasagomes | derekh, something really fishy in the unittests for python3.5 and that patch to bump the timeout | 09:43 |
| lucasagomes | derekh, somewhere _get_baremetal_client() is being mocked and it's leaking on the test I just added | 09:44 |
| * lucasagomes digs more | 09:44 | |
| *** shardy has joined #tripleo | 09:44 | |
| derekh | hmm, weird | 09:46 |
| *** saneax-_-|AFK is now known as saneax | 09:52 | |
| *** akrivoka has joined #tripleo | 09:55 | |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Implement Validation Detail modal https://review.openstack.org/365921 | 09:55 |
| *** dtantsur|afk is now known as dtantsur | 09:59 | |
| *** pgadiya has quit IRC | 10:02 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/tripleo-common: Test baremetal: Correctly stop the mocks https://review.openstack.org/365958 | 10:02 |
| lucasagomes | derekh, ^ a-ha, I think I find it | 10:02 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/tripleo-common: Ironic Client: Bump the max_retries and retry_interval https://review.openstack.org/365694 | 10:03 |
| derekh | lucasagomes: nice :-) | 10:03 |
| jpich | lucasagomes: Yeah, something similar solves it for me locally as well -- sorry about this | 10:04 |
| lucasagomes | jpich, o/ I didn't look at the other modules, maybe the patch should fix it all | 10:04 |
| * lucasagomes looks | 10:04 | |
| jpich | lucasagomes: Looks like the other place that uses the patcher stops it correctly :) | 10:05 |
| lucasagomes | jpich, yay, great then! Thanks for verifying | 10:06 |
| openstackgerrit | Tomas Sedovic proposed openstack/tripleo-validations: Make the ini-based validations more robust https://review.openstack.org/366025 | 10:06 |
| jpich | lucasagomes: Thanks for finding and fixing this! | 10:07 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Move AllNodesValidationDeployments into jinja template loop https://review.openstack.org/337587 | 10:07 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Convert overcloud.yaml to support jinja2 templating https://review.openstack.org/315679 | 10:07 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Move RedisVirtualIP hiera into VipMap https://review.openstack.org/365791 | 10:07 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Convert allNodesConfig properties to composable jinja2 https://review.openstack.org/365794 | 10:07 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Move AllNodesDeployments into jinja template loop https://review.openstack.org/337267 | 10:07 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Move role ResourceGroups inside the jinja2 loop https://review.openstack.org/365793 | 10:07 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Move role deployment steps into overcloud.yaml https://review.openstack.org/365763 | 10:07 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Remove *ExtraConfig parameters from overcloud.yaml https://review.openstack.org/365792 | 10:07 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Move per-role NetIpListMap's into jinja template loop https://review.openstack.org/364749 | 10:07 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Convert SwiftDevicesAndProxyConfig to composable format https://review.openstack.org/364748 | 10:07 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Convert deploy steps to jinja2 loop https://review.openstack.org/365796 | 10:07 |
| shadower | mandre: https://review.openstack.org/366025 noticed a couple of (unrelated) failures when testing the removed become. | 10:07 |
| shardy | jistr: Hey, I rebased the custom-roles series as it conflicted with jaosorior's patch that landed | 10:08 |
| shardy | jistr: are you still working on the SwiftDevicesAndProxyConfig patches? | 10:08 |
| shardy | those are the head of the branch now, so it'd be good to get them ready to land asap | 10:08 |
| *** bvandenh has joined #tripleo | 10:09 | |
| mandre | shadower: gotcha... do you mind creating an issue in LP for the bug you found? | 10:10 |
| *** sshnaidm|afk is now known as sshnaidm | 10:10 | |
| shadower | mandre: will do | 10:10 |
| *** pgadiya has joined #tripleo | 10:10 | |
| mandre | shadower: thx, I'd like to start tracking bugs more effectively | 10:12 |
| shadower | mandre: yea agreed | 10:12 |
| openstackgerrit | Tomas Sedovic proposed openstack/tripleo-validations: Make the ini-based validations more robust https://review.openstack.org/366025 | 10:15 |
| *** pgadiya has quit IRC | 10:15 | |
| shadower | mandre: done ^ | 10:15 |
| openstackgerrit | Tomas Sedovic proposed openstack/tripleo-validations: Make the ini-based validations more robust https://review.openstack.org/366025 | 10:16 |
| *** nyechiel_ has joined #tripleo | 10:16 | |
| *** pgadiya has joined #tripleo | 10:17 | |
| *** dmanchad has joined #tripleo | 10:22 | |
| *** nyechiel_ has quit IRC | 10:22 | |
| * jistr back from lunch | 10:25 | |
| jistr | shardy: hi, yea i just tested it, overcloud deploys and swift seems to work, so i think they're ready to go | 10:26 |
| jistr | i'll un-wip the puppet one | 10:26 |
| shardy | jistr: ack, thanks - I've also been testing it locally and all seemed OK, but I've not checked things closely other than a basic deployment completes | 10:27 |
| *** mcornea|afk is now known as mcornea | 10:27 | |
| *** zoliXXL is now known as zoli|lunch | 10:28 | |
| *** ramishra has quit IRC | 10:30 | |
| *** paramite has joined #tripleo | 10:31 | |
| jistr | shardy: i tried also uploading a swift object and looked at the .builder files briefly, they seem to be sane on both controller and swift node | 10:31 |
| jistr | so i hope we're ok there | 10:31 |
| shardy | jistr: Ok, sounds good - lets see if the rebased t-h-t patch passes CI then hopefully we can land them both | 10:32 |
| *** ramishra has joined #tripleo | 10:32 | |
| *** pgadiya has quit IRC | 10:35 | |
| *** athomas has quit IRC | 10:37 | |
| *** akrivoka has quit IRC | 10:38 | |
| *** athomas has joined #tripleo | 10:41 | |
| *** abregman is now known as abregman|brb | 10:41 | |
| *** pblaho has joined #tripleo | 10:43 | |
| jtomasek | mandre, shadower: found a bug here https://github.com/openstack/tripleo-common/blob/master/workbooks/validations.yaml#L122 | 10:43 |
| jtomasek | the action name is wrong | 10:43 |
| *** tobias_fiberdata has joined #tripleo | 10:45 | |
| *** HenryG_ is now known as HenryG | 10:46 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Add bootstrap_node and vip_data to hierarchy for all roles https://review.openstack.org/366049 | 10:47 |
| *** tobias-fiberdata has quit IRC | 10:48 | |
| shadower | jtomasek: ouch, I'll send a patch | 10:50 |
| jtomasek | shadower: we'll need to track it for rc1 and get it merged asap, it blocks running validation groups automatically from GUI | 10:51 |
| *** pgadiya has joined #tripleo | 10:51 | |
| openstackgerrit | Julie Pichon proposed openstack/python-tripleoclient: Remove excessive output when configuring nodes https://review.openstack.org/366051 | 10:51 |
| shadower | jtomasek: tell you what: you submit the bug and I'll do the fix in the meantime? | 10:52 |
| shadower | -) | 10:52 |
| shardy | jaosorior: ^^ FYI mcornea found an issue ref bug 1620485 which I think will mean you also can't use your vip_data from allNodesConfig on non-Controller roles | 10:52 |
| openstack | bug 1620485 in tripleo "Deployment fails when MongoDB runs on a different node than controller with missing bootstrap_nodeid hiera" [High,In progress] https://launchpad.net/bugs/1620485 - Assigned to Steven Hardy (shardy) | 10:52 |
| shardy | jaosorior: that patch should fix it I think | 10:52 |
| jtomasek | shadower: ack:) | 10:52 |
| *** abregman|brb is now known as abregman | 10:52 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Customize undercloud and overcloud with virt-customize https://review.openstack.org/361112 | 10:53 |
| jaosorior | shardy: ok | 10:55 |
| jtomasek | shadower, shardy https://bugs.launchpad.net/tripleo/+bug/1620573 | 10:58 |
| openstack | Launchpad bug 1620573 in tripleo "Incorrect action name in run_groups validation workflow" [High,New] - Assigned to Tomas Sedovic (tsedovic) | 10:58 |
| *** dprince has joined #tripleo | 11:00 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts https://review.openstack.org/357765 | 11:01 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add VIP names to allNodesConfig https://review.openstack.org/365895 | 11:01 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add parameters for internal TLS https://review.openstack.org/365942 | 11:01 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add HAProxy TLS handled by certmonger as composable service https://review.openstack.org/356430 | 11:01 |
| openstackgerrit | Tomas Sedovic proposed openstack/tripleo-common: Fix the validation action name in the workbook https://review.openstack.org/366055 | 11:03 |
| shadower | jtomasek: ^ | 11:03 |
| shadower | jtomasek: I've checked the other action names and they should be all fine. I *have* checked this before but apparently something slipped through :-( | 11:04 |
| jaosorior | shardy: the fix makes sense. I messed up cause I was seeing in the nodes that vip_data was indeed a file in /etc/puppet. But had forgotten to add it to the hierarchy list :/ | 11:05 |
| jtomasek | shadower: yeah, it got through a review. it is quite hard to catch this | 11:05 |
| jaosorior | shardy: is there a way to pass inputs to a softwareconfig through a way that doesn't involve doing so through a softwaredeployment? I'm looking to passing parameters (with get_param for instance) to a softwareconfig but defer the softwaredeployment to later in the process | 11:07 |
| jaosorior | shardy: for instance, the TLS stuff that's there at the moment. Would be cool to be able to move that to the haproxy profile... but currently we can't do that cause the software deployment requires the reference to the server. | 11:07 |
| jaosorior | and we don't have such a reference in the service profiles. | 11:08 |
| jpich | jtomasek, shadower: Wish we could unit tests the workflows' yaml, would have spared me some confusion too :) | 11:10 |
| shadower | yeah | 11:11 |
| *** shardy_afk has quit IRC | 11:11 | |
| shardy | jaosorior: Yes, you can just pass the values directly into the SoftwareConfig resource, e.g look at the old VipConfig resource you recently removed :) | 11:12 |
| shardy | that worked like that, as does allNodesConfig | 11:12 |
| *** rhallisey has joined #tripleo | 11:13 | |
| jaosorior | shardy: but that's a structuredconfig | 11:13 |
| shardy | jaosorior: StructuredConfig works the same as SoftwareConfig, the only difference is some variable subsitution and the output format | 11:14 |
| jaosorior | shardy: right, so in this case I need to do the get_param calls inside the script definition? | 11:14 |
| shardy | http://docs.openstack.org/developer/heat/template_guide/openstack.html#OS::Heat::SoftwareConfig-prop-inputs | 11:15 |
| shardy | jaosorior: No, you can just define an input, then set the default of the input to get_param: foo | 11:15 |
| shardy | I suppose that works a little different to how we currently use StructuredConfig, you're right | 11:16 |
| jaosorior | shardy: will the default values appear when inspecting the stack? | 11:16 |
| *** tremble has quit IRC | 11:16 | |
| shardy | jaosorior: define inspecting - you can retrieve the inputs schema for a SoftwareConfig, but I'm not sure why you'd need to? | 11:16 |
| jaosorior | crap | 11:17 |
| jaosorior | the ovb jobs seem broken | 11:17 |
| jaosorior | derekh: hey dude, any idea what this is? http://logs.openstack.org/83/365583/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/32dbf65/console.html#_2016-09-06_11_08_49_398967 | 11:17 |
| shadower | \o/ | 11:18 |
| jaosorior | shardy: not the input schema, but the actual inputs given. Or the defaults in this case. | 11:18 |
| derekh | jaosorior: looks like an error while creating testenvs, is it happening now on all jobs? | 11:19 |
| jaosorior | derekh: I've seen it in 4 jobs now | 11:19 |
| *** tremble has joined #tripleo | 11:19 | |
| shardy | jaosorior: Not sure, it's possibly accessible via the show attribute: http://docs.openstack.org/developer/heat/template_guide/openstack.html#OS::Heat::SoftwareConfig-attr-show | 11:19 |
| derekh | jaosorior: taking a look | 11:19 |
| shardy | again, I don't understand why, as you just need to reference the ID and the config encapsulates the input schema, which includes the defaults? | 11:20 |
| derekh | sshnaidm: ^^ fyi, taking a look on the te-broker to find the error | 11:20 |
| jaosorior | shardy: and there is no way of making inputs hidden? | 11:20 |
| openstackgerrit | Keith Schincke proposed openstack/tripleo-heat-templates: Add support for deploying Ceph RGW role https://review.openstack.org/289027 | 11:20 |
| jaosorior | shardy: if it's key material, it would be best to not show that when someone tries to inspect it. | 11:21 |
| shardy | jaosorior: the parameters can be marked hidden, but we can't hide the value from all API output, or the SoftwarDeployment can't read it | 11:21 |
| jaosorior | shardy: in this case parameters are marked as hidden, so that part is covered. | 11:22 |
| jaosorior | ok | 11:22 |
| shardy | jaosorior: Yeah, so that will hide them from casual inspection e.g via horizon or whatever | 11:22 |
| *** lucasagomes is now known as lucas-hungry | 11:22 | |
| shardy | but if you've got credentials to access the heat API, you'll be able to get the values or the SoftwareDeployment couldn't consume them | 11:22 |
| jaosorior | shardy: alright, so I guess there's no harm in moving the get_param to the default value of those inputs. As we can already view them when checking the softwaredeployment anyway | 11:24 |
| shardy | jaosorior: example https://paste.fedoraproject.org/422773/73161112/ | 11:25 |
| jaosorior | shardy: alright | 11:25 |
| shardy | you can see the inputs schema there, and it contains the default | 11:25 |
| jaosorior | shardy: right now I guess that's also the case for the keys that we inject with the current solution, right? | 11:26 |
| *** bkopilov has quit IRC | 11:27 | |
| shardy | jaosorior: yup | 11:27 |
| *** zephcom has quit IRC | 11:28 | |
| openstackgerrit | Julie Pichon proposed openstack/tripleo-common: Clean-up messages in the 'configure' workflow https://review.openstack.org/366061 | 11:28 |
| *** tremble has quit IRC | 11:28 | |
| *** thrash|g0ne is now known as thrash | 11:28 | |
| *** zephcom has joined #tripleo | 11:29 | |
| openstackgerrit | Julie Pichon proposed openstack/tripleo-common: Clean up messages in the 'configure' workflow https://review.openstack.org/366061 | 11:29 |
| derekh | jaosorior: hmmm, looks like heat is loosing connection to the db | 11:30 |
| derekh | heat/heat-engine.log:2016-09-06 11:18:01.232 46222 INFO heat.engine.stack [-] Stack DELETE FAILED (baremetal_8175): Resource DELETE failed: DBConnectionError: resources.baremetal_env.resources.openstack_baremetal_servers.resources[1]: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') | 11:30 |
| skramaja | shardy: we have added an environment file for SR-IOV, can you please take a look? https://review.openstack.org/#/c/363617/ has a +2 | 11:30 |
| derekh | jaosorior: lots of them ^ | 11:30 |
| *** tremble has joined #tripleo | 11:30 | |
| *** tremble has quit IRC | 11:30 | |
| *** tremble has joined #tripleo | 11:30 | |
| jaosorior | what the hell | 11:30 |
| jaosorior | sshnaidm: you had mentioned there were some issues you were seeing in the periodic jobs involving the heat database. Is this realted? | 11:31 |
| jaosorior | *related | 11:31 |
| jaosorior | derekh: wait... is this related to the te-broker issue? | 11:32 |
| sshnaidm | jaosorior, I don't think so, in this case connection is lost, in mine it was incorrect table value | 11:32 |
| derekh | jaosorior: this is a problem on the rh1 cloud itself, | 11:32 |
| jaosorior | derekh: oh | 11:32 |
| jaosorior | O_O | 11:32 |
| jaosorior | I don't have access to that :/ | 11:33 |
| jaosorior | derekh: is there a way I can help figure out what's up there? Like a place to check logs or something of the sort | 11:33 |
| sshnaidm | derekh, which log file is it? | 11:33 |
| derekh | jaosorior: nope, the logs are only available to the cloud admins | 11:33 |
| jaosorior | ok | 11:34 |
| derekh | sshnaidm: heat/heat-engine.log | 11:34 |
| sshnaidm | derekh, on overcloud controller? | 11:34 |
| shardy | skramaja: lgtm, approved - thanks! | 11:34 |
| derekh | sshnaidm: yup | 11:35 |
| shardy | skramaja: is that the last patch to complete the sr-iov blueprint? | 11:35 |
| skramaja | thanks shardy.. one more pending.. | 11:35 |
| skramaja | CI is failing.. i will let u know once done.. | 11:35 |
| jaosorior | jistr, shardy: got time for a quick review? https://review.openstack.org/#/c/365583 I'm trying to get those FQDN facts to be more usable through t-h-t. So that gets them to match the network names. It passed CI (nonha and ha) in two different runs, but right now the ovb jobs are broken :/ | 11:35 |
| shardy | skramaja: Ok, thanks | 11:35 |
| shadower | jaosorior: can haz a quick review? 2 line patch https://review.openstack.org/#/c/366010/ | 11:36 |
| jaosorior | shadower: sure | 11:36 |
| shadower | cheers | 11:36 |
| derekh | sshnaidm: jaosorior I'm guessing some sql queries have gotten too slow due to table's growing | 11:36 |
| derekh | http://paste.openstack.org/show/567125/ | 11:37 |
| derekh | shardy: FYI ^^ problems creating envs on RH1 | 11:37 |
| jaosorior | derekh: Are those leftovers in the database? There is a cron-job available to clean those up, no? | 11:37 |
| sshnaidm | derekh, I see this also: http://paste.openstack.org/show/567126/ | 11:38 |
| openstackgerrit | Merged openstack/tripleo-validations: Remove `become: true` from the inventory script https://review.openstack.org/366010 | 11:40 |
| derekh | sshnaidm: yup, may also be related | 11:40 |
| derekh | jaosorior: doesn't look like we have any clean up cron jobs | 11:40 |
| flepied | periodic job succeeded to promote last night or was it promoted by hand? | 11:40 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Run Validations automatically https://review.openstack.org/366068 | 11:41 |
| jaosorior | derekh: bummer, but there is this available: heat-manage purge_deleted -g days 30 | 11:42 |
| jaosorior | derekh: to clean up the deleted stacks | 11:42 |
| *** fragatina has joined #tripleo | 11:42 | |
| openstackgerrit | Tomas Sedovic proposed openstack/tripleo-validations: Check for differences between introspected HW https://review.openstack.org/353978 | 11:42 |
| *** xuao has joined #tripleo | 11:42 | |
| *** dprince has quit IRC | 11:43 | |
| *** fragatina has quit IRC | 11:43 | |
| derekh | jaosorior: thanks, I havn't ever run it, but I think we should give it a whirl | 11:43 |
| *** dprince has joined #tripleo | 11:43 | |
| derekh | sshnaidm: bnemec "heat-manage purge_deleted -g days 30" <-- on the RH1 overcloud controller, what do ye think? | 11:44 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Retry stack-delete in case of failure https://review.openstack.org/350640 | 11:44 |
| sshnaidm | flepied, I think manually by weshay | 11:44 |
| flepied | ok thx sshnaidm | 11:45 |
| shardy | derekh: do we even need 30 days of deleted stuff in the DB? | 11:45 |
| shardy | I'm just wondering if anyone ever looks at it | 11:45 |
| sshnaidm | derekh, isn't it cleaned after heat stack is deleted? | 11:45 |
| shadower | jtomasek: so if you pull tripleo-validations and checkout this: https://review.openstack.org/#/c/353978/ it should be working | 11:46 |
| shardy | sshnaidm: No, heat soft-deletes stack like Nova does | 11:46 |
| *** pkovar has joined #tripleo | 11:46 | |
| shadower | jtomasek: (that's the patch from the morning I wanted to see the details of) | 11:46 |
| *** tobias-fiberdata has joined #tripleo | 11:46 | |
| derekh | shardy: I've never looked at them, we could probably just keep a day or 2 just incase anybody ever want to debug something | 11:46 |
| shardy | some data is deleted when the stack is deleted, not not everything so there's an audit trail e.g the stack owner and events | 11:46 |
| derekh | sshnaidm: the stacks are deleted but in the DB they still exist and marked as deleted | 11:47 |
| shardy | derekh: Yeah, I was thinking something like 7 days would be ample, but I guess any purge is better than none :) | 11:47 |
| sshnaidm | shardy, derekh so there will be a problem with any long-running openstack? | 11:48 |
| derekh | shardy: ok, before I run it, I want to quickly see if I can turn on slow query logging to see how bad things have gotten | 11:48 |
| *** jpena is now known as jpena|lunch | 11:48 | |
| jtomasek | shadower: hm, I see still the same error. I did git review -d 353978 and sudo pip install . | 11:49 |
| derekh | sshnaidm: possibly, (if this is the actual problem) | 11:49 |
| shardy | sshnaidm: Yes, but normally with TripleO deployed OpenStack we setup cron jobs which periodically purge the Heat (and other) DB's | 11:49 |
| *** tobias_fiberdata has quit IRC | 11:49 | |
| jtomasek | shadower: can you try to run that workflow? | 11:50 |
| sshnaidm | derekh, could it be problem with all these non-deleted stacks (DELETE_FAILED) that take resources and don't allow to create a new ones? | 11:50 |
| *** bfournie has quit IRC | 11:50 | |
| derekh | anybody know if you can turn on log_slow_queries on a running mysql server? | 11:50 |
| *** masco has quit IRC | 11:50 | |
| derekh | sshnaidm: yup, we should clean those up also | 11:50 |
| EmilienM | hello | 11:51 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Hook internal TLS flag to apache-based services https://review.openstack.org/366075 | 11:51 |
| shadower | jtomasek: I didn't have time to set up the mistral env properly yet :-( (so I tested it directly with ansible) | 11:51 |
| shadower | jtomasek: let me have a look | 11:51 |
| derekh | got it "SET GLOBAL slow_query_log = 'ON'" | 11:51 |
| openstackgerrit | Brad P. Crochet proposed openstack-infra/tripleo-ci: Add Zaqar to scenario002 https://review.openstack.org/365026 | 11:52 |
| shadower | jtomasek: can you grep for "become" in tripleo-common/scripts/tripleo-ansible-inventory? | 11:52 |
| shadower | jtomasek: it should come out empty | 11:52 |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-heat-templates: Composable Zaqar services https://review.openstack.org/331682 | 11:52 |
| derekh | sshnaidm: just gonna leave that for a couple of minutes, then we can purge the stale entries and delete the DELETE_FAILED stacks | 11:52 |
| dtantsur | hmm, is it normal for HA on virt to take 1.5 hours to deploy? I think it used to be faster | 11:52 |
| *** abregman has quit IRC | 11:53 | |
| sshnaidm | derekh, ok | 11:53 |
| *** abregman has joined #tripleo | 11:54 | |
| thrash | EmilienM: would you mind checking that I have the depends-on in a sane state for the zaqar patches? We have scenario002 being triggered correctly via zuul. | 11:54 |
| thrash | EmilienM: https://review.openstack.org/#/c/331681/ | 11:54 |
| EmilienM | thrash: you need the tripleo-ci patch in depends-on | 11:54 |
| jaosorior | thrash: are you looking into setting up zaqar for the overcloud? | 11:54 |
| EmilienM | thrash: otherwise you won't have the right pingtest and right template to deploy. | 11:55 |
| thrash | EmilienM: https://review.openstack.org/#/c/365026/ and https://review.openstack.org/#/c/331682/ I switched up the dependency so I think I have it correct now | 11:55 |
| thrash | jaosorior: yes. | 11:55 |
| EmilienM | thrash: yes it's good now! | 11:55 |
| jtomasek | shadower: hmm, I don't have that script | 11:55 |
| thrash | EmilienM: swwweeeeetttt | 11:55 |
| jaosorior | thrash: I was looking into getting zaqar's websocket endpoint to be terminated by haproxy | 11:55 |
| shadower | jtomasek: I'm sorry -- tripleo-validations/scripts/... | 11:55 |
| jaosorior | thrash: but I never got it to work with TLS | 11:55 |
| jtomasek | ah, ok | 11:55 |
| shadower | jtomasek: which come to think of it needs setup.py as well to get it working properly | 11:56 |
| jaosorior | thrash: I guess you'll need that to get it working | 11:56 |
| jaosorior | thrash: else you won't get websockets in the overcloud | 11:56 |
| jtomasek | shadower: 'become' is not in that script | 11:56 |
| thrash | jaosorior: https://review.openstack.org/#/c/331682/ is the zaqar composable. | 11:56 |
| shardy | Reviews of https://review.openstack.org/#/c/365783/ would be good | 11:56 |
| jaosorior | or you might but when someone enables TLS, it'll break | 11:56 |
| thrash | jaosorior: ack | 11:57 |
| shardy | without it upload-puppet-modules breaks now we switched to mistral/swift for plan storage | 11:57 |
| shadower | jtomasek: right, so try running "sudo python setup.py install" for tripleo-validations and that should work | 11:57 |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Handle missing mistral environment https://review.openstack.org/365907 | 11:57 |
| jaosorior | thrash: so this was my attempt at adding it to haproxy https://review.openstack.org/#/c/360329/ | 11:57 |
| jaosorior | thrash: but yeah, I think that needs to work before getting zaqar into the overcloud | 11:59 |
| thrash | jaosorior: +1 | 12:00 |
| jaosorior | thrash: TLS for the public endpoints is a pretty common use-case | 12:00 |
| thrash | jaosorior: it won't be enabled by default to start. | 12:00 |
| jaosorior | thrash: so, if you have any ideas on how to get the websockets over TLS with HAProxy in the middle working... would be cool. I didn't figure it out :( | 12:00 |
| thrash | jaosorior: I'll take a look | 12:01 |
| *** jeckersb_gone is now known as jeckersb | 12:01 | |
| jaosorior | EmilienM: could you take a look at this? https://review.openstack.org/#/c/365583/ | 12:01 |
| EmilienM | jaosorior: sure | 12:02 |
| jtomasek | shadower: I tried a buch of tricks but still the same result http://paste.openstack.org/show/567131/ | 12:02 |
| shadower | aaaaa | 12:02 |
| *** tobias_fiberdata has joined #tripleo | 12:02 | |
| *** zoli|lunch is now known as zoliXXL | 12:03 | |
| *** jeckersb is now known as jeckersb_gone | 12:03 | |
| EmilienM | jaosorior: it looks like ovb jobs aren't green | 12:04 |
| EmilienM | and never be | 12:04 |
| jaosorior | EmilienM: ovb is broken in general | 12:04 |
| EmilienM | that is not true | 12:04 |
| jaosorior | derekh: ^^ | 12:04 |
| shadower | jtomasek: does "grep become /usr/bin/tripleo-ansible-inventory" come out empty? | 12:04 |
| shardy | Yeah, we've been suffering from some specific problems | 12:04 |
| EmilienM | I saw successful jobs this week end | 12:04 |
| shardy | until recently OVB was working much better than our previous CI setup | 12:04 |
| jaosorior | EmilienM: broke today | 12:04 |
| EmilienM | it was actually fast | 12:04 |
| EmilienM | ok but your patch was not passing OVB jobs before either | 12:05 |
| jtomasek | shadower: yes | 12:05 |
| jaosorior | EmilienM: wasn't it? | 12:05 |
| *** tobias-fiberdata has quit IRC | 12:05 | |
| EmilienM | anyway I don't think we should merge this one without OVB vote | 12:05 |
| EmilienM | jaosorior: the ha job wasn't | 12:05 |
| shadower | jtomasek: I'm so confused. Any chance I could poke around? | 12:05 |
| EmilienM | shardy: what broke? | 12:05 |
| jtomasek | shadower: sure PM | 12:06 |
| jaosorior | EmilienM: first run failed HA and passed nonha, second run failed nonha and passed ha | 12:06 |
| *** trown|outtypewww is now known as trown | 12:06 | |
| jaosorior | EmilienM: it's a problem with rh1 cloud itself. | 12:06 |
| shardy | EmilienM: there is an error for heat connecting to the DB which broke some recent jobs, derekh is investigating | 12:07 |
| jaosorior | EmilienM: this job depends on the puppet change I passed you and is green on ovb https://review.openstack.org/#/c/365585/ | 12:07 |
| shardy | and before that we were experiencing performance problems which have been improved by reconfiguring the hardware, and switching back to cached images | 12:07 |
| *** jayg|g0n3 is now known as jayg | 12:08 | |
| *** saneax is now known as saneax-_-|AFK | 12:08 | |
| *** pgadiya has quit IRC | 12:10 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Convert SwiftDevicesAndProxyConfig to composable format https://review.openstack.org/364748 | 12:10 |
| derekh | sshnaidm: no slow queries got logged, I'm just ganna run the purge command, or do you want to do it ? | 12:11 |
| sshnaidm | derekh, I will | 12:12 |
| *** maeca1 has joined #tripleo | 12:12 | |
| derekh | sshnaidm: ok | 12:12 |
| *** akrivoka has joined #tripleo | 12:12 | |
| EmilienM | jaosorior: ok, then good | 12:13 |
| *** fragatina has joined #tripleo | 12:13 | |
| sshnaidm | derekh, done | 12:14 |
| *** saneax-_-|AFK is now known as saneax | 12:15 | |
| jaosorior | EmilienM: hey dude, do you have a write up (or something) on how the scenario tests work in tripleo? | 12:16 |
| EmilienM | jaosorior: it's on my list | 12:16 |
| EmilienM | jaosorior: for now I have written this https://github.com/openstack-infra/tripleo-ci#service-testing-matrix | 12:16 |
| EmilienM | jaosorior: you can see this example https://review.openstack.org/#/c/365026/ | 12:17 |
| EmilienM | jaosorior: and https://review.openstack.org/#/c/365023/ | 12:17 |
| jaosorior | I see | 12:18 |
| jaosorior | EmilienM: are the scenarios deployed in ovb or is it in the multinode setup? | 12:19 |
| *** akrivoka has quit IRC | 12:19 | |
| *** akrivoka has joined #tripleo | 12:20 | |
| EmilienM | jaosorior: multinode | 12:21 |
| *** liverpooler has quit IRC | 12:21 | |
| *** tobias-fiberdata has joined #tripleo | 12:21 | |
| jaosorior | nice! | 12:21 |
| jaosorior | EmilienM: and is there a way to run those manually? | 12:22 |
| *** fragatina has quit IRC | 12:22 | |
| *** xuao has quit IRC | 12:24 | |
| *** tobias_fiberdata has quit IRC | 12:25 | |
| *** tobias_fiberdata has joined #tripleo | 12:25 | |
| *** tobias-fiberdata has quit IRC | 12:26 | |
| *** lucas-hungry is now known as lucasagomes | 12:27 | |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Run Validations automatically https://review.openstack.org/366068 | 12:27 |
| *** pgadiya has joined #tripleo | 12:28 | |
| *** rlandy has joined #tripleo | 12:30 | |
| *** zoliXXL is now known as zoli|brb | 12:32 | |
| *** fultonj has joined #tripleo | 12:32 | |
| derekh | sshnaidm: the tables don't seem to have gotten any smaller, did the command take long to run ? | 12:34 |
| sshnaidm | derekh, nope | 12:35 |
| sshnaidm | derekh, it was quick | 12:35 |
| jaosorior | derekh: how many days did you give? | 12:36 |
| sshnaidm | heat-manage purge_deleted -g days 30 | 12:36 |
| derekh | sshnaidm: try 7 days maybe to see what happens | 12:36 |
| jaosorior | derekh, sshnaidm: Might want to try it with 7 days? | 12:36 |
| derekh | snap | 12:36 |
| jaosorior | derekh: ?? | 12:37 |
| *** myoung|afk is now known as myoung | 12:37 | |
| *** bfournie has joined #tripleo | 12:38 | |
| jaosorior | derekh, sshnaidm if that doesn't work either, might be worth specifying the path to the config file explicitly | 12:38 |
| derekh | jaosorior: snap : we said the same thing at the same time, maybe its an irish thing | 12:38 |
| derekh | brb | 12:38 |
| sshnaidm | derekh, done, try to check now pls | 12:38 |
| sshnaidm | jaosorior, it tries to use: /usr/share/heat/heat-dist.conf,/etc/heat/heat.conf | 12:39 |
| jaosorior | shardy: ok, so then lets hope that 7 days does the trick | 12:40 |
| EmilienM | if anyone has off-items for our tripleo meeting this week https://etherpad.openstack.org/p/tripleo-meeting-items | 12:40 |
| *** dmsimard|pto is now known as dmsimard | 12:43 | |
| EmilienM | shardy: I saw your link, isn't it for the release item? | 12:46 |
| shardy | EmilienM: Yeah we can cover it during the release topic, I just wanted to highlight it | 12:47 |
| EmilienM | ok | 12:47 |
| shardy | so folks can start thinking about the status and check out the RC1 link | 12:47 |
| *** jpena|lunch is now known as jpena | 12:48 | |
| *** jaosorior has quit IRC | 12:48 | |
| *** noslzzp has joined #tripleo | 12:49 | |
| *** jaosorior has joined #tripleo | 12:49 | |
| jaosorior | /exit | 12:49 |
| *** jaosorior has quit IRC | 12:49 | |
| derekh | sshnaidm: nope, those 4 tables are still the same size | 12:49 |
| derekh | sshnaidm: who are you running it as? /me wonders if it needs to be root | 12:50 |
| *** pcaruana has quit IRC | 12:50 | |
| sshnaidm | derekh, yep, sudo | 12:50 |
| *** jaosorior has joined #tripleo | 12:51 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Customize undercloud and overcloud with virt-customize https://review.openstack.org/361112 | 12:51 |
| derekh | sshnaidm: how about we leave out the days thing alltogether, we shouldn't need them anyways | 12:51 |
| * derekh is just thinking out loud here | 12:51 | |
| sshnaidm | derekh, sorry, don't follow | 12:52 |
| sshnaidm | derekh, you mean to purge all? | 12:52 |
| derekh | sshnaidm: just run "heat-manage purge_deleted" and see what happens, it should only purge things that have already been deleted anyways | 12:53 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Save console logs from all vms to files https://review.openstack.org/365369 | 12:54 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Fix aodh auth url to remove suffix https://review.openstack.org/365117 | 12:54 |
| sshnaidm | derekh, done, but it was fast.. | 12:55 |
| sshnaidm | derekh, I'm not sure it does something.. | 12:55 |
| EmilienM | pradk: /me still investigating why gnocchi/aodh fails in scenario001 | 12:55 |
| EmilienM | shardy: when you have time I would like to revisit https://review.openstack.org/#/c/364355/ | 12:55 |
| EmilienM | shardy: put your feedback | 12:56 |
| jaosorior | EmilienM: I've actually seen that same failure in another commit | 12:56 |
| jaosorior | from scenario001 | 12:56 |
| jaosorior | didn't figure it out. Seems to me like the stack creation is left IN_PROGRESS and it doesn't actually say what failed :/ | 12:56 |
| *** tzumainn has joined #tripleo | 12:56 | |
| derekh | sshnaidm: ya, looks like it did nothing, | 12:56 |
| shardy | EmilienM: thanks - that's probably a good topic for the meeting actually, as I'd like wider feedback on it | 12:57 |
| shardy | I'm +1 on it tho, we just had discussed os-*-config not being tightly coupled to the other tripleo deliverables | 12:57 |
| shardy | and those (other than os-cloud-config) are consumed outside of tripleo | 12:58 |
| shardy | so perhaps the intermediary model makes sense for those? | 12:58 |
| derekh | shardy: any ideas? purge_deleted doesn't seem to be deleting anything for the large tables... | 12:58 |
| derekh | also the table size thing mightn't be the actual problem just a hunch | 12:58 |
| dtantsur | shardy, EmilienM, I kind of thought os-net-config can be consumed outside of tripleo too.. | 12:59 |
| shardy | dtantsur: Yeah, it can, although I'm not sure if anyone does | 12:59 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Customize undercloud and overcloud with virt-customize https://review.openstack.org/361112 | 13:00 |
| jaosorior | dtantsur: do you know if ironic is running over apache in the undercloud? | 13:02 |
| dtantsur | jaosorior, no, we had problems with it and had to revert | 13:02 |
| dtantsur | this is a goal for Ocata, I guess | 13:02 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates: Add steps to containerized compute deployment https://review.openstack.org/346927 | 13:03 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates: WIP: Containerized Services for Composable Roles https://review.openstack.org/330659 | 13:03 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates: Bind mount files to run DiD in latest atomic host https://review.openstack.org/347218 | 13:03 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates: Be more inclusive in insecure registry regex https://review.openstack.org/366111 | 13:03 |
| *** bkopilov has joined #tripleo | 13:03 | |
| *** trozet has quit IRC | 13:04 | |
| *** zoli|brb is now known as zoliXXL | 13:05 | |
| *** cdearborn has joined #tripleo | 13:05 | |
| *** Goneri has joined #tripleo | 13:07 | |
| derekh | sshnaidm: or maybe there is a filehandle leak somewhere, this seems a tad high | 13:07 |
| derekh | [root@overcloud-controller-0 ~]# wc lsof | 13:07 |
| derekh | 1825664 19669833 316782766 lsof | 13:07 |
| shadower | mandre: so this doesn't pass OS_PASSWORD in which breaks the discovery diff stuff when running from mistral: https://github.com/openstack/tripleo-common/blob/6c39473ce5f647f717c32662e5dc0edd178295f7/tripleo_common/utils/validations.py#L82 | 13:09 |
| *** lblanchard has joined #tripleo | 13:09 | |
| shadower | mandre: any ideas how to fix that? The validations should be able to get the auth information to the undercloud somehow | 13:09 |
| dtantsur | shadower, I suspect you can use OS_AUTH_TOKEN to create a session | 13:10 |
| mandre | shadower: you should be able to pass the token the openstack command you use | 13:10 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-quickstart: Make undercloud service workers configurable https://review.openstack.org/366116 | 13:10 |
| shadower | dtantsur, mandre: I tried that but it didn't seem to work. Maybe I just used it wrong | 13:10 |
| dtantsur | shadower, please paste your code | 13:10 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Customize undercloud and overcloud with virt-customize https://review.openstack.org/361112 | 13:11 |
| openstackgerrit | Waldemar Znoinski proposed openstack/diskimage-builder: fix systemd resource deadlock https://review.openstack.org/365797 | 13:11 |
| dtantsur | shadower, I think you need this auth plugin: http://docs.openstack.org/developer/keystoneauth/api/keystoneauth1.identity.html#keystoneauth1.identity.v2.Token | 13:11 |
| dtantsur | shadower, or rather http://docs.openstack.org/developer/keystoneauth/api/keystoneauth1.identity.generic.html#keystoneauth1.identity.generic.Token | 13:12 |
| dtantsur | to support both V2 and V3 | 13:12 |
| jaosorior | yay for generic plugins :D | 13:12 |
| *** pgadiya has quit IRC | 13:13 | |
| sshnaidm | derekh, I don't something about this in journal.. | 13:13 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-quickstart: DO NOT MERGE: Testing workers https://review.openstack.org/366119 | 13:13 |
| sshnaidm | derekh, but found another issue http://paste.openstack.org/show/567154/ | 13:13 |
| shadower | dtantsur: so this is using the commandline client. And afaics python-openstackclient doesn't understand the OS_AUTH_TOKEN env variable | 13:14 |
| dtantsur | shadower, I'm pretty sure I made it work with a token once, lemme try | 13:14 |
| shadower | dtantsur: thanks. I'll have a look at the plugin stuff in the meantime | 13:14 |
| sshnaidm | derekh, I'm concerned why we can delete any stack there, it seems like vm is still up when heat tries to delete the subnet (with ports) | 13:15 |
| sshnaidm | s/can/can't/ | 13:15 |
| mandre | shadower: openstack command accepts a --os-token CLI option | 13:17 |
| derekh | sshnaidm: ya, there could be a couple of errors, its as if the delete_ports commands arn't working here http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/te-broker/destroy-env and then the stack can't be deleted | 13:17 |
| *** akshai has joined #tripleo | 13:18 | |
| derekh | sshnaidm: ok, heres what I think we can try, restart haproxy, mysql and heat-engine | 13:18 |
| derekh | sshnaidm: then delete all the old resources that had failed to delete or were left behine | 13:18 |
| derekh | sshnaidm: and see what kind of errors we are left with | 13:18 |
| sshnaidm | derekh, ok | 13:18 |
| mandre | shadower: dunno if that helps | 13:18 |
| *** trozet has joined #tripleo | 13:19 | |
| derekh | sshnaidm: ok, you wanna do it or will I? | 13:19 |
| *** jpena is now known as jpena|off | 13:19 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Added environment file for SR-IOV deployment https://review.openstack.org/363617 | 13:19 |
| sshnaidm | derekh, I'll handle this | 13:19 |
| shadower | mandre: yeah but it seems to be a bit different. It expects the service URL directly (instead of the generic keystone one) and when I did that, I was getting 404 from it | 13:20 |
| openstackgerrit | Merged openstack/puppet-tripleo: Make fqdn fact names match service net map https://review.openstack.org/365583 | 13:20 |
| mandre | shadower: i believe you need to use it together with --os-auth-type | 13:20 |
| shadower | mandre: trying that now but unsure which value to pass in :-) | 13:21 |
| derekh | sshnaidm: ok, there might be a lot of old resources to delete so if you want to split it up give me a shout | 13:21 |
| bandini | matbu: I am seeing this during M->N upgrades (after the undercloud upgrade, during the makor-upgrade-pacemaker-init step): https://paste.fedoraproject.org/422872/73168050/ . Ever seen the same? | 13:22 |
| dtantsur | shadower, mandre, token auth seems broken in OSC 3.2.0... bothering #openstack-sdks now | 13:22 |
| shadower | lovely | 13:23 |
| matbu | bandini: /me looks | 13:23 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: Use Redis VIP when deploying with keepalived https://review.openstack.org/364917 | 13:23 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: Set Redis VIP on all nodes https://review.openstack.org/366128 | 13:23 |
| dtantsur | shadower, this is what should work according to their docs: openstack --os-token $token --os-auth-url https://192.0.2.2:13000/v2.0 server list | 13:23 |
| matbu | bandini: hm weird, i already seen that last week i think, on a CI job, but i didn't reproduce it | 13:23 |
| bandini | matbu: It is the second time I hit it. Will try to debug more and open a bug then | 13:24 |
| matbu | bandini: which newton delorean hash are you using ? | 13:24 |
| *** tremble has quit IRC | 13:24 | |
| shadower | dtantsur: thanks. How do you get the token btw? Sorry if that looks like a silly question but it gets passed through mistral for me | 13:25 |
| shadower | would love to test it manually though | 13:25 |
| mandre | shadower: 'openstack token issue' should do the trick | 13:26 |
| shadower | thanks | 13:26 |
| dtantsur | yep, more specifically $ token=$(openstack token issue -f value -c id) | 13:26 |
| bandini | matbu: https://paste.fedoraproject.org/422875/47316842/ | 13:27 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Move role deployment steps into overcloud.yaml https://review.openstack.org/365763 | 13:27 |
| shardy | dprince: Hey, I'd appreciate your feedback on ^^ when you get a moment | 13:27 |
| dprince | shardy: looking | 13:28 |
| mandre | thanks dtantsur, I really like how osc can easily integrate in shell scripts | 13:28 |
| shardy | dprince: note that the steps will be moved inside the jinja2 loop (later in that series, still testing it) | 13:28 |
| *** tremble has joined #tripleo | 13:28 | |
| dtantsur | mandre, ++ very cool | 13:28 |
| shardy | dprince: also I'm considering optimizing such that role resources aren't generated at all when $roleCount is zero | 13:28 |
| matbu | bandini: weird, the current-passed-ci is what is used for : https://ci.centos.org/view/rdo/view/tripleo-periodic/job/tripleo-quickstart-upgrade-major-mitaka-to-master/ | 13:28 |
| shardy | dprince: but the first step is moving all the Post stuff into overcloud.yaml | 13:28 |
| dprince | shardy: rather than moving it all into overcloud.yaml | 13:30 |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: WIP -- clean full upgrade review https://review.openstack.org/364859 | 13:30 |
| dprince | shardy: could we create a puppet/post.yaml? | 13:30 |
| dprince | shardy: that is what we are aiming for w/ docker anyways... | 13:30 |
| shardy | dprince: I suppose we could, but then I'll have to j2 template two files | 13:30 |
| shardy | I guess that would be OK | 13:30 |
| bandini | matbu: very odd | 13:30 |
| dprince | shardy: I think it is actually good that it would be 2 | 13:31 |
| bandini | matbu: is there a way for me to trigger that periodic job on a specific patch? | 13:31 |
| shardy | dprince: do you have a patch showing how the post.yaml stuff is expected to work? | 13:31 |
| jrist | mornin' all | 13:32 |
| shardy | dprince: I'm fine with moving it, but we'll need to modify the mistral action to find all j2.yaml files, not just the overcloud one | 13:32 |
| shardy | shouldn't be too hard | 13:32 |
| dprince | shardy: sharing across baremetal vs. containers is going difficult if not impossible | 13:33 |
| dprince | shardy: 2 templates I think is a good design actually | 13:33 |
| dprince | shardy: trying to share things too much is sometimes bad, as is sharing to little | 13:33 |
| shardy | dprince: Ok, so we won't be doing the same step approach, or it's just going to be implemented very differently? | 13:33 |
| shardy | I assumed we'd just plug in different *Config resources, which is why I moved them all into nested stacks | 13:33 |
| dprince | shardy: we'll have steps. but they will mean slightly different things | 13:34 |
| shardy | dprince: Ok, cool, I'll try reworking into puppet/post.yaml and see how it looks, thanks for the feedback! | 13:34 |
| shardy | (this patch works, so hopefully will be cut/paste) | 13:34 |
| *** hjensas has quit IRC | 13:35 | |
| *** jpena|off is now known as jpena | 13:36 | |
| dprince | shardy: there may be a case for keeping some things at the top level too | 13:38 |
| dprince | shardy: in that we could keep cluster-wide deployment in lockstep across the roles without breakpoints | 13:38 |
| dprince | shardy: even still, we would need to consider templating mechanism to have different steps for puppet, vs alternatives like containers | 13:39 |
| shardy | dprince: perhaps we should land it all at the top-level, then refactor when we know exactly what e.g container integration will look like? | 13:39 |
| shardy | dprince: I've tried to keep puppet-isms out of overcloud.yaml, but if you look at the end of that series, it's getting pretty lightweight when j2 templates it all | 13:40 |
| dprince | https://review.openstack.org/#/c/330659/18/docker/post.yaml | 13:40 |
| pradk | EmilienM, ok, well logs indicate you still have redis issue, which perhaps means the redis vip patch with keepalived isnt part of your rebase ? | 13:41 |
| EmilienM | pradk: oh right | 13:41 |
| EmilienM | pradk: I also need it | 13:41 |
| shardy | dprince: interesting, but we'd still need to template that for all roles, so the steps between roles can be aligned | 13:41 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates: Add steps to containerized compute deployment https://review.openstack.org/346927 | 13:41 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates: WIP: Containerized Services for Composable Roles https://review.openstack.org/330659 | 13:41 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates: Bind mount files to run DiD in latest atomic host https://review.openstack.org/347218 | 13:41 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates: Have docker start script honor configuration https://review.openstack.org/366138 | 13:41 |
| shardy | dprince: If you're OK with that, I'll move all the post stuff into a puppet/post.yaml | 13:42 |
| EmilienM | pradk: wait, it has it in the serie | 13:42 |
| dprince | shardy: I like puppet/post.yaml the best for now I think | 13:42 |
| *** saneax is now known as saneax-_-|AFK | 13:42 | |
| shardy | then j2 template it like I did in https://review.openstack.org/#/c/365796/ | 13:42 |
| openstackgerrit | Merged openstack/puppet-tripleo: Map ctlplane network to management fqdn https://review.openstack.org/365588 | 13:42 |
| openstackgerrit | Merged openstack/tripleo-common: Change default deploy artifacts container name https://review.openstack.org/365783 | 13:42 |
| shardy | dprince: ack, Ok I'll go with that then, thanks! | 13:42 |
| dprince | shardy: it doesn't solve the lockstep/no-breakpoints feature thing I mentioned but that is... a new feature. We can do it later | 13:43 |
| shardy | dprince: ack, yeah - pretty pushed for time with this already so I'd rather tackle that in ocata | 13:43 |
| *** pblaho has quit IRC | 13:43 | |
| *** pblaho has joined #tripleo | 13:44 | |
| *** yamahata has quit IRC | 13:44 | |
| *** pblaho has quit IRC | 13:44 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Fix aodh auth url to remove suffix https://review.openstack.org/365117 | 13:44 |
| *** tobias_fiberdata has quit IRC | 13:44 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Fix aodh auth url to remove suffix https://review.openstack.org/365117 | 13:45 |
| EmilienM | pradk: ok now it is :) | 13:45 |
| *** pblaho has joined #tripleo | 13:45 | |
| pradk | cool | 13:46 |
| jaosorior | EmilienM: did you change anything to get scenario001 to work? | 13:46 |
| *** xuao has joined #tripleo | 13:48 | |
| EmilienM | jaosorior: where does it work? | 13:51 |
| *** radeks has joined #tripleo | 13:54 | |
| EmilienM | tripleo meeting in 5 minutes | 13:54 |
| *** tobias_fiberdata has joined #tripleo | 13:56 | |
| ansiwen | mwhahaha: hi, thanks for reviewing my provider | 13:59 |
| dprince | d0ugal: nice on finally landing this https://review.openstack.org/#/c/342261/ | 14:00 |
| ansiwen | mwhahaha: I have no idea if V2_0 or V3 has to be used, so I will use V3 now. | 14:00 |
| dprince | d0ugal: wasn't the easiest patch to land | 14:01 |
| mwhahaha | sure, i think v3 is preferred for new stuff | 14:01 |
| ansiwen | mwhahaha: can you comment on the puppet type? I removed the name parameter. but now I realised I need project and user as parameters, because else everithing is run as administrator. but strictly neither is unique, so do I have to mark both as "namevar" ? | 14:02 |
| mwhahaha | ansiwen: refresh my memory, which review is it? | 14:04 |
| ansiwen | mwhahaha: sure sorry | 14:05 |
| ansiwen | mwhahaha: https://review.openstack.org/#/c/364580 | 14:05 |
| mandre | EmilienM: regarding your comment in https://review.openstack.org/#/c/363167/, should it be a require or a notify? | 14:06 |
| mwhahaha | ansiwen: that's a good question, give me a few minutes and i'll get back to you | 14:07 |
| EmilienM | mandre: tripleo meeting atm | 14:07 |
| ansiwen | mwhahaha: I'm currently reading this: https://docs.puppet.com/puppet/latest/reference/lang_resources.html#namenamevar | 14:07 |
| *** eggmaster has joined #tripleo | 14:08 | |
| d0ugal | dprince: heh, thanks :) Yeah, it took a while! | 14:09 |
| *** jcoufal has joined #tripleo | 14:09 | |
| mandre | EmilienM: zut, I forgot the meeting, thanks for the reminder | 14:10 |
| mwhahaha | ansiwen: technically i think they would both be the name var since the unique value would be user + project right? | 14:12 |
| *** trozet has quit IRC | 14:12 | |
| mwhahaha | ansiwen: i think you might want to use the name as the user by default and then project as an additional option. Alternatively support a name of user@project and then you could just split them apart in code. | 14:12 |
| *** trozet has joined #tripleo | 14:13 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Re-add undercloud.yaml https://review.openstack.org/352037 | 14:13 |
| mwhahaha | ansiwen: there's something similar in puppet-keystone around a composite namevar, you could check with chem around it since he wrote it i think | 14:13 |
| ansiwen | mwhahaha: actually I'm not even sure about that... maybe it's possible to create several ec2 credentials per user/project pair. the uniqe id would be the access id, that is created by the provider itself, I guess | 14:13 |
| mwhahaha | oh is it one of those auto generated ids? | 14:14 |
| mwhahaha | does it support setting the id? | 14:14 |
| mwhahaha | I think the nova flavor provider or the glance image provider might handle a case similarly. i'll have to double check | 14:14 |
| chem | ansiwen: mwhahaha I wouldn't go the composite namevar way, it add a layer of complexity not required for this. | 14:15 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for aodh https://review.openstack.org/365457 | 14:16 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for keystone https://review.openstack.org/327029 | 14:16 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Add class to write overcloud VIPs into /etc/hosts https://review.openstack.org/357762 | 14:16 |
| ansiwen | I don't think it supports proposing an ID | 14:16 |
| ansiwen | chem: so, what would you do? | 14:16 |
| ansiwen | mwhahaha: I don't think it supports proposing an ID | 14:17 |
| *** michchap has quit IRC | 14:17 | |
| chem | ansiwen: how can you check that the resource has been created ? | 14:17 |
| chem | ansiwen: on the command line | 14:17 |
| mwhahaha | that's unfortunate. | 14:17 |
| ansiwen | chem: well, the commandline returns the credential values when you create them, and you can do "openstack ec2 credentials list" | 14:18 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Hook internal TLS flag to apache-based services https://review.openstack.org/366075 | 14:18 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts https://review.openstack.org/357765 | 14:18 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add parameters for internal TLS https://review.openstack.org/365942 | 14:18 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add HAProxy TLS handled by certmonger as composable service https://review.openstack.org/356430 | 14:18 |
| chem | ansiwen: can you past a openstack ec2 credentials list result when your resource has been created ? | 14:19 |
| d0ugal | jtomasek: Are you still cherry-picking any reviews to make the UI work? or has everything landed? | 14:19 |
| ansiwen | chem, mwhahaha: can't I just make both user and project a normal argument, and the title has been chosen in a unique way by the provider-user? | 14:19 |
| jaosorior | EmilienM: not sure what you meant by "this is not a profile but a regular class". | 14:20 |
| chem | ansiwen: yes, that would be the right way for this | 14:20 |
| EmilienM | jaosorior: can we follow the meeting maybe? | 14:20 |
| EmilienM | and do code review a bit later | 14:20 |
| mwhahaha | You'd still need to be able to determine if it already exists | 14:20 |
| jaosorior | EmilienM: I'll try... got a meeting in after the tripleo meeting | 14:20 |
| mwhahaha | and if you can do multiple user/project you don't have a way to determine uniqueness | 14:20 |
| chem | ansiwen: composite namevar should be support in old puppet is not that great ... | 14:20 |
| *** links has quit IRC | 14:21 | |
| ansiwen | mwhahaha: exists is implemented already | 14:21 |
| *** oshvartz has quit IRC | 14:22 | |
| ansiwen | mwhahaha: actually, the provider also creates the tempest.conf... so it can check there... | 14:22 |
| mwhahaha | k then yea just let the name be the regular name var and add user/project as params | 14:22 |
| ansiwen | mwhahaha: so the name var will be unused within the provider code... just by the compiler to identify the resource | 14:23 |
| ansiwen | correct? | 14:23 |
| chem | ansiwen: correct | 14:23 |
| ansiwen | ok | 14:23 |
| *** xuao has quit IRC | 14:24 | |
| chem | ansiwen: mwhahaha again, not the best way, but the pragmatic way. It's not really a driver to the ec2 command, just a driver to the entry line in tempest conf. So let's keep this simple. | 14:25 |
| *** zigo has quit IRC | 14:25 | |
| *** xuao has joined #tripleo | 14:25 | |
| ansiwen | right | 14:25 |
| mwhahaha | sure | 14:25 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Add ipv6 nic-configs https://review.openstack.org/364479 | 14:25 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Add IPv6 network configuration for ipv6 job types https://review.openstack.org/363674 | 14:25 |
| *** michchap has joined #tripleo | 14:26 | |
| *** tobias-fiberdata has joined #tripleo | 14:27 | |
| *** akshai has quit IRC | 14:27 | |
| sshnaidm | bnemec, btw, did you track all performance tuning anywhere? I mean what was done and where | 14:29 |
| bnemec | sshnaidm: http://etherpad.corp.redhat.com/rh1-profile-switch | 14:29 |
| b00tcat | I'm getting an error when deploying an overcloud with VMs as stated in the documentation - How can I further debug this? http://paste.fedoraproject.org/422923/72162147 | 14:29 |
| sshnaidm | bnemec, cool, thanks | 14:30 |
| bnemec | Actually, since CI is hosed right now anyway, this might be a good time to switch the controller profile to performance. | 14:30 |
| bnemec | That _could_ mitigate the issues we're seeing there. | 14:30 |
| sshnaidm | bnemec, yeah, let's try | 14:30 |
| *** tobias_fiberdata has quit IRC | 14:30 | |
| bnemec | derekh: dprince: ^Thoughts on switching the performance profile of the controller now since we're basically in the middle of an outage anyway? | 14:31 |
| sshnaidm | bnemec, just wait for finishing heat deletes.. | 14:31 |
| dprince | bnemec: the risk is something else goes wrong and increases the outage | 14:32 |
| dprince | bnemec: do you have a guess on how much this might improve perf on the controller? | 14:32 |
| *** zigo has joined #tripleo | 14:32 | |
| *** zigo is now known as Guest39147 | 14:33 | |
| bnemec | dprince: The performance improvement on the computes was about 20% from my testing. | 14:33 |
| bnemec | Although that was in vms, whereas everything's running on baremetal on the controller. | 14:33 |
| dprince | bnemec: it is tempting, but I'd not like to increase the outage I think | 14:33 |
| dprince | bnemec: I'd say we wait | 14:34 |
| dprince | bnemec: my vote anyways | 14:34 |
| derekh | bnemec: how heavely loaded is the controller at peak times | 14:34 |
| *** Ryjedo has joined #tripleo | 14:34 | |
| bnemec | derekh: Very. At times the CPU will spike to 100% for extended periods of time. | 14:35 |
| *** xuao has quit IRC | 14:35 | |
| bnemec | Not often, but I've seen it happen even after tuning the heat-engine workers. | 14:35 |
| dprince | derekh: yep, it gets fairly loaded and is never idle | 14:35 |
| bnemec | sshnaidm: Have you seen the cleanup-stack script on the controller? | 14:35 |
| derekh | bnemec: dprince ok | 14:35 |
| sshnaidm | bnemec, nope | 14:35 |
| derekh | sshnaidm: sounds like you want that script | 14:35 |
| bnemec | I've been using that to delete DELETE_FAILED stacks. | 14:35 |
| sshnaidm | bnemec, sounds useful, where is it? | 14:36 |
| bnemec | sshnaidm: It's on the controller itself in the heat-admin home directory. | 14:36 |
| bnemec | Since I can't get to the heat api from my local system. | 14:36 |
| derekh | bnemec: so the only thing the reboot would help is the testenv creation time, if that is excessive then we should reboot ASAP if it isn't then waiting wouldn't be too bad | 14:37 |
| *** akshai has joined #tripleo | 14:37 | |
| dprince | derekh: my thought was that it could be somehow related to the missing neutron ports issue | 14:38 |
| bnemec | derekh: Well, we'd switch the performance profile, which may help lower the CPU load too. | 14:38 |
| dprince | derekh: load that is, but I haven't confirmed that | 14:38 |
| *** Guest39147 has quit IRC | 14:39 | |
| derekh | bnemec: yup, agreed it may help reduce the load, but from the perspective of the CI runtimes, it will only change the time it takes to create a testenv | 14:39 |
| derekh | unless it also helps the neutron ports problem dprince mentioned | 14:40 |
| derekh | 2016-09-05 16:01:05.470859 | + /opt/stack/new/tripleo-ci/toci_gate_test.sh::L233: sleep 1200 | 14:40 |
| derekh | 2016-09-05 16:10:08.137657 | 2016-09-05 16:10:08,136 - testenv-client - INFO - Received job : { | 14:40 |
| bnemec | derekh: Right, but it may also help with those node locked errors. If the controller is overloaded and can't respond to the bmc requests, it could cause those timeouts. | 14:40 |
| derekh | 9 minutes, could be a lot better | 14:40 |
| bnemec | It looks like job runtimes had already dropped a lot before this latest outage. | 14:41 |
| *** zigo_ has joined #tripleo | 14:42 | |
| derekh | bnemec: yup, probably a combination of the image caching and perfomance improvements | 14:42 |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 14:43 |
| *** kjw3 has joined #tripleo | 14:45 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 14:45 |
| pabelanger | derekh: bnemec: we should whiteboard how it would look like having a local nodepool server manage OVB nodes for tripleo. Specifically, now the shared network would look | 14:45 |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Handle image location based on CI_ENV and REL_TYPE https://review.openstack.org/359796 | 14:46 |
| derekh | pabelanger: can nodepool manage heat stacks? | 14:46 |
| d0ugal | shardy: Do you think it is a bit late/disruptive to try and do this? https://review.openstack.org/#/c/357682/ | 14:47 |
| ayoung | dprince, I see that apevec pinned master to Keyston N3 to avoid the Credentials encryption test breaking. | 14:47 |
| *** zigo_ has quit IRC | 14:47 | |
| d0ugal | shardy: Decided it probably wasn't worth holding the meeting for :) | 14:47 |
| ayoung | I have a proof of concept on using Fernet. | 14:47 |
| EmilienM | jaosorior: ok so profiles are different from regular manifests | 14:47 |
| dprince | ayoung: okay, good to know | 14:47 |
| EmilienM | jaosorior: profiles deploy services | 14:47 |
| pabelanger | derekh: no, it uses shade to launch the nodes | 14:48 |
| EmilienM | jaosorior: think containers. 1 profile could be 1 container | 14:48 |
| pabelanger | I keep forgetting you are using heat to do it | 14:48 |
| jaosorior | EmilienM: ah, right | 14:48 |
| ayoung | dprince, I'm using a Heat (swift ) artifact. | 14:48 |
| jaosorior | EmilienM: So that doesn't have to do with the "services" we deploy in the service chain | 14:48 |
| jaosorior | right? | 14:48 |
| *** zigo_ has joined #tripleo | 14:48 | |
| jaosorior | cause we have stuff that is not a "profile" in a sense, in that list | 14:48 |
| ayoung | EmilienM, do we have support for Fernet keys in the Keystone puppet module? | 14:49 |
| derekh | pabelanger: I'm sure it can be done without heat, but it would be nice to keep using the OVB heat templates and take advantage to any improvements | 14:49 |
| jaosorior | EmilienM: such as the tripleo packages, firewall, CA-related things, and I'm trying to add the /etc/hosts additions https://review.openstack.org/#/c/357765/ | 14:49 |
| shardy | d0ugal: I guess now is better than after we branch stable/newton | 14:49 |
| openstackgerrit | Tomas Sedovic proposed openstack/tripleo-common: Allow the validations to run openstack commands https://review.openstack.org/366175 | 14:49 |
| shardy | d0ugal: one question, do we have folks consuming this yet which requires deprecating the old names? | 14:50 |
| EmilienM | ayoung: of course | 14:50 |
| pabelanger | derekh: well, we could start with having nodepool manage the images. Build / upload them into the cloud | 14:50 |
| EmilienM | ayoung: we even gate on it | 14:50 |
| shardy | tzumainn: ^^ are you going to be impacted by changing the mistral action names? | 14:50 |
| EmilienM | ayoung: https://github.com/openstack/puppet-openstack-integration#description | 14:50 |
| openstackgerrit | Tomas Sedovic proposed openstack/tripleo-validations: Check for differences between introspected HW https://review.openstack.org/353978 | 14:50 |
| d0ugal | shardy: The CLI and the GUI will need updated at least - if we just remove them that'll break CI until they update. | 14:51 |
| derekh | pabelanger: but for that part we're only using a vanilla centos cloud image | 14:51 |
| d0ugal | I really wish we didn't encourage direct action calls, but that is another subject. | 14:51 |
| shadower | mandre, jtomasek: so https://review.openstack.org/366175 and https://review.openstack.org/353978 should fix the discovery diff issue | 14:51 |
| shardy | d0ugal: Ok, I was thinking it'd just be the workflows inside tripleo-common | 14:51 |
| EmilienM | ayoung: https://github.com/openstack/puppet-keystone/blob/master/manifests/init.pp#L411-L416 | 14:51 |
| shardy | so if we can't do it in one patch this seems OK | 14:51 |
| shardy | d0ugal: can we switch everything over before the release? | 14:51 |
| shadower | mandre, jtomasek: it does so by fixing up the mistral environment (which is different than I expected) | 14:51 |
| d0ugal | shardy: Nah, some actions are called directly by the CLI | 14:51 |
| EmilienM | jaosorior: right we have stuffs that shouldn't be profiles | 14:51 |
| EmilienM | jaosorior: we should move it later | 14:51 |
| d0ugal | shadower: I think so, the patches are trivial. I can prepare them. | 14:52 |
| d0ugal | shardy: ^ | 14:52 |
| EmilienM | jaosorior: but for this new one, i think we should put it as a regular manifest | 14:52 |
| shadower | jtomasek: but the validation will fail until you run node introspection | 14:52 |
| d0ugal | shadower: (sorry) | 14:52 |
| jaosorior | EmilienM: it already is | 14:52 |
| shardy | d0ugal: relatedly, did you have a chance to check out wiring in the action to render overcloud.yaml? | 14:52 |
| jaosorior | ah wait | 14:52 |
| shadower | d0ugal: no worries | 14:52 |
| shardy | d0ugal: I was going to take a look later if not | 14:52 |
| pabelanger | derekh: right, but it looks like you online deploy the image once. Then reused it. Also means you are not using diskimage-builder to build them too | 14:52 |
| jaosorior | EmilienM: nevermind; yeah, did the change so now it's just manifests/vip_hosts.pp | 14:52 |
| ayoung | EmilienM, But how about a way to say "here is the Key to use" | 14:52 |
| EmilienM | ayoung: how to do it? by generating the file? | 14:53 |
| pabelanger | moving to nodepool, we get new images every day, build with diskimage-builder. | 14:53 |
| openstackgerrit | Tomas Sedovic proposed openstack/tripleo-validations: Check for differences between introspected HW https://review.openstack.org/353978 | 14:53 |
| tzumainn | shardy, thanks for asking! rwsu is currently adding ironic node registration to manageiq, and he's using the mistral workflows and actions there | 14:53 |
| ayoung | EmilienM, well, while I did a tarball, the right way is, I think, to have the hiera data speficy the keys in a hidden field | 14:53 |
| ayoung | EmilienM, http://adam.younglogic.com/2016/09/fernet-overcloud/ | 14:53 |
| derekh | pabelanger: sorry hold on I got mixed up there, which nodes are you talking about? | 14:53 |
| EmilienM | ayoung: you can specify which directory to use keys and say puppet to not generate the keys | 14:54 |
| jaosorior | ayoung: thought you didn't like injecting keys | 14:54 |
| ayoung | jaosorior, I don't | 14:54 |
| d0ugal | shardy: I started, but I've really made any progress. I've been a bit distracted today if I'm honest. | 14:54 |
| EmilienM | ayoung: so puppet will just take care of writting the path in keystone.conf | 14:54 |
| ayoung | jaosorior, but I also hate passwords an we have them all over the place | 14:54 |
| shardy | tzumainn: Ok, so it looks like we're going for a late rename of some of them prior to the newton release, is that manageable for you? | 14:54 |
| openstackgerrit | Merged openstack/puppet-tripleo: Convert ringbuilder to build devices array https://review.openstack.org/364737 | 14:54 |
| shardy | tzumainn: the alternative will be to leave the old names in place for a cycle I guess | 14:54 |
| ayoung | EmilienM, if you look inside the "repo" it looks like this | 14:54 |
| pabelanger | derekh: oops, I was looking at prepare-ovh-cloud.sh, let me find where OVB nodes are created | 14:54 |
| shardy | tzumainn: https://review.openstack.org/#/c/357682/8/setup.cfg | 14:55 |
| tzumainn | shardy, I think the rename is manageable, we'll go along with whatever you guys release for newton! | 14:55 |
| jaosorior | ayoung: what happened to the custodia-based approach? | 14:55 |
| jaosorior | ayoung: or can't we do it with barbican? Heat has support for it | 14:55 |
| shardy | tzumainn: Ok, obviously we'll be more cautious about this stuff after it's released :) | 14:55 |
| shardy | d0ugal: Ok, np, I think I know what needs to happen so I'll hack on my patch until it works | 14:55 |
| ayoung | EmilienM, http://paste.openstack.org/show/567189/ | 14:55 |
| shardy | d0ugal: can you show me an example where an action is called directly please? | 14:55 |
| jaosorior | ayoung: http://docs.openstack.org/developer/heat/template_guide/openstack.html#OS::Barbican::Secret | 14:56 |
| shardy | cut/paste ftw ;) | 14:56 |
| derekh | pabelanger: for the ovb nodes the image that is used is irrelevant, the the CI undercloud boots them with ironic they get the overcloud-full image built during that CI job | 14:56 |
| ayoung | jaosorior, custodia is not going to be ready for a while | 14:56 |
| ayoung | jaosorior, barbican is not deployed on the undercloud, and I don't want to force it just for this. | 14:56 |
| *** anshul has quit IRC | 14:56 | |
| ayoung | barbican also implies that it is a "pull" and that the controller nodes have credentials in the undercloud keystone | 14:56 |
| ayoung | all do-able, but more complex | 14:57 |
| *** ifarkas is now known as ifarkas_afk | 14:57 | |
| openstackgerrit | Merged openstack/puppet-tripleo: Add Ceph RGW listener to HAProxy https://review.openstack.org/362816 | 14:57 |
| pabelanger | derekh: Right. That's the step I missed, I didn't see ironic in the picture. So, in that case, we can't do anything today | 14:57 |
| EmilienM | ayoung: I have no idea how to implement the keystone-manage credential setup thing in tripleo | 14:58 |
| jaosorior | ayoung: alright. Then I guess hieradata is the way to go | 14:58 |
| EmilienM | we do support it now in puppet-keystone but only config/generation | 14:58 |
| EmilienM | in tripleo we need to export the credential somewhere and collect it on other keystone nodes | 14:58 |
| d0ugal | shardy: Creating a plan requires calling actions. https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/workflows/plan_management.py#L85-L88 | 14:58 |
| derekh | pabelanger: yup afraid so, we specifically want to test that ironic is working so I'm not sure much can be done there, | 14:59 |
| d0ugal | shardy: create_container is an action call, so is create_deployment_plan - in the middle we upload to swift directly. | 14:59 |
| d0ugal | shardy: I've been -1 to this approach, but so far I've been the only one :) | 14:59 |
| derekh | pabelanger: but as more jobs may move to the multinode model (away from OVB) we might be relying on it less | 14:59 |
| pabelanger | d0ugal: ya, we need to wait for zuulv3. That should expose baremetal provisioning | 14:59 |
| pabelanger | derekh: Yup, or that | 14:59 |
| d0ugal | shardy: the patch florianf linked will also have the GUI call the deploy action directly. We hadn't ever discussed that being called directly. | 14:59 |
| ayoung | EmilienM, I'll get you some guidance. I think the right thing will be to generate all keys on the undercloud and copy out to the controllers | 15:00 |
| *** pradk has quit IRC | 15:00 | |
| florianf | d0ugal, shardy: We do call some actions directly if we want immediate output (like listing the plans). | 15:01 |
| EmilienM | ayoung: right now I have zero time to work on it. i'm focused on getting tripleo RC1 done | 15:01 |
| shardy | d0ugal: Ok, well perhaps that's something to revisit in ocata | 15:01 |
| d0ugal | florianf, shardy: Yeah, that is one we had discussed as a direct call. I don't think calling deploy is wise.... and this is why I don't like it, it is very vague :/ | 15:01 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Convert SwiftDevicesAndProxyConfig to composable format https://review.openstack.org/364748 | 15:02 |
| shardy | I agree it's not an ideal pattern, perhaps we can refactor things to make the workflows more easily consumable | 15:02 |
| ayoung | EmilienM, this falls into that category. I'll work on it, but I am in over my head | 15:02 |
| tzumainn | shardy, haha, I trust you guys ;) | 15:02 |
| d0ugal | tzumainn: wow | 15:02 |
| *** rodrigods has quit IRC | 15:02 | |
| *** rodrigods has joined #tripleo | 15:02 | |
| EmilienM | ayoung: where does it fall in RC1? | 15:02 |
| tzumainn | d0ugal, I know, right? | 15:02 |
| jpich | florianf: I still think it's fine for the simple actions, but 'deploy' seems a bit more prone to changes/issues if I understand the UI patch correctly | 15:02 |
| EmilienM | ayoung: is there a launchpad bug somewhere? | 15:02 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Move role deployment steps into overcloud.yaml https://review.openstack.org/365763 | 15:03 |
| *** jpena is now known as jpena|away | 15:03 | |
| openstackgerrit | Merged openstack/puppet-tripleo: Include heat base profile outside of step https://review.openstack.org/352713 | 15:04 |
| ayoung | EmilienM, this is the breakage from Friday | 15:05 |
| florianf | jpich, d0ugal: I think in this case it doesn't make much difference from the UI side. So switching to call the workflow sounds fine to me. jtomasek's the owner of the patch, let's see what he says. | 15:05 |
| *** zoliXXL is now known as zoli|mtg | 15:05 | |
| pabelanger | EmilienM: derekh: shardy: dprince: How does this week look to enable shared networks for OVB? https://review.openstack.org/#/c/357308/ This should be a noop change but will allow different projects to interact with the networks created for OVB | 15:05 |
| d0ugal | florianf: k, thanks :) | 15:05 |
| jpich | florianf: Cool! | 15:06 |
| dprince | pabelanger: we are in the middle of an outage. Until that is over I would like to wait | 15:07 |
| dprince | pabelanger: otherwise we can't tell if something breaks... | 15:07 |
| sshnaidm | bnemec, your script could be run in parallel, right? | 15:08 |
| bnemec | sshnaidm: Should be. Even with the script I'm getting weird errors deleting stacks though. :-( | 15:08 |
| sshnaidm | bnemec, do you delete stack now too? | 15:09 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart: Add config parameter to collect-logs ci-script https://review.openstack.org/364394 | 15:09 |
| bnemec | sshnaidm: It's failing to delete subnets that are already deleted due to port conflicts, which makes no sense. | 15:09 |
| pabelanger | dprince: sure, once we are back online of course | 15:09 |
| sshnaidm | bnemec, and the port still exists? | 15:10 |
| bnemec | sshnaidm: No, the ports were deleted correctly. It's nonsense. | 15:10 |
| shardy | bnemec: is convergence enabled? | 15:12 |
| sshnaidm | bnemec, hmm.. curiouser and curiouser | 15:12 |
| * shardy isn't sure if we're discussing undercloud or overcloud heat here | 15:12 | |
| bnemec | shardy: Shouldn't be. This is a Mitaka cloud. | 15:12 |
| shardy | bnemec: ah | 15:12 |
| bnemec | Retrying stack-delete enough times did eventually make it go away. | 15:13 |
| *** yamahata has joined #tripleo | 15:14 | |
| *** dtrainor has quit IRC | 15:15 | |
| *** dtrainor has joined #tripleo | 15:15 | |
| sshnaidm | bnemec, are you going to tune there performance and shut it down? | 15:15 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: Set Redis VIP on all nodes https://review.openstack.org/366128 | 15:16 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: Use Redis VIP when deploying with keepalived https://review.openstack.org/364917 | 15:16 |
| bnemec | sshnaidm: I wasn't planning to. dprince didn't want to add more variables to the outage, which is fair. | 15:16 |
| sshnaidm | bnemec, btw, I run this script for deleting stack right now, it may clash with you try if we deleted it in the same time | 15:16 |
| *** aufi has quit IRC | 15:17 | |
| bnemec | sshnaidm: I've been picking random stacks from the middle of the list, so if you start at the top we should be pretty safe. :-) | 15:17 |
| sshnaidm | bnemec, pity, I would restart it completely.. | 15:17 |
| bnemec | We still need to do that at some point. | 15:17 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Fix aodh auth url to remove suffix https://review.openstack.org/365117 | 15:17 |
| derekh | sshnaidm: bnemec also we were still seeing db connection problems up until sshnaidm stoped the te_broker, so something needs to be restarted | 15:18 |
| *** thrash is now known as thrash|biab | 15:18 | |
| *** ebarrera has quit IRC | 15:18 | |
| bnemec | derekh: Oh look, my last stack-delete failed on a mysql connection error. | 15:19 |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Work around deletion of _member_ role assignments on upgrade https://review.openstack.org/307352 | 15:19 |
| *** pradk has joined #tripleo | 15:20 | |
| *** oshvartz has joined #tripleo | 15:20 | |
| *** zigo_ is now known as zigo | 15:20 | |
| *** fragatina has joined #tripleo | 15:20 | |
| derekh | sshnaidm: bnemec so we've aready restarted haproxy and heat-engine so far, I'd say either mysql is next or a complete restart, after which monitor open files, tcp connections etc... and figure out whats leaking | 15:21 |
| derekh | dprince: ^ | 15:21 |
| bnemec | Yeah, strangely there don't appear to be a huge number of db connections open. | 15:21 |
| EmilienM | jistr: you take https://bugs.launchpad.net/tripleo/+bug/1620657 ? | 15:22 |
| openstack | Launchpad bug 1620657 in tripleo "OpenConrail loadbalancer plugin value is incorrect" [High,New] | 15:22 |
| dprince | derekh: ack. FWIW if we are doing a complete restart may as well let bnemec change the CPU settings | 15:22 |
| bnemec | | Max_used_connections | 891 | | 15:22 |
| openstackgerrit | Merged openstack/tripleo-common: Add List Roles Action https://review.openstack.org/330283 | 15:22 |
| bnemec | That's really not very many. | 15:22 |
| derekh | bnemec: it may also be the size of the heat db tables aswell, causing things to timeout, | 15:22 |
| derekh | bnemec: ya | 15:22 |
| derekh | dprince: yup, makes sense | 15:22 |
| dprince | derekh: could we just delete/recreate the heat db tables? | 15:23 |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 15:23 |
| bnemec | 5.4G Sep 6 14:11 raw_template.ibd | 15:23 |
| bnemec | Ouch | 15:23 |
| derekh | dprince: maybe, its crazy enough that it just might work ;-) | 15:23 |
| derekh | bnemec: http://paste.openstack.org/show/567125/ | 15:24 |
| derekh | bnemec: we ran heat-manage purge on it earlier but it did nothing | 15:24 |
| sshnaidm | bnemec, we tried heat-manage purge , but it just didn't work | 15:24 |
| shardy | We do have a new enough heat to have https://bugs.launchpad.net/heat/+bug/1524387 fixed, right? | 15:26 |
| openstack | Launchpad bug 1524387 in Ubuntu Cloud Archive liberty "heat-manage purge_deleted broken" [Undecided,In progress] - Assigned to Seyeong Kim (xtrusia) | 15:26 |
| shardy | That was fixed during mitaka, so I assume so | 15:27 |
| *** fragatina has quit IRC | 15:27 | |
| bnemec | Ouch, lots of db connection errors in the heat logs. | 15:27 |
| shardy | therve: ^^ Hey are you aware of any more recent heat-manage purge_deleted bugs that may not have made it onto mitaka yet? | 15:27 |
| shardy | I recall we've had a few issues but am failing to find all-the-bugs atm | 15:28 |
| derekh | shardy: looks like we have that fix | 15:28 |
| sshnaidm | we have openstack-heat-engine-6.0.1-0.20160722173744.656ac73.el7.centos.noarch | 15:29 |
| bnemec | mongo is eating 20 GB of memory on the controller | 15:29 |
| bnemec | ! | 15:29 |
| shardy | :( | 15:29 |
| derekh | bnemec: ya, I was gonna bring that up after this | 15:29 |
| shardy | bnemec: do we even need it? | 15:29 |
| bnemec | We shouldn't. | 15:30 |
| EmilienM | stop ceilo and mongo | 15:30 |
| EmilienM | we don't need it | 15:30 |
| bnemec | We aren't using ceilometer. | 15:30 |
| shardy | or zaqar | 15:30 |
| bnemec | I bounced mongo, which freed up the memory. | 15:31 |
| dprince | I stopped ceilo last week | 15:31 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Move role deployment steps into overcloud.yaml https://review.openstack.org/365763 | 15:31 |
| shardy | dprince: ^^ I think that reworks things into /post.yaml like you wanted | 15:31 |
| dprince | shardy: looking | 15:32 |
| openstackgerrit | Merged openstack/tripleo-common: Fix pep8 errors when running with python3 https://review.openstack.org/365724 | 15:33 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Cleanup haproxy service from explicit services enablement https://review.openstack.org/362800 | 15:33 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: pingtest: run 'openstack stack failures list' when failure https://review.openstack.org/363918 | 15:34 |
| dprince | shardy: one quick comment on passing just 'step_config' into post.yaml | 15:34 |
| derekh | Ok, we gotta make a decision here ;-), | 15:34 |
| derekh | 1. restart mysql | 15:34 |
| derekh | 2. restart controller | 15:34 |
| derekh | 3. delete and recreate the heat database | 15:34 |
| derekh | that the order I reckon we should try thing in, thoughts? | 15:34 |
| derekh | bnemec: sshnaidm dprince shardy ^ | 15:35 |
| dprince | shardy: I think it would be better to just pass role_data in... | 15:35 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Hook internal TLS flag to apache-based services https://review.openstack.org/366075 | 15:35 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts https://review.openstack.org/357765 | 15:35 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add VIP names to allNodesConfig https://review.openstack.org/365895 | 15:35 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add parameters for internal TLS https://review.openstack.org/365942 | 15:35 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add HAProxy TLS handled by certmonger as composable service https://review.openstack.org/356430 | 15:35 |
| shardy | dprince: ack, yup true | 15:35 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add bootstrap_node and vip_data to hierarchy for all roles https://review.openstack.org/366049 | 15:35 |
| * shardy seds things | 15:35 | |
| sshnaidm | derekh, +1 | 15:35 |
| bnemec | derekh: Sounds reasonable | 15:35 |
| dprince | derekh: if we can easily control the resources I'd go for bouncing 1 and 3 first | 15:36 |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: WIP -- clean full upgrade review https://review.openstack.org/364859 | 15:36 |
| dprince | derekh: if that doesn't work then follow up with #2 and let bnemec make his change | 15:36 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: DO NOT MERGE - TESTING BARBICAN https://review.openstack.org/353262 | 15:36 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add Barbican to the overcloud - WIP https://review.openstack.org/352458 | 15:36 |
| dprince | derekh: to the BIOS | 15:36 |
| derekh | dprince: easily contol the resources? | 15:36 |
| dprince | derekh: sorry, like can we make sure any reasources created by those heat stacks get cleaned up elsewhere? | 15:37 |
| dprince | derekh: nova and neutron stufff that gets created... | 15:37 |
| derekh | I should be able to do "systemctl restart mariadb" right ? there is no galera magic going on is there? | 15:37 |
| derekh | dprince: ahh, ok, ya we should be able to do that I think | 15:37 |
| dprince | derekh: correct, without pacemaker systemd is fine | 15:38 |
| bnemec | Yeah | 15:38 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Update OpenContrail loadbalancer plugin value https://review.openstack.org/366215 | 15:38 |
| *** matbu is now known as matbu|bbl | 15:38 | |
| EmilienM | jistr: I took it ^ | 15:39 |
| jpich | I'd like to help self-triage the bugs I open at least, is there a process to get the right Launchpad permissions? | 15:39 |
| bnemec | jpich: I think shardy needs to add you to the tripleo drivers team. | 15:41 |
| EmilienM | jpich: let me add you, a sec | 15:41 |
| shardy | jpich: I'll add you, sec | 15:41 |
| jpich | bnemec, EmilienM, shardy: Thanks! | 15:41 |
| shardy | jpich: done | 15:41 |
| EmilienM | he was faster :( | 15:42 |
| bnemec | derekh: dprince: I need to leave shortly for an appt. If we do delete the heat dbs, someone please grab a list of the stacks beforehand so we know what we need to clean up. | 15:42 |
| jpich | Haha | 15:42 |
| jpich | Cheers! | 15:42 |
| *** pcaruana has joined #tripleo | 15:42 | |
| bnemec | That will at least make our lives easier. | 15:42 |
| derekh | bnemec: will do | 15:42 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart: Add config parameter to collect-logs ci-script https://review.openstack.org/364394 | 15:43 |
| derekh | its still restarting | 15:44 |
| *** thrash|biab is now known as thrash | 15:44 | |
| EmilienM | shardy: can you define "tripleo ongoing" milestone? | 15:44 |
| shardy | EmilienM: it's for stuff unrelated to the release deliverables, like constantly fixing CI | 15:44 |
| EmilienM | ok | 15:45 |
| dprince | derekh: I stopped openstack-heat-api-cloudwatch, and all the ceilo* processes last week | 15:45 |
| derekh | mysql taking its time restarting, I guess its flushing data to disk, gonna wait it out | 15:46 |
| dprince | derekh: ack, we could stop nova-api to make it easier on it perhaps | 15:47 |
| *** bvandenh has quit IRC | 15:47 | |
| dprince | derekh: and neutron-server. Those two APIs get queried a lot | 15:47 |
| *** leanderthal is now known as leanderthal|afk | 15:48 | |
| derekh | dprince: done and done | 15:48 |
| * dprince likes watching the haproxy broadcast messages as derekh restarts things | 15:48 | |
| dprince | derekh: I think mariadb is restarted? | 15:50 |
| dprince | derekh: systemd indicates active since 2 minutes ago or so... | 15:50 |
| derekh | dprince: yup, we're back, the other two services are now running again too | 15:50 |
| derekh | sshnaidm: now lets see if we can delete things again | 15:50 |
| * derekh restarted heat-engine also for the laugh | 15:51 | |
| *** bana_k has joined #tripleo | 15:52 | |
| dprince | derekh: going to clear out the heat stacks too? Or wait? | 15:52 |
| dprince | derekh: I mean the heat database? | 15:53 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart: Add config parameter to collect-logs ci-script https://review.openstack.org/364394 | 15:53 |
| derekh | sshnaidm: dprince I'm wondering if the db problems cause resources to get created that heat doesn't have record of, so I'm going to first try and manually delete everything outside of heat | 15:55 |
| derekh | the delete the stacks which should have nothing left in them | 15:55 |
| dprince | derekh: ack | 15:56 |
| sshnaidm | derekh, it's what bnemec's script does actually | 15:56 |
| sshnaidm | derekh, but if there is faster way to delete everything.. | 15:56 |
| pabelanger | derekh: shardy: EmilienM: something wrong with triple-test-cloud-rh1? We are getting exceptions when nodepool is deleting nodes | 15:57 |
| pabelanger | http://grafana.openstack.org/dashboard/db/nodepool-tripleo-test-cloud | 15:57 |
| sshnaidm | pabelanger, db was restarted | 15:58 |
| pabelanger | sshnaidm: what was the issue? | 15:58 |
| sshnaidm | pabelanger, ovb is not functional now, there are a lot of them actually.. | 15:59 |
| pabelanger | ok | 16:00 |
| sshnaidm | pabelanger, if shortly - environments are failed to create, heat mostly fails to delete and create stacks | 16:00 |
| pabelanger | sshnaidm: so you believe there is issues with heat running on tripleo-test-cloud-rh1? | 16:01 |
| shardy | derekh: if heat makes the call to create something then can't update the resource record with the ID, we could potentially leak resources | 16:02 |
| sshnaidm | pabelanger, yes, not sure it's all fault of heat, but the issues we see firstly on its actions | 16:02 |
| derekh | shardy: ok, I'm deleting eveything now and bypassing heat to see how that goes | 16:03 |
| dprince | derekh, sshnaidm, bnemec: before and after restart memory usuages here: http://paste.openstack.org/show/567202/ | 16:03 |
| derekh | dprince: sshnaidm bnemec FYI what I'm running, very slow ---> neutron port-list | grep baremetal- | awk '{print $2}' | xargs -t -n 1 neutron port-delete | 16:04 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Move role deployment steps into puppet/post.yaml https://review.openstack.org/365763 | 16:07 |
| shardy | dprince: ^^ addressed your comments I think | 16:07 |
| shardy | wfm locally too | 16:07 |
| *** mcornea has quit IRC | 16:07 | |
| *** abregman has quit IRC | 16:08 | |
| *** dsariel has quit IRC | 16:13 | |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: keystone: configure credentials https://review.openstack.org/366240 | 16:13 |
| derekh | sshnaidm: dprince bnemec also running this ->> nova list | grep baremetal- | awk '{print $4}' | xargs -n 1 -t nova delete | 16:13 |
| derekh | these are going to take ages | 16:14 |
| derekh | back in a bit | 16:14 |
| *** derekh is now known as derekh_afk | 16:14 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Handle image location based on CI_ENV and REL_TYPE https://review.openstack.org/359796 | 16:15 |
| jistr | EmilienM: ok thanks :)) | 16:16 |
| jtomasek | d0ugal: I dont' cherry pick anything at the moment, although there is a bunch of patches up that fixes various stuff | 16:16 |
| jistr | re the lbaas value BZ | 16:16 |
| jtomasek | d0ugal, florianf: re deploy action, we can call whatever is prefered. In my patch I called deploy action directly because calling workflow did not bring any additional value | 16:17 |
| *** nyechiel_ has joined #tripleo | 16:17 | |
| jtomasek | Originally I thought deploy workflow would run until the deployment finishes | 16:17 |
| sshnaidm | derekh_afk, if no ports are left, we can try stack deletes | 16:17 |
| jtomasek | shardy: sorry for missing the meeting:/ | 16:18 |
| d0ugal | jtomasek: Yeah, I think the deploy will do that eventually - it can't now because of token timeouts. | 16:20 |
| d0ugal | jtomasek: I'd guess the deploy workflow is a big candidate for adding extra checks, steps etc. so please do use the workflow | 16:21 |
| jtomasek | d0ugal: yes, makes total sense | 16:21 |
| *** rhallisey has quit IRC | 16:21 | |
| jtomasek | d0ugal: I'll update my GUI patch to run workflow | 16:22 |
| d0ugal | jtomasek: Thanks! | 16:22 |
| *** rhallisey has joined #tripleo | 16:22 | |
| jtomasek | jrist: do you need help with GUI status? | 16:23 |
| jrist | jtomasek: a short summary would be great. what patches you think we need | 16:23 |
| jtomasek | d0ugal, shardy: I think I can put together a list of non GUI patches that I see as an absolute requirement for RC1 GUI | 16:23 |
| jtomasek | jrist: ok, I can do that | 16:23 |
| *** yamahata has quit IRC | 16:23 | |
| *** zoli|mtg is now known as zoli|gone | 16:23 | |
| d0ugal | jtomasek: I think we also need a list of GUI patches that are needed. | 16:24 |
| *** zoli|gone is now known as zoli_gone-proxy | 16:24 | |
| jtomasek | d0ugal: yep, I'll put it together and send to ML | 16:25 |
| d0ugal | cool | 16:25 |
| d0ugal | I'll be sending the Mistral status tomorrow morning | 16:25 |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Handle image location based on CI_ENV and REL_TYPE https://review.openstack.org/359796 | 16:25 |
| jtomasek | d0ugal: thanks for that 'output' mistral patch! | 16:25 |
| *** dbecker has joined #tripleo | 16:25 | |
| jtomasek | d0ugal: I am so glad it went in... now we need to make sure it is included in tripleo RC1 mistral | 16:26 |
| d0ugal | jtomasek: no worries, I'm glad it was welcome upstream. I thought there would be come debating. | 16:26 |
| *** akshai has quit IRC | 16:26 | |
| d0ugal | s/come/some/ | 16:26 |
| *** dtantsur is now known as dtantsur|afk | 16:28 | |
| *** akshai has joined #tripleo | 16:29 | |
| *** jlinkes has quit IRC | 16:31 | |
| derekh_afk | sshnaidm: yup, about half way through deleting all the ports, | 16:34 |
| derekh_afk | sshnaidm: all but one of the instances are gone | 16:34 |
| derekh_afk | sshnaidm: then I'll delete all the subnets and networks | 16:35 |
| derekh_afk | sshnaidm: and finially we can delete the stacks | 16:35 |
| derekh_afk | sshnaidm: at least that how I hope it will go.... | 16:35 |
| *** derekh_afk is now known as derekh | 16:35 | |
| *** nyechiel_ has quit IRC | 16:38 | |
| *** fragatina has joined #tripleo | 16:42 | |
| *** fragatina has quit IRC | 16:43 | |
| *** lucasagomes is now known as lucas-dinner | 16:44 | |
| *** fragatina has joined #tripleo | 16:44 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Override allNodes redis_vip with correct RedisVirtualIP value https://review.openstack.org/365791 | 16:47 |
| *** weshay is now known as weshay_bbiab | 16:47 | |
| *** pradk has quit IRC | 16:47 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Remove *ExtraConfig parameters from overcloud.yaml https://review.openstack.org/365792 | 16:48 |
| *** fragatina has quit IRC | 16:52 | |
| *** jcoufal_ has joined #tripleo | 16:58 | |
| *** hogepodge has quit IRC | 16:58 | |
| *** jpich has quit IRC | 16:59 | |
| *** trown is now known as trown|lunch | 17:00 | |
| *** anshul has joined #tripleo | 17:00 | |
| *** jcoufal has quit IRC | 17:01 | |
| *** fzdarsky is now known as fzdarsky|afk | 17:01 | |
| *** hogepodge has joined #tripleo | 17:03 | |
| *** bana_k has quit IRC | 17:03 | |
| *** florianf has quit IRC | 17:04 | |
| *** ohamada has quit IRC | 17:05 | |
| shardy | https://review.openstack.org/#/c/363188 needs a +A if anyone has a moment | 17:05 |
| *** yamahata has joined #tripleo | 17:06 | |
| *** florianf has joined #tripleo | 17:10 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/361501 | 17:12 |
| *** pradk has joined #tripleo | 17:16 | |
| *** lblanchard has quit IRC | 17:18 | |
| *** bana_k has joined #tripleo | 17:20 | |
| therve | shardy, Mitaka branch, no. Mitaka release, very possible | 17:21 |
| shardy | therve: Ok, I think we've gone with the delete-the-world approach now, but if there's any specific patches you can identify we could check and perhaps confirm it's not an unfixed problem | 17:22 |
| *** pkovar has quit IRC | 17:24 | |
| therve | shardy, Nevermind, I was thinking of liberty | 17:25 |
| therve | Mitaka ought to be good | 17:25 |
| *** lblanchard has joined #tripleo | 17:27 | |
| shardy | therve: Ok, thanks, something of a mystery then, unfortunately we didn't have a chance to properly debug it | 17:28 |
| *** akshai has quit IRC | 17:30 | |
| *** fragatina has joined #tripleo | 17:32 | |
| *** fragatina has quit IRC | 17:32 | |
| *** fragatina has joined #tripleo | 17:33 | |
| shardy | Is there a fix for the "Duplicate entry for Environment: ['name']]" error on plan creation? | 17:34 |
| shardy | d0ugal: ^^ perhaps you know if you're still around | 17:34 |
| *** paramite has quit IRC | 17:36 | |
| *** jaosorior has quit IRC | 17:36 | |
| *** florianf has quit IRC | 17:40 | |
| openstackgerrit | Merged openstack/tripleo-common: Better namespace the list of TripleO actions https://review.openstack.org/357682 | 17:43 |
| *** chem has quit IRC | 17:49 | |
| *** chem has joined #tripleo | 17:49 | |
| *** dbecker has quit IRC | 17:51 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-common: Wire in jinja templating for custom roles https://review.openstack.org/362465 | 17:53 |
| *** anshul has quit IRC | 17:53 | |
| *** fragatina has quit IRC | 17:54 | |
| *** florianf has joined #tripleo | 17:54 | |
| *** fragatina has joined #tripleo | 17:55 | |
| *** akshai has joined #tripleo | 17:57 | |
| *** crinkle_ is now known as crinkle | 18:07 | |
| *** athomas has quit IRC | 18:07 | |
| bnemec | shardy: therve: I don't think we wiped the heat db, so it's still going to need cleaning. | 18:07 |
| bnemec | And even if we didn't, we'd still need to fix the problem because we can't have the whole cloud blowing up once a month. :-) | 18:07 |
| bnemec | *even if we did | 18:08 |
| shardy | bnemec: sure, we just haven't done enough debugging yet to identify the problem | 18:08 |
| shardy | I was just hoping we'd be missing some recent fix, sounds like we're not | 18:08 |
| *** trown|lunch is now known as trown | 18:12 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Add missing reference to ExternalPort for DVR https://review.openstack.org/363188 | 18:13 |
| derekh | bnemec: still deleting resources btw | 18:14 |
| derekh | bnemec: I guess you are too cause I've noticed my loops are trying to delete things that are already delete | 18:14 |
| bnemec | derekh: Yeah, I still see a bunch of delete failures in the heat logs though. :-( | 18:15 |
| bnemec | derekh: No, I'm actually not. Maybe sshnaidm is? | 18:15 |
| sshnaidm | derekh, bnemec yeah, I'm trying do delete stacks too | 18:15 |
| derekh | bnemec: I havn't even gone near heat yet, I've deleted all ports am nearly finished deleting subnets and nets | 18:15 |
| derekh | bnemec: sshnaidm and I've delete all nova instances | 18:15 |
| sshnaidm | derekh, yeah, I saw, only ports and subnets are left | 18:16 |
| sshnaidm | derekh, actually subnets and networks, I see ports are already done | 18:16 |
| derekh | sshnaidm: yup, ports are all deleted, subnets nearly finished | 18:17 |
| *** pradk has quit IRC | 18:18 | |
| sshnaidm | about 40 stacks remain | 18:18 |
| openstackgerrit | Steven Hardy proposed openstack/python-tripleoclient: Allow and render overcloud.j2.yaml template https://review.openstack.org/365735 | 18:21 |
| *** fragatin_ has joined #tripleo | 18:22 | |
| sshnaidm | maybe it's better to have rh2 cloud working, even with this ntp issue | 18:23 |
| sshnaidm | it's just 10 minutes of job, not critical at all, and it could support and back up CI | 18:23 |
| *** fragatina has quit IRC | 18:25 | |
| *** shardy has quit IRC | 18:27 | |
| *** rhallisey has quit IRC | 18:28 | |
| sshnaidm | I still see in logs "One or more ports have an IP allocation from this subnet", although no ports at all, wtf | 18:29 |
| sshnaidm | maybe it's neutron which database we should purge | 18:30 |
| *** rhallisey has joined #tripleo | 18:30 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Add constraint to prohibit balance-tcp from BondInterfaceOvsOptions https://review.openstack.org/355073 | 18:31 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Update OpenContrail loadbalancer plugin value https://review.openstack.org/366215 | 18:31 |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient: Generate Keystone credentials for overcloud https://review.openstack.org/366287 | 18:35 |
| *** pradk has joined #tripleo | 18:35 | |
| bnemec | sshnaidm: Actually, I think the rh2 ntp problem is fixed. | 18:38 |
| bnemec | We need to make sure the flavors in rh2 are all updated since we changed some of them. | 18:39 |
| bnemec | But other than that I think we could turn it back on. | 18:39 |
| sshnaidm | bnemec, they are not afaik | 18:39 |
| bnemec | I have a patch up to start using it again: https://review.openstack.org/353163 | 18:39 |
| sshnaidm | bnemec, should I delete the old flavor and create a new one, right? | 18:41 |
| bnemec | sshnaidm: I just updated both of them. They should be right now. | 18:41 |
| bnemec | sshnaidm: But yes, that's what you would do. Or you use Horizon like I did and it does it for you. :-) | 18:42 |
| bnemec | I'm going to drop WIP on the re-enable patch. | 18:42 |
| sshnaidm | bnemec, undercloud should be with 4 cpus | 18:43 |
| bnemec | sshnaidm: Whoops, fixed. | 18:43 |
| sshnaidm | bnemec, great, so maybe we'll enable rh2 when we are fixing rh1 | 18:44 |
| bnemec | sshnaidm: Yeah, I dropped WIP on the infra patch to re-enable it. | 18:45 |
| bnemec | pabelanger: ^It would be good to get rh2 running again while we address the issues in rh1. | 18:46 |
| pabelanger | bnemec: I wanted to finish the slip between ovb / nodepool. Clean up any existing issues before moving it back into production. | 18:49 |
| *** weshay_bbiab is now known as weshay | 18:51 | |
| *** mwhahaha has quit IRC | 18:53 | |
| *** mwhahaha has joined #tripleo | 18:54 | |
| sshnaidm | bnemec, pabelanger maybe let's continue here | 19:00 |
| sshnaidm | <pabelanger> sshnaidm: Which is causing warnings to appear in nodepool.o.o. If you create a new project, openstackzuul, which shared network access to the openstack-nodepool project, we'd update nodepool.o.o to use that | 19:00 |
| sshnaidm | derekh, bnemec, can we do this now? I don't know about possible problems it may appear | 19:01 |
| pabelanger | I also updated the review | 19:01 |
| *** tzumainn has quit IRC | 19:03 | |
| sshnaidm | 2016-09-06 19:01:41.909 5804 ERROR heat.engine.stack [-] Unexpected exception in delete | 19:04 |
| sshnaidm | very informative | 19:04 |
| *** jprovazn has quit IRC | 19:05 | |
| sshnaidm | derekh, stackw are done, ooff | 19:05 |
| sshnaidm | bnemec, what have we decided about restart of server? are you gonna tune it? | 19:05 |
| derekh | sshnaidm: ya, I'm creating a env now to see how it goes | 19:07 |
| bnemec | sshnaidm: I think we had decided not to unless it's necessary. | 19:07 |
| sshnaidm | derekh, restarting services there | 19:07 |
| bnemec | If we do end up restarting then yes, I'll change the tuning on the system. | 19:07 |
| derekh | sshnaidm: restarting what? | 19:07 |
| bnemec | It should be a safe operation. If anything, I expect it to help withour current issues. | 19:07 |
| bnemec | *with our | 19:08 |
| sshnaidm | derekh, sorry, nova , neutron and heat | 19:08 |
| sshnaidm | derekh, done | 19:08 |
| *** fzdarsky|afk is now known as fzdarsky | 19:08 | |
| *** tzumainn has joined #tripleo | 19:09 | |
| *** akshai has quit IRC | 19:09 | |
| derekh | sshnaidm: ok, gonna create a couple of test envs now to see how they go | 19:10 |
| sshnaidm | derekh, at least delete worked :) | 19:10 |
| derekh | sshnaidm: bnemec btw this is how I'm testing it on the te-broker | 19:11 |
| derekh | [root@te-broker openstack-virtual-baremetal]# /opt/stack/tripleo-ci/scripts/te-broker/create-env derekh1 5 ce28ba92-5060-4301-9727-4d99933b8687 | 19:11 |
| derekh | ce28ba92-5060-4301-9727-4d99933b8687 is the uuid of a test instance I created | 19:11 |
| *** pcaruana has quit IRC | 19:11 | |
| sshnaidm | I see it does good | 19:12 |
| *** antipsychiatry has joined #tripleo | 19:17 | |
| *** fragatin_ has quit IRC | 19:17 | |
| *** fragatina has joined #tripleo | 19:18 | |
| beagles | hey guys, is there a good diagram that illustrates how network isolation is hooked up? | 19:18 |
| derekh | sshnaidm: | baremetal_derekh1 | 5ccdc577-31cf-40fe-ae15-cc1dcc016d67 | Stack CREATE completed successfully | CREATE_COMPLETE | 2016-09-06T19:15:15 | | 19:18 |
| derekh | sshnaidm: deleting it again, then I think we can try re-enabling the te-broker | 19:19 |
| sshnaidm | derekh, so..let's try broker? | 19:19 |
| sshnaidm | derekh, ok | 19:19 |
| derekh | sshnaidm: fire ahead | 19:19 |
| *** pradk- has joined #tripleo | 19:20 | |
| sshnaidm | derekh, started | 19:20 |
| *** pradk has quit IRC | 19:21 | |
| *** pradk- is now known as pradk | 19:22 | |
| sshnaidm | derekh, running a job, let's see.. | 19:24 |
| *** tesseract- has quit IRC | 19:25 | |
| *** akshai has joined #tripleo | 19:25 | |
| *** tzumainn has quit IRC | 19:27 | |
| derekh | sshnaidm: actually we should stop it for a minute again and restart geard very quickly | 19:27 |
| derekh | will explain why in a second | 19:28 |
| sshnaidm | ok, sure | 19:28 |
| * bnemec predicts a lot of pain spinning up 50 testenvs at once | 19:28 | |
| bnemec | Although I guess the queue is empty, so maybe not. | 19:29 |
| derekh | sshnaidm: bassically geard has a bunch of jobs queued for ci tests that were killed long ago | 19:29 |
| derekh | sshnaidm: restarting it will clear that queue | 19:29 |
| sshnaidm | derekh, aa, that's why it was creating so many envs | 19:30 |
| *** mcornea has joined #tripleo | 19:30 | |
| derekh | sshnaidm: yup | 19:30 |
| derekh | sshnaidm: it would have went all the way up to 80 | 19:30 |
| derekh | sshnaidm: I'm deleting the ones it created and we can start again | 19:30 |
| sshnaidm | derekh, ok | 19:30 |
| derekh | bnemec: ya, the zuul queue is pretty much empty so once we restart there shouldn't be a big rush | 19:31 |
| *** coolsvap_ has quit IRC | 19:32 | |
| *** mbound has joined #tripleo | 19:32 | |
| sshnaidm | derekh, do you delete them with "heat stack-delete"? | 19:34 |
| derekh | /opt/stack/tripleo-ci/scripts/te-broker/destroy-env <envnum> | 19:34 |
| derekh | sshnaidm: I started using that script but am also doing heat stack deletes to speed things up | 19:35 |
| sshnaidm | derekh, can you use "destroy-env"? | 19:36 |
| derekh | sshnaidm: yup | 19:38 |
| derekh | /opt/st.... test sshnaidm do you see this ? | 19:38 |
| *** rcernin has joined #tripleo | 19:41 | |
| * derekh notes people don't see his message when they start with a "/" | 19:41 | |
| derekh | <derekh> /opt/stack/tripleo-ci/scripts/te-broker/destroy-env <envnum> | 19:42 |
| derekh | <derekh> sshnaidm: I started using that script but am also doing heat stack deletes to speed things up | 19:42 |
| derekh | <derekh> /opt/st.... test sshnaidm do you see this ? | 19:42 |
| openstackgerrit | Merged openstack/tripleo-quickstart: Handle image location based on CI_ENV and REL_TYPE https://review.openstack.org/359796 | 19:43 |
| *** rbowen is now known as DrBacchus | 19:44 | |
| bnemec | Actually I did see the ones that started with / | 19:45 |
| derekh | bnemec: ok, looks like I jumped to the wrong conclusion | 19:47 |
| *** weshay is now known as weshay_mtg | 19:47 | |
| bnemec | I do run into that occasionally though. | 19:47 |
| derekh | so many ports take ages to delete :-( | 19:47 |
| bnemec | Particularly when I try to /rant. :-) | 19:48 |
| derekh | lol | 19:48 |
| *** rcernin has quit IRC | 19:48 | |
| bnemec | Yeah, and the apis in this cloud are just _slow_. | 19:49 |
| *** pradk has quit IRC | 19:49 | |
| bnemec | I'll be very curious to see if that's still the case once we fix the performance profile. | 19:49 |
| sshnaidm | /me can see | 19:49 |
| bnemec | It seems like the default cpu scaling doesn't scale fast enough to handle our workload properly. | 19:50 |
| *** akuznetsov has joined #tripleo | 19:52 | |
| *** akuznetsov has quit IRC | 19:55 | |
| derekh | sshnaidm: te_broker and geard started up again, rechecking something | 19:56 |
| *** antipsychiatry has quit IRC | 19:57 | |
| bnemec | \o/ finally got rid of the 1292 instance that has been stuck deleting for about a month. | 19:57 |
| *** akuznetsov has joined #tripleo | 19:58 | |
| *** davidlenwell has quit IRC | 19:59 | |
| *** florianf has quit IRC | 20:00 | |
| derekh | bnemec: did you do anything special beside keep trying? I tried a force-delete earlier but it didn't seem to work | 20:01 |
| bnemec | derekh: I did a nova reset-state on it first, then tried regular delete again. | 20:01 |
| derekh | ack | 20:02 |
| bnemec | I figured we needed to clear it out since we'll be resetting the testenv counter here. :-) | 20:03 |
| bnemec | derekh: So are we back up and running? | 20:04 |
| bnemec | Ah, I see testenvs creating. We must be. | 20:04 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Don't send metrics to graphite in failed jobs https://review.openstack.org/365114 | 20:05 |
| bnemec | Here goes nothing. :-) | 20:05 |
| derekh | bnemec: yup, fingers crossed we're back in business | 20:11 |
| *** akuznetsov has quit IRC | 20:11 | |
| *** davidlenwell has joined #tripleo | 20:12 | |
| *** ChanServ sets mode: +v davidlenwell | 20:12 | |
| derekh | First job got an env | 20:13 |
| *** jayg is now known as jayg|g0n3 | 20:14 | |
| derekh | and the second | 20:14 |
| *** jpena|away is now known as jpena|off | 20:15 | |
| bnemec | six more just went create complete | 20:18 |
| bnemec | Although I have to admit 6.5 minutes to create one of these simple stacks seems excessive. | 20:19 |
| bnemec | KeyError: 'provision-1' | 20:26 |
| bnemec | Hmm, not good. | 20:26 |
| bnemec | It claimed that the heat stack completed, but apparently it didn't. | 20:27 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-common: Wire in jinja templating for custom roles https://review.openstack.org/362465 | 20:28 |
| bnemec | Yeah, we've still got problems. Three jobs have already failed on problems setting up the testenv. | 20:30 |
| derekh | bnemec: stop it with the good news | 20:30 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Change NeutronWorkers default to result in previous behavior https://review.openstack.org/364483 | 20:31 |
| derekh | bnemec: I think is does seem better but not good enough, wanna stop it and do the heat db delete or controller reboot so? | 20:32 |
| *** radeks has quit IRC | 20:34 | |
| bnemec | derekh: I dunno. Heat says it created all the things successfully. Maybe there's a race between when neutron says the port is created, nova says the port is attached, and when it actually happens. | 20:38 |
| bnemec | I'm trolling logs to try to figure out what might have happened. | 20:38 |
| *** ayoung has quit IRC | 20:41 | |
| derekh | and the other one failed on a different error, | 20:41 |
| *** Goneri has quit IRC | 20:41 | |
| bnemec | Yeah, I see a 404 in there too. | 20:43 |
| bnemec | Which interestingly is an error I saw on Friday as well when trying to list the instances from the command-line. | 20:43 |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient: Generate Keystone credentials for overcloud https://review.openstack.org/366287 | 20:44 |
| *** pradk has joined #tripleo | 20:44 | |
| bnemec | But why would a simple nova list 404? | 20:44 |
| * bnemec shrugs | 20:44 | |
| derekh | 2016-09-06 20:17:02.059 19033 INFO nova.osapi_compute.wsgi.server [req-34de622f-2a14-421d-b223-6a58cb730e0b ba119eef29ce49f5b8697f4d63948e3c b79291658f384b7ebbc9019b6349e5c9 - - -] 192.168.112.44 "GET /v2.1/b79291658f384b7ebbc9019b6349e5c9/servers/ HTTP/1.1" status: 404 len: 240 time: 1.5273101 | 20:45 |
| derekh | it 404'd alright | 20:45 |
| *** weshay_mtg is now known as weshay | 20:47 | |
| EmilienM | dprince: please review https://review.openstack.org/#/c/366240/ | 20:47 |
| EmilienM | dprince: requirement for Newton | 20:47 |
| EmilienM | dprince: /me working on overcloud now | 20:47 |
| *** bfournie has quit IRC | 20:48 | |
| dprince | EmilienM: looking | 20:48 |
| EmilienM | thx | 20:48 |
| dprince | EmilienM: keystone makes me sad | 20:49 |
| EmilienM | dprince: and me? | 20:49 |
| dprince | EmilienM: so much backwards incompatability | 20:49 |
| EmilienM | i'm super sad | 20:49 |
| EmilienM | that is terrible | 20:49 |
| EmilienM | and this is not finished | 20:49 |
| EmilienM | the overcloud part is crazy | 20:49 |
| dprince | EmilienM: thanks for looking into these | 20:50 |
| EmilienM | we have no choice | 20:50 |
| EmilienM | we can't bump keystone otherwise | 20:50 |
| *** tzumainn has joined #tripleo | 20:50 | |
| EmilienM | weshay, dmsimard: where is /etc/gnocchi ? https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-promote-master-delorean-minimal_pacemaker-109/overcloud-controller-0/etc/ | 20:51 |
| EmilienM | or maybe you don't install it? | 20:51 |
| EmilienM | damn wrong windo | 20:51 |
| *** fzdarsky has quit IRC | 20:53 | |
| trown | EmilienM: should we recheck on https://review.openstack.org/#/c/366240/ or is OVB jobs known issue? | 20:53 |
| EmilienM | ovb jobs are down | 20:53 |
| EmilienM | my patch pass the undercloud-multinode patch, we can go ahead IMHO | 20:54 |
| EmilienM | but we can also wait, no opinion | 20:54 |
| trown | k, if that somehow passed multinode and it is actually broken, then the multinode job is very broken | 20:55 |
| trown | so lets go dor it | 20:55 |
| trown | for it even | 20:55 |
| *** oneswig has joined #tripleo | 20:55 | |
| derekh | bnemec: rabbit is using a lot of RAM aswell, I wonder if its worth a restart /me is kind of just trying random things now | 20:56 |
| EmilienM | trown: thanks | 20:57 |
| EmilienM | trown: we need that work to be done so we can unpin keystone | 20:57 |
| trown | yep | 20:57 |
| * trown is a bit confused about how it is ok to pin keystone in the first place... but gets that it is crunch time | 20:58 | |
| bnemec | derekh: Worth a shot. | 20:58 |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient: Generate Keystone credentials for overcloud https://review.openstack.org/366287 | 20:59 |
| EmilienM | trown: FYI that is my work for overcloud ^ | 21:00 |
| EmilienM | trown: highly WIP now. | 21:00 |
| EmilienM | trown: but I expect to finish it tonight | 21:00 |
| trown | EmilienM: k I will check it out tomorrow | 21:00 |
| derekh | bnemec: done | 21:02 |
| *** trown is now known as trown|outtypewww | 21:02 | |
| derekh | bnemec: sshnaidm I gotta run, will pick up on this tomorrow, | 21:02 |
| bnemec | derekh: Okay, have a good one. | 21:03 |
| derekh | bnemec: let us know if you find anything, or if you want to to try the reboot send intructions about the cpu scaling thing (or if you decide to do it yourself that fine by me) | 21:04 |
| bnemec | derekh: Yeah, I think I'll hold off for now. I don't see any more env failures since those first few. | 21:04 |
| derekh | bnemec: ok, ttyl | 21:05 |
| *** derekh has quit IRC | 21:05 | |
| *** lblanchard has quit IRC | 21:09 | |
| *** rhallisey has quit IRC | 21:12 | |
| *** dprince has quit IRC | 21:16 | |
| openstackgerrit | Merged openstack/diskimage-builder: Revert "Revert "Pre-install pip/virtualenv packages"" https://review.openstack.org/341934 | 21:21 |
| *** mcornea has quit IRC | 21:23 | |
| *** jcoufal_ has quit IRC | 21:29 | |
| *** rhallisey has joined #tripleo | 21:34 | |
| *** mbound has quit IRC | 21:40 | |
| *** pradk has quit IRC | 21:43 | |
| *** rhallisey has quit IRC | 22:00 | |
| *** oneswig has quit IRC | 22:02 | |
| *** cdearborn has quit IRC | 22:08 | |
| *** Goneri has joined #tripleo | 22:10 | |
| openstackgerrit | Merged openstack/instack-undercloud: keystone: configure credentials https://review.openstack.org/366240 | 22:11 |
| *** jeckersb_gone is now known as jeckersb | 22:17 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Configure Keystone credentials https://review.openstack.org/366400 | 22:26 |
| EmilienM | ok this is the overcloud version ^ | 22:26 |
| stevebaker | bnemec: so ah, what do you think about deleting all the elements in tripleo-image-elements which are not used? | 22:27 |
| *** Goneri has quit IRC | 22:29 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Expose parameter to enable combination alarms https://review.openstack.org/363748 | 22:30 |
| *** jeckersb is now known as jeckersb_gone | 22:31 | |
| EmilienM | I would need some review on https://review.openstack.org/#/c/364916/ https://review.openstack.org/#/c/366128/ https://review.openstack.org/#/c/364917/ and https://review.openstack.org/#/c/365117/ | 22:34 |
| EmilienM | it will fix Aodh and Gnocchi in TripleO | 22:34 |
| EmilienM | weshay: ^ | 22:34 |
| EmilienM | gate-tripleo-ci-centos-7-scenario001-multinode-nv is now green | 22:34 |
| EmilienM | which means Telemetry is working ! | 22:35 |
| *** rajinir has joined #tripleo | 22:36 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-image-elements: Make os-collect-config element package driven https://review.openstack.org/366403 | 22:37 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-image-elements: Make os-apply-config package driven https://review.openstack.org/366404 | 22:37 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-image-elements: Make os-refresh-config element package driven https://review.openstack.org/366405 | 22:37 |
| EmilienM | trown|outtypewww: ^ | 22:38 |
| stevebaker | EmilienM: do you know if this makes Aodh use keystone v3? https://review.openstack.org/#/c/365117/ | 22:38 |
| EmilienM | probably | 22:39 |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient: Replace agent elements with package python-heat-agent-puppet https://review.openstack.org/366406 | 22:39 |
| EmilienM | stevebaker: i'm not sure if we need to set a auth_plugin specific to keystone v3 | 22:39 |
| EmilienM | we use 'password' everywhere | 22:39 |
| *** yamahata has quit IRC | 22:40 | |
| stevebaker | EmilienM: we have a heat trusts Aodh bug which this might help with https://bugzilla.redhat.com/show_bug.cgi?id=1367138#c16 | 22:41 |
| openstack | bugzilla.redhat.com bug 1367138 in openstack-heat "Autoscaling with trust notifier doesn't work" [Unspecified,New] - Assigned to sbaker | 22:41 |
| EmilienM | stevebaker: we set auth_type = password I confirm | 22:43 |
| EmilienM | does it matter? | 22:43 |
| stevebaker | EmilienM: in heat? | 22:44 |
| openstackgerrit | Steven Hardy proposed openstack/python-tripleoclient: Get template contents from plan, not local path https://review.openstack.org/365735 | 22:45 |
| EmilienM | stevebaker: in aodh | 22:45 |
| stevebaker | EmilienM: ah, I don't actually know. I've not looked into that bug yet. | 22:47 |
| *** yamahata has joined #tripleo | 22:49 | |
| *** akshai has quit IRC | 22:53 | |
| *** rlandy is now known as rlandy|bbl | 22:55 | |
| *** lblanchard has joined #tripleo | 23:01 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient: Add libffi-dev to bindep.txt https://review.openstack.org/366413 | 23:01 |
| EmilienM | pabelanger: ^ | 23:01 |
| pabelanger | EmilienM: looks right | 23:02 |
| EmilienM | k | 23:02 |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient: Generate Keystone credentials for overcloud https://review.openstack.org/366287 | 23:03 |
| *** ayoung has joined #tripleo | 23:10 | |
| *** dbecker has joined #tripleo | 23:20 | |
| *** akshai has joined #tripleo | 23:22 | |
| *** maeca2 has joined #tripleo | 23:23 | |
| *** maeca1 has quit IRC | 23:25 | |
| openstackgerrit | Merged openstack/diskimage-builder: fix systemd resource deadlock https://review.openstack.org/365797 | 23:33 |
| *** akshai has quit IRC | 23:33 | |
| EmilienM | I am going to self approve https://review.openstack.org/#/c/366413/ because CI is broken for tripleoclient | 23:38 |
| EmilienM | except if anyone around can look | 23:39 |
| EmilienM | bnemec, thrash ^ | 23:39 |
| thrash | EmilienM: looking | 23:49 |
| EmilienM | thrash: see my comment | 23:49 |
| EmilienM | is it ok? | 23:49 |
| thrash | EmilienM: lgtm | 23:49 |
| EmilienM | thx | 23:50 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-image-elements: Remove unused elements https://review.openstack.org/366417 | 23:51 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!