| *** achadha_ has joined #tripleo | 00:02 | |
| *** achadha has quit IRC | 00:05 | |
| *** achadha_ has quit IRC | 00:06 | |
| pabelanger | stevebaker: mwhahaha: ya, check-tripleo-jobs template needs to be add for tripleo-quickstart-extras | 00:07 |
|---|---|---|
| *** hexo_ is now known as kokot | 00:08 | |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 00:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Ian Main (imain) | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712098 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 00:10 |
| openstack | Launchpad bug 1712098 in tripleo "Periodic master tempest job with 107 failures" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 00:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 00:10 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add Ceilometer API and Collector service to roles_data https://review.openstack.org/494589 | 00:10 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates master: Make rolling update interactive. https://review.openstack.org/496449 | 00:16 |
| *** thrash is now known as thrash|g0ne | 00:40 | |
| *** achadha has joined #tripleo | 00:40 | |
| *** limao has joined #tripleo | 00:44 | |
| *** yamahata has quit IRC | 00:47 | |
| *** achadha has quit IRC | 00:50 | |
| *** achadha has joined #tripleo | 00:51 | |
| *** itlinux has joined #tripleo | 00:51 | |
| *** achadha has quit IRC | 00:52 | |
| *** achadha has joined #tripleo | 00:53 | |
| *** achadha has quit IRC | 00:54 | |
| *** achadha has joined #tripleo | 00:55 | |
| *** fpan has quit IRC | 00:55 | |
| *** cshastri has joined #tripleo | 00:57 | |
| *** dixiaoli has joined #tripleo | 00:59 | |
| *** achadha has quit IRC | 00:59 | |
| *** funzo has joined #tripleo | 01:04 | |
| *** itlinux has quit IRC | 01:04 | |
| openstackgerrit | Paul Belanger proposed openstack/tripleo-quickstart master: Clean up repo_cmd_before command for setup-repo https://review.openstack.org/496085 | 01:05 |
| *** fzdarsky_ has joined #tripleo | 01:06 | |
| *** funzo has quit IRC | 01:08 | |
| *** fzdarsky has quit IRC | 01:09 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712098 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 01:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Ian Main (imain) | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 01:10 |
| openstack | Launchpad bug 1712098 in tripleo "Periodic master tempest job with 107 failures" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 01:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 01:10 |
| *** achadha has joined #tripleo | 01:10 | |
| *** achadha has quit IRC | 01:11 | |
| *** achadha has joined #tripleo | 01:12 | |
| *** achadha has quit IRC | 01:16 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Accept multiple registries in DockerInsecureRegistryAddress https://review.openstack.org/492613 | 01:20 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates master: WIP container ovs-agent, ensure br-ex exists https://review.openstack.org/493695 | 01:28 |
| *** itlinux has joined #tripleo | 01:29 | |
| openstackgerrit | Merged openstack/tripleo-common master: Add Barbican Config Image to overcloud containers https://review.openstack.org/493493 | 01:32 |
| openstackgerrit | Merged openstack/tripleo-common master: Configure mistral event engine container https://review.openstack.org/495966 | 01:35 |
| openstackgerrit | Merged openstack/tripleo-common master: Add Qdrouterd to the overcloud containers https://review.openstack.org/491889 | 01:35 |
| *** itlinux has quit IRC | 01:35 | |
| *** achadha has joined #tripleo | 01:40 | |
| *** itlinux has joined #tripleo | 01:43 | |
| *** achadha_ has joined #tripleo | 01:43 | |
| *** achadha has quit IRC | 01:44 | |
| *** ecerquei has quit IRC | 01:47 | |
| *** achadha_ has quit IRC | 01:48 | |
| *** bkopilov has quit IRC | 01:48 | |
| *** ramishra has joined #tripleo | 01:48 | |
| *** achadha has joined #tripleo | 01:53 | |
| *** rbowen has quit IRC | 01:54 | |
| *** itlinux has quit IRC | 01:56 | |
| *** ramishra has quit IRC | 02:02 | |
| *** ramishra has joined #tripleo | 02:05 | |
| *** itlinux has joined #tripleo | 02:06 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 02:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712098 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 02:10 |
| openstack | Launchpad bug 1712098 in tripleo "Periodic master tempest job with 107 failures" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 02:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 02:10 |
| *** dixiaoli has quit IRC | 02:15 | |
| *** tzumainn has joined #tripleo | 02:15 | |
| *** dixiaoli has joined #tripleo | 02:19 | |
| *** cdearborn_ has quit IRC | 02:19 | |
| *** mdnadeem has joined #tripleo | 02:21 | |
| *** itlinux has quit IRC | 02:21 | |
| *** achadha has quit IRC | 02:28 | |
| *** weshay is now known as weshay_PTO_AM | 02:37 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates master: WIP container ovs-agent, ensure br-ex exists https://review.openstack.org/493695 | 02:48 |
| *** itlinux has joined #tripleo | 02:50 | |
| *** itlinux has quit IRC | 02:57 | |
| *** bkopilov has joined #tripleo | 03:04 | |
| *** funzo has joined #tripleo | 03:04 | |
| *** funzo has quit IRC | 03:09 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712098 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 03:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 03:10 |
| openstack | Launchpad bug 1712098 in tripleo "Periodic master tempest job with 107 failures" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 03:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 03:10 |
| *** itlinux has joined #tripleo | 03:12 | |
| *** itlinux has quit IRC | 03:14 | |
| *** itlinux has joined #tripleo | 03:17 | |
| *** itlinux has quit IRC | 03:30 | |
| *** tzumainn has quit IRC | 03:30 | |
| *** gbarros has joined #tripleo | 03:31 | |
| *** gkadam has joined #tripleo | 03:40 | |
| *** limao has quit IRC | 03:42 | |
| *** udesale has joined #tripleo | 03:43 | |
| *** limao has joined #tripleo | 03:43 | |
| *** rlandy has quit IRC | 03:45 | |
| *** psahoo has joined #tripleo | 03:47 | |
| *** leitan has joined #tripleo | 03:47 | |
| *** jobcespedes has quit IRC | 03:57 | |
| *** links has joined #tripleo | 03:57 | |
| *** funzo has joined #tripleo | 04:05 | |
| *** funzo has quit IRC | 04:10 | |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 04:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712098 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 04:10 |
| openstack | Launchpad bug 1712098 in tripleo "Periodic master tempest job with 107 failures" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 04:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 04:10 |
| *** shreshtha has joined #tripleo | 04:17 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Revert "Filter image list by deployed services" https://review.openstack.org/496445 | 04:17 |
| *** ykarel has joined #tripleo | 04:17 | |
| *** shreshtha_ has joined #tripleo | 04:20 | |
| *** shreshtha has quit IRC | 04:22 | |
| *** achadha has joined #tripleo | 04:23 | |
| *** mdnadeem has quit IRC | 04:28 | |
| *** itlinux has joined #tripleo | 04:30 | |
| *** itlinux has quit IRC | 04:34 | |
| *** fpan has joined #tripleo | 04:43 | |
| *** nyechiel has joined #tripleo | 04:44 | |
| *** dsariel has joined #tripleo | 04:47 | |
| *** pgadiya has joined #tripleo | 04:50 | |
| *** mdnadeem has joined #tripleo | 04:52 | |
| *** nyechiel has quit IRC | 04:56 | |
| *** pgadiya has quit IRC | 04:58 | |
| *** leitan has quit IRC | 05:01 | |
| *** jtomasek_ has joined #tripleo | 05:01 | |
| *** jtomasek has quit IRC | 05:02 | |
| *** sdake has quit IRC | 05:02 | |
| *** jpena|off has quit IRC | 05:02 | |
| *** afazekas has quit IRC | 05:02 | |
| *** sdoran has quit IRC | 05:02 | |
| *** kokot has quit IRC | 05:02 | |
| *** zul has quit IRC | 05:02 | |
| *** Hazelesque has quit IRC | 05:02 | |
| *** hexo_ has joined #tripleo | 05:02 | |
| *** Hazelesque has joined #tripleo | 05:02 | |
| *** hexo_ is now known as hexo | 05:02 | |
| *** afazekas has joined #tripleo | 05:02 | |
| *** sdake has joined #tripleo | 05:03 | |
| *** sdake has quit IRC | 05:03 | |
| *** sdake has joined #tripleo | 05:03 | |
| *** rbowen has joined #tripleo | 05:03 | |
| *** jpena|off has joined #tripleo | 05:04 | |
| *** radez has quit IRC | 05:06 | |
| *** trown|outtypewww has quit IRC | 05:06 | |
| *** rook has quit IRC | 05:06 | |
| *** trown has joined #tripleo | 05:06 | |
| *** itlinux has joined #tripleo | 05:07 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712098 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 05:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 05:10 |
| openstack | Launchpad bug 1712098 in tripleo "Periodic master tempest job with 107 failures" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 05:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 05:10 |
| *** tvignaud has quit IRC | 05:10 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Docker: Enable TLS in the internal network for libvirt https://review.openstack.org/496351 | 05:11 |
| *** radez has joined #tripleo | 05:11 | |
| *** rook has joined #tripleo | 05:12 | |
| *** rook is now known as Guest91464 | 05:12 | |
| *** itlinux has quit IRC | 05:19 | |
| *** yprokule has joined #tripleo | 05:19 | |
| *** dpawar has joined #tripleo | 05:20 | |
| *** aditya_r has joined #tripleo | 05:22 | |
| *** tvignaud has joined #tripleo | 05:22 | |
| *** ratailor has joined #tripleo | 05:25 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Configure listen_address for libvirtd when TLS is enabled https://review.openstack.org/496492 | 05:25 |
| *** tvignaud has quit IRC | 05:26 | |
| *** tvignaud has joined #tripleo | 05:28 | |
| *** aufi has joined #tripleo | 05:30 | |
| *** masco has joined #tripleo | 05:32 | |
| *** fpan has quit IRC | 05:33 | |
| *** skramaja has joined #tripleo | 05:35 | |
| *** chem has quit IRC | 05:40 | |
| *** aditya_ra has joined #tripleo | 05:43 | |
| *** aditya_r has quit IRC | 05:44 | |
| *** iranzo has joined #tripleo | 05:47 | |
| *** iranzo has quit IRC | 05:47 | |
| *** iranzo has joined #tripleo | 05:47 | |
| *** pdeore has joined #tripleo | 05:48 | |
| *** jfrancoa has joined #tripleo | 05:52 | |
| *** dmacpher has joined #tripleo | 05:55 | |
| *** achadha has quit IRC | 05:58 | |
| *** achadha has joined #tripleo | 05:58 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/472607 | 06:00 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo master: Certmonger: Only attempt to reload haproxy is it's active https://review.openstack.org/496501 | 06:03 |
| kristaps_ | morning guys | 06:05 |
| *** funzo has joined #tripleo | 06:06 | |
| kristaps_ | i have issue with glance images, when i upload it size is 0. i show logs to #glance folks and they said " it looks like in the log that the image is being uploaded to swift, but rbd:// is ceph" but actually there is no ceph | 06:08 |
| kristaps_ | any ideas? | 06:08 |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 06:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712098 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 06:10 |
| openstack | Launchpad bug 1712098 in tripleo "Periodic master tempest job with 107 failures" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 06:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 06:10 |
| *** funzo has quit IRC | 06:10 | |
| *** prateek has joined #tripleo | 06:12 | |
| *** prateek has left #tripleo | 06:12 | |
| jaosorior | kristaps_: I suggest you wait a couple of hours for more folks to log in. And a dude whose handle is gfidente might be able to help with that. | 06:12 |
| *** shardy has joined #tripleo | 06:13 | |
| kristaps_ | jaosorior: okey thank you | 06:13 |
| kristaps_ | jaosorior: maybe any quick ideas? | 06:15 |
| *** shrjoshi has joined #tripleo | 06:15 | |
| *** shreshtha_ has quit IRC | 06:16 | |
| *** rcernin has joined #tripleo | 06:16 | |
| jaosorior | kristaps_: I'm not very acquainted with that area to be honest. Sorry. | 06:16 |
| kristaps_ | jaosorior: but there is no special settings whoich i could configure before undercloud deployment for glance? | 06:17 |
| *** dparkes has joined #tripleo | 06:19 | |
| openstackgerrit | Steven Hardy proposed openstack/os-net-config master: This patch adds initial support for the Contrail vRouter interface https://review.openstack.org/492492 | 06:20 |
| *** agurenko has joined #tripleo | 06:23 | |
| jaosorior | kristaps_: the undercloud shouldn't use ceph at all, which is why it's pretty confusing that you're getting that. | 06:27 |
| *** ffiore has joined #tripleo | 06:30 | |
| *** ykarel_ has joined #tripleo | 06:30 | |
| jaosorior | shardy, bandini: Could you check these out if you ahve time https://review.openstack.org/#/c/496345/ https://review.openstack.org/#/c/496349/ | 06:30 |
| kristaps_ | jaosorior: can i just re-run undercloud installation if there no existing overcloud? | 06:32 |
| bandini | jaosorior: on it | 06:32 |
| jaosorior | kristaps_: yep | 06:32 |
| *** ykarel has quit IRC | 06:33 | |
| *** ramishra has quit IRC | 06:35 | |
| *** ramishra has joined #tripleo | 06:37 | |
| *** anshul has joined #tripleo | 06:38 | |
| jaosorior | bandini: thanks! | 06:38 |
| bandini | ;) | 06:38 |
| jaosorior | mandre: let me know when you're around | 06:43 |
| *** bogdando has joined #tripleo | 06:45 | |
| *** achadha_ has joined #tripleo | 06:45 | |
| mandre | jaosorior: good morning | 06:46 |
| jaosorior | mandre: sup dude! how's it going? | 06:47 |
| *** achadha has quit IRC | 06:48 | |
| jaosorior | mandre: have you gone forward with the redis SSL work? I just finished the libvirtd TLS bits. So we could pair that if you want. | 06:48 |
| mandre | jaosorior: so the deployment i kicked off last night failed on config file generation which is weird because my patch didn't touch anything around that area, trying to debug now | 06:49 |
| *** limao has quit IRC | 06:49 | |
| jaosorior | mandre: regarding the haproxy check. Apparently we can enable ssl for the tcp-check https://www.haproxy.com/doc/aloha/7.0/haproxy/healthchecks.html#checking-any-service so we're covered on that area. | 06:49 |
| jaosorior | just a matter of enabling the config option "tcp-check connect ssl" if TLS everywhere is used. | 06:50 |
| mandre | jaosorior: sure, do you want me to post the WIP patch so you can have a peek already? | 06:50 |
| jaosorior | mandre: sure! that would be great | 06:50 |
| jaosorior | mandre: I'm gonna go for a quick jog. But if you want, we can starting pairing this after that. | 06:51 |
| *** dmacpher has quit IRC | 06:51 | |
| openstackgerrit | Martin André proposed openstack/tripleo-common master: Add apache to redis image for tls https://review.openstack.org/496519 | 06:54 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: TLS proxy for redis https://review.openstack.org/496520 | 06:54 |
| *** cschwede_ has joined #tripleo | 06:55 | |
| *** limao has joined #tripleo | 06:55 | |
| jaosorior | mandre: we would need to add the TLS proxy code in puppet-tripleo. but that should be fairly fast. | 06:56 |
| mandre | jaosorior: yep, let's sync after you're back and I had my breakfast | 06:56 |
| jaosorior | mandre: ack | 06:57 |
| *** shardy has quit IRC | 06:58 | |
| *** shardy has joined #tripleo | 06:59 | |
| *** sdoran has joined #tripleo | 07:00 | |
| bandini | anyone around for a one-liner review https://review.openstack.org/#/c/495965/ ? | 07:00 |
| *** zul has joined #tripleo | 07:00 | |
| *** jlinkes has joined #tripleo | 07:02 | |
| *** jpich has joined #tripleo | 07:04 | |
| *** ebarrera has joined #tripleo | 07:04 | |
| skramaja | shardy: even after taking the reduced set of images, a big list of DockerXXXImages are not used - http://logs.openstack.org/00/495300/3/check/gate-tripleo-ci-centos-7-containers-multinode/1604f4f/logs/undercloud/home/jenkins/overcloud_deploy.log.txt.gz#_2017-08-21_13_36_04 | 07:05 |
| *** achadha_ has quit IRC | 07:05 | |
| skramaja | shardy: i still feel that filterint out these parameters will enhance the useability. WDYT | 07:05 |
| shardy | skramaja: Hmm, why are those parameters defined if the services aren't enabled in the job? | 07:08 |
| shardy | I thought stevebaker's patch would stop that - are we still waiting on a quickstart patch to use it perhaps? | 07:08 |
| * shardy looks | 07:08 | |
| *** dtrainor has quit IRC | 07:09 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 07:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712098 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 07:10 |
| openstack | Launchpad bug 1712098 in tripleo "Periodic master tempest job with 107 failures" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 07:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 07:10 |
| shardy | skramaja: it was reverted see https://review.openstack.org/#/c/496445/ | 07:10 |
| kristaps_ | guys, anyone has any idea, why when upload images to glance i got size 0, and in log see Cannot determine image size because it is either not provided in the request or chunked-transfer encoding is used. Adding image as a segmented object to Swift | 07:10 |
| *** florianf has joined #tripleo | 07:10 | |
| openstackgerrit | Merged openstack/tripleo-validations master: Do not output empty variables in inventory https://review.openstack.org/495965 | 07:10 |
| openstackgerrit | Merged openstack/tripleo-validations master: Make sure to keep case when for host definition in host inventory. https://review.openstack.org/495975 | 07:10 |
| skramaja | shardy: ohh.. ok. | 07:10 |
| skramaja | shardy: probably i will run it locally and see it once before we push it. | 07:11 |
| shardy | skramaja: if possible, I'd like to leave the warnings for all parameters, at least for now, as we know the cost of having many unused images is quite high | 07:11 |
| *** ykarel__ has joined #tripleo | 07:11 | |
| shardy | e.g even if we don't pull them on deploy, we pull them during the prepare/upload step which wastes a bunch of time and disk space | 07:11 |
| *** pcaruana has joined #tripleo | 07:11 | |
| shardy | so it's probably good to warn the user about it? | 07:12 |
| *** ffiore has quit IRC | 07:12 | |
| skramaja | shardy: ok. if we have a way to avoid it, the it is OK to warn, but docker.yaml environment is default and we dont want user to override (mostly) | 07:13 |
| skramaja | shardy: let me try it locally and will provide feedback. | 07:13 |
| *** ykarel_ has quit IRC | 07:13 | |
| shardy | skramaja: ack sounds good, thanks! | 07:14 |
| *** jprovazn has joined #tripleo | 07:15 | |
| *** tesseract-RH has joined #tripleo | 07:17 | |
| *** oidgar has joined #tripleo | 07:20 | |
| *** dixiaoli_ has joined #tripleo | 07:29 | |
| *** jpena|off is now known as jpena | 07:29 | |
| shardy | skramaja: hey https://review.openstack.org/#/c/450264 passed CI now, thanks for the review - are you happy to approve it? | 07:29 |
| shardy | I've rechecked the next two patches in the series so hopefully we can land them later | 07:29 |
| skramaja | sure | 07:29 |
| shardy | thanks! | 07:29 |
| *** oidgar has quit IRC | 07:30 | |
| kristaps_ | guys, anyone has any idea, why when upload images to glance i got size 0, and in log see Cannot determine image size because it is either not provided in the request or chunked-transfer encoding is used. Adding image as a segmented object to Swift | 07:31 |
| *** hjensas has joined #tripleo | 07:31 | |
| *** hjensas has quit IRC | 07:31 | |
| *** hjensas has joined #tripleo | 07:31 | |
| *** daidv has joined #tripleo | 07:32 | |
| *** dixiaoli has quit IRC | 07:32 | |
| *** dtrainor has joined #tripleo | 07:32 | |
| shardy | kristaps_: Hi, not seen that error myself, but providing a paste of the logs, and the full command used to deploy (is this overcloud or undercloud) may help figure it out | 07:33 |
| shardy | sounds like something is misconfigured, but it's hard to say exactly what without more info | 07:33 |
| *** ykarel__ is now known as ykarel | 07:34 | |
| *** hjensas has quit IRC | 07:39 | |
| kristaps_ | shardy: im talking about overcloud, after openstack overcloud image upload i see in glance that are images are size 0 http://paste.openstack.org/raw/619072/ , if i grep one of the images in glance logs i get http://paste.openstack.org/raw/619132/ | 07:39 |
| kristaps_ | shardy: sorry not overcloud but undercloud | 07:39 |
| openstackgerrit | Flavio Percoco proposed openstack/tripleo-heat-templates master: WIP: Deploy OpenShift using OOO on the overcloud https://review.openstack.org/494470 | 07:40 |
| kristaps_ | shardy: before uploading images not empty ofcourse )) | 07:41 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Move deprecated SchedulerHints logic to overcloud.j2.yaml https://review.openstack.org/495324 | 07:41 |
| shardy | kristaps_: Ok, have you made any changes to the default undercloud.conf? | 07:42 |
| openstackgerrit | Martin Kopec proposed openstack/tripleo-quickstart-extras master: Allow removing of options from tempest conf https://review.openstack.org/477079 | 07:42 |
| shardy | and are the swift services all running OK? | 07:42 |
| kristaps_ | shardy: yesterday in glance irc i get answer : that looks a bit odd ... it looks like in the log that the image is being uploaded to swift, but rbd:// is ceph | 07:42 |
| shardy | kristaps_: can you paste your glance.conf please? | 07:43 |
| kristaps_ | shardy: only enable ssl, add my NIC and hostname | 07:43 |
| shardy | sorry glance-api.conf and glance-swift.conf | 07:44 |
| shardy | where do you see the rbd:// ? | 07:45 |
| *** suuuper has joined #tripleo | 07:46 | |
| *** amoralej|off is now known as amoralej | 07:46 | |
| *** hjensas has joined #tripleo | 07:46 | |
| *** hjensas has quit IRC | 07:46 | |
| *** hjensas has joined #tripleo | 07:46 | |
| *** agurenko has quit IRC | 07:47 | |
| *** gbarros has quit IRC | 07:48 | |
| *** athomas has joined #tripleo | 07:49 | |
| *** cylopez has joined #tripleo | 07:53 | |
| kristaps_ | shardy: glancec-swift.conf http://paste.openstack.org/raw/619134/ | 07:53 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Separate config_volume for ringbuilder https://review.openstack.org/494008 | 07:57 |
| kristaps_ | shardy: can i send you glance-api.cong as i file? | 07:59 |
| *** egonzalez has joined #tripleo | 08:00 | |
| *** agurenko has joined #tripleo | 08:00 | |
| shardy | kristaps_: yes, or just tell me where you see rbd:// ? | 08:00 |
| shardy | the glance-api.conf should have a line like stores=glance.store.filesystem.Store,glance.store.swift.Store | 08:01 |
| shardy | and default_store=swift | 08:01 |
| shardy | kristaps_: I assume you've got plenty of disk space? | 08:01 |
| *** openstackgerrit has quit IRC | 08:03 | |
| kristaps_ | shardy: it was glance folk comment about one one the log, i can try to find it | 08:03 |
| *** suuuper has quit IRC | 08:04 | |
| kristaps_ | shardy: there is no prbolems with space | 08:05 |
| *** suuuper has joined #tripleo | 08:05 | |
| shardy | kristaps_: does swift work if you do "openstack container create test" ? | 08:05 |
| *** funzo has joined #tripleo | 08:07 | |
| kristaps_ | shardy: stack@undercloud glance]$ openstack container create test | 08:07 |
| kristaps_ | +---------------------------------------+-----------+------------------------------------+ | 08:07 |
| kristaps_ | | account | container | x-trans-id | | 08:07 |
| kristaps_ | +---------------------------------------+-----------+------------------------------------+ | 08:07 |
| kristaps_ | | AUTH_e1270b1e77f34774935ab237b134e47a | test | tx3d7fdf4048594abab55e9-00599d378e | | 08:07 |
| shardy | you could then do e.g openstack object create test overcloud-full.qcow2 | 08:07 |
| kristaps_ | +---------------------------------------+-----------+------------------------------------+ | 08:07 |
| *** derekh has joined #tripleo | 08:07 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 08:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712098 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 08:10 |
| kristaps_ | shardy: done http://paste.openstack.org/raw/619139/ | 08:10 |
| openstack | Launchpad bug 1712098 in tripleo "Periodic master tempest job with 107 failures" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 08:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 08:10 |
| shardy | Ok so swift is working fine, so the problem must be the glance config | 08:10 |
| *** funzo has quit IRC | 08:11 | |
| *** hjensas has quit IRC | 08:11 | |
| *** suuuper has quit IRC | 08:11 | |
| *** hjensas has joined #tripleo | 08:14 | |
| *** hjensas has quit IRC | 08:14 | |
| *** hjensas has joined #tripleo | 08:14 | |
| *** suuuper has joined #tripleo | 08:15 | |
| kristaps_ | shardy: share to you glance-api.conf? | 08:18 |
| *** lucas-afk is now known as lucasagomes | 08:18 | |
| *** paramite has joined #tripleo | 08:20 | |
| * shardy cries | 08:22 | |
| shardy | kristaps_: yes, please | 08:24 |
| *** openstackgerrit has joined #tripleo | 08:24 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: Add -s (silent) to curl command for CRL refresh https://review.openstack.org/496345 | 08:24 |
| kristaps_ | shardy: did you receive? | 08:25 |
| *** kristaps_ has quit IRC | 08:25 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-heat-templates master: Specify the start count to 0 for the update step loop https://review.openstack.org/496397 | 08:26 |
| *** lvdombrkr has joined #tripleo | 08:27 | |
| lvdombrkr | shardy: did you receive it i send you via irc client | 08:28 |
| shardy | lvdombrkr: you PM'd me a checksum | 08:28 |
| *** mdnadeem has quit IRC | 08:32 | |
| *** mdnadeem has joined #tripleo | 08:32 | |
| *** ffiore has joined #tripleo | 08:35 | |
| lvdombrkr | shardy: https://files.fm/u/83b3sp8y | 08:37 |
| *** mcornea has joined #tripleo | 08:38 | |
| lvdombrkr | shardy: get it? | 08:44 |
| shardy | lvdombrkr: thanks, sorry I can't see any problem with the file, I'm still confused about the rbd:// comment | 08:44 |
| * shardy looks around for a glance expert | 08:45 | |
| shardy | flaper87: ^^ maybe you could sanity check the glance config above? | 08:45 |
| flaper87 | shardy: checking | 08:46 |
| shardy | thanks! | 08:46 |
| *** hjensas has quit IRC | 08:47 | |
| *** aramteke has joined #tripleo | 08:48 | |
| flaper87 | shardy: lvdombrkr looks correct | 08:50 |
| * flaper87 reads the backlog to get more context | 08:51 | |
| *** oidgar has joined #tripleo | 08:51 | |
| *** rwsu has joined #tripleo | 08:51 | |
| lvdombrkr | flaper87: in few words issue is that i have 0 size images in glance | 08:51 |
| *** aditya_ra has quit IRC | 08:51 | |
| shardy | http://paste.openstack.org/raw/619132/ is the glance log | 08:52 |
| * flaper87 checks glance's logs | 08:52 | |
| flaper87 | 2017-08-22 21:10:27.945 5176 DEBUG glance_store._drivers.swift.store [req-6150da2d-a2aa-4e83-8919-41e693992fe5 3646e98ad7c5458bac33d44c10950fe9 e1270b1e77f34774935ab237b134e47a - default default] Not writing zero-length chunk. add /usr/lib/python2.7/site-packages/glance_store/_drivers/swift/store.py:911 | 08:53 |
| flaper87 | it looks like something is failing on glance side. I would first check if the data is actually getting to glance | 08:54 |
| flaper87 | Make sure the upload request is actually sending data | 08:54 |
| lvdombrkr | flaper87: how can i check it? | 08:54 |
| flaper87 | I used to do this putting a breakpoint somewhere in glance and verifying the data was being sent correctly | 08:54 |
| flaper87 | From the logs, it looks like the request is being sent chunked | 08:55 |
| lvdombrkr | flaper87: can you give more instructions how to check it, its no clear for me | 08:57 |
| *** salmankhan has joined #tripleo | 08:57 | |
| lvdombrkr | flaper87: any ideas why its hapening? | 08:58 |
| lvdombrkr | flaper87: what i know that finally this upladed image is corrupt cause i cant boot from it.. thats was reason why i start investigation | 08:59 |
| *** hjensas has joined #tripleo | 09:00 | |
| *** hjensas has quit IRC | 09:00 | |
| *** hjensas has joined #tripleo | 09:00 | |
| flaper87 | lvdombrkr: have you tried uploading new images manually? | 09:01 |
| *** hjensas has quit IRC | 09:01 | |
| lvdombrkr | flaper87: yes, same result | 09:02 |
| flaper87 | you could start from there. IF that results in 0-sized images, then you can proceed to debug using breakpoints | 09:02 |
| flaper87 | mmh | 09:02 |
| flaper87 | ok | 09:02 |
| flaper87 | is that using v2 or v1? | 09:02 |
| flaper87 | it seemed v2 from the logs | 09:02 |
| lvdombrkr | flaper87: yes v2 i suggest | 09:03 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud master: Undercloud/Certmonger: Only attempt to reload haproxy is it's active https://review.openstack.org/496564 | 09:04 |
| *** ebarrera has quit IRC | 09:04 | |
| *** tosky has joined #tripleo | 09:04 | |
| openstackgerrit | David Sariel proposed openstack/tripleo-validations master: Add containers_sanity_check https://review.openstack.org/483403 | 09:06 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud master: Undercloud/Certmonger: Only attempt to reload haproxy is it's active https://review.openstack.org/496564 | 09:06 |
| *** ebarrera has joined #tripleo | 09:06 | |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 09:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712098 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 09:10 |
| openstack | Launchpad bug 1712098 in tripleo "Periodic master tempest job with 107 failures" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 09:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 09:10 |
| openstackgerrit | Alfredo Moralejo proposed openstack/tripleo-heat-templates stable/newton: Make get-occ-config.sh support custom roles https://review.openstack.org/496566 | 09:10 |
| openstackgerrit | Martin André proposed openstack/tripleo-common master: Add apache to redis image for tls https://review.openstack.org/496519 | 09:13 |
| *** sdake has quit IRC | 09:15 | |
| *** sdake has joined #tripleo | 09:16 | |
| *** bkopilov has quit IRC | 09:16 | |
| *** sdake is now known as Guest28943 | 09:16 | |
| openstackgerrit | Merged openstack/tripleo-common master: Download and run upgrade/deploy_steps_playbooks for upgrade https://review.openstack.org/490847 | 09:17 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-ui master: Imported Translations from Zanata https://review.openstack.org/496568 | 09:17 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Write user parameters environment to swift https://review.openstack.org/450264 | 09:21 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo master: HAProxy: Make certmonger bundle the cert and key on renewal https://review.openstack.org/496572 | 09:22 |
| *** aramteke has quit IRC | 09:23 | |
| oidgar | guys, remind me, what is the new way to create the docker local registry for overcloud images? | 09:23 |
| jaosorior | adarazs: any idea what's wrong with the ovb jobs today? They've been failing a lot today | 09:23 |
| adarazs | jaosorior: no, let me take a look | 09:24 |
| *** limao has quit IRC | 09:26 | |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: [WIP] TLS proxy for redis https://review.openstack.org/496520 | 09:26 |
| shardy | oidgar: https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-prep-containers/templates/overcloud-prep-containers.sh.j2#L18 shows the commands used | 09:32 |
| oidgar | shardy: thanks! | 09:33 |
| adarazs | jaosorior: looks like the "no more mirrors to try" -- rpm download issues :/ | 09:37 |
| adarazs | sshnaidm|off: do you know what's going on? | 09:38 |
| *** ffiore has quit IRC | 09:41 | |
| *** ffiore_ has joined #tripleo | 09:41 | |
| *** sshnaidm|off is now known as sshnaidm | 09:42 | |
| *** adarazs is now known as adarazs_lunch | 09:42 | |
| sshnaidm | adarazs_lunch, looking.. | 09:42 |
| jaosorior | mandre: hey dude | 09:46 |
| *** gfidente has joined #tripleo | 09:48 | |
| *** gfidente has quit IRC | 09:48 | |
| *** gfidente has joined #tripleo | 09:48 | |
| lvdombrkr | shardy: i checked right now my first undercloud installation (on other HW) and there is also 0 size images | 09:50 |
| lvdombrkr | that means, i do wrong steps | 09:50 |
| lvdombrkr | in installation | 09:50 |
| *** dixiaoli_ has quit IRC | 09:52 | |
| mandre | hey jaosorior, I'm looking at the puppet-tripleo part now | 09:53 |
| jaosorior | bandini: | 09:54 |
| mandre | jaosorior: I'm trying to replicate the changes made to other services, not exactly sure what I'm doing | 09:54 |
| jaosorior | bandini: when redis is managed by pacemaker, is it binding to the VIP or the regular IP ? | 09:54 |
| jaosorior | mandre: wanna pair it in bluejeans? | 09:54 |
| lvdombrkr | shardy: im deploying undercloud by this documentation : https://docs.openstack.org/tripleo-docs/latest/install/installation/installation.html | 09:55 |
| *** dpawar has quit IRC | 09:55 | |
| lvdombrkr | version Ocata | 09:56 |
| lvdombrkr | wihout CEPH | 09:56 |
| lvdombrkr | without puppet | 09:56 |
| mandre | jaosorior: maybe a bit later, it's about lunch time here | 09:57 |
| jaosorior | mandre: ok, let me know when you're back | 09:57 |
| mandre | jaosorior: what port should the tls proxy be listening on for redis? | 09:57 |
| lvdombrkr | shardy: can you confirm that i can falow this documetation? | 09:57 |
| lvdombrkr | shardy: i try to understood what goes wrong | 09:58 |
| *** ykarel_ has joined #tripleo | 09:58 | |
| jaosorior | mandre: the same as redis. then the actual redis server will listen on localhost instead. | 09:59 |
| bandini | jaosorior: haproxy binds to the redis_vip and then forwards the requests to haproxy which binds to the host ip (hope I got the question right?) | 09:59 |
| shardy | lvdombrkr: yes the documentation should work - do you have selinux enforcing? | 09:59 |
| lvdombrkr | shardy: its permessive | 10:00 |
| *** ykarel has quit IRC | 10:01 | |
| jaosorior | bandini: you did. Thanks! | 10:01 |
| bandini | jaosorior: the second 'haproxy' in my sentence above should read 'redis' (sorry about that) | 10:02 |
| jaosorior | bandini: haha I understood :P | 10:02 |
| bandini | :) | 10:03 |
| jaosorior | bandini: are you acquainted with the redis heartbeat resource agent? | 10:03 |
| bandini | jaosorior: very little. but I have to start looking at it because I might need to change some stuff around. what's up? | 10:03 |
| jaosorior | bandini: I'm taking a read. researching how to do TLS on redis. And I was wondering if there are some changes needed on how to access it if it's behind a TLS proxy. | 10:05 |
| jaosorior | bandini: now, from the code https://github.com/ClusterLabs/resource-agents/blob/master/heartbeat/redis I can see that there are some commands ran with the redis cli client... I don't know yet if we'll need some changes there or not y et | 10:06 |
| bandini | jaosorior: ack yeah, I presume the redis stanza in haproxy.cfg will need a bit of love as well | 10:07 |
| shardy | rajinir: Hi, can we mark https://blueprints.launchpad.net/tripleo/+spec/dellemc-unity-cinder implemented now the patches on the whiteboard merged? | 10:07 |
| *** funzo has joined #tripleo | 10:07 | |
| shardy | pike-rc1 is planned for tomorrow, so we need to update the status of all the blueprints | 10:08 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Set DNS for rh1 cloud before installation https://review.openstack.org/496596 | 10:08 |
| jaosorior | bandini: it does. I think I got the answer for that one though | 10:08 |
| bandini | oh nice | 10:08 |
| jaosorior | bandini: there is are tcp-check statements in the haproxy config, that send commands to redis. it would be a matter of adding 'tcp-check connect ssl' and it would do the same via TLS | 10:09 |
| bandini | jaosorior: ah simple enough, nice | 10:09 |
| *** ykarel__ has joined #tripleo | 10:10 | |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712098 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 10:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 10:10 |
| openstack | Launchpad bug 1712098 in tripleo "Periodic master tempest job with 107 failures" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 10:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 10:10 |
| jaosorior | bandini: problem is, the blog posts that I find, while they require a TLS proxy to be in front of redis (which is not a problem) they also require a TLS proxy in front of the client, which is strange :/ | 10:11 |
| *** funzo has quit IRC | 10:12 | |
| sshnaidm | jaosorior, adarazs_lunch, seems like problem with DNS in infra causes OVB failures, trying to find somebody to check it.. | 10:12 |
| bandini | jaosorior: oh wow yeah that sounds odd | 10:12 |
| *** ykarel_ has quit IRC | 10:12 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Move deprecated SchedulerHints logic to overcloud.j2.yaml https://review.openstack.org/495324 | 10:14 |
| jaosorior | bandini: do you know how the redis client reads the redis connection? | 10:15 |
| *** pblaho has joined #tripleo | 10:16 | |
| bandini | nope never looked at that stuff close enough | 10:17 |
| *** hjensas has joined #tripleo | 10:19 | |
| *** hjensas has quit IRC | 10:19 | |
| *** hjensas has joined #tripleo | 10:19 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Move deprecated SchedulerHints logic to overcloud.j2.yaml https://review.openstack.org/495324 | 10:24 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Fix hardcoded references to deprecated *ExtraConfig parameters https://review.openstack.org/496604 | 10:24 |
| *** milan has joined #tripleo | 10:25 | |
| lvdombrkr | shardy: that glance issue is can be related that i enable wrong repositories? | 10:26 |
| jaosorior | bandini: this doesn't look good :( https://github.com/antirez/redis/issues/2178 | 10:28 |
| jaosorior | mandre: ^^ | 10:28 |
| shardy | lvdombrkr: it could be, hard to be sure without figuring out the root cause of your problem | 10:29 |
| shardy | lvdombrkr: you could remove the repo files you added to /etc/yum.repos.d, then run this script: | 10:29 |
| shardy | https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/tripleo.sh | 10:29 |
| *** ykarel__ is now known as ykarel | 10:29 | |
| shardy | export STABLE_RELEASE=ocata | 10:29 |
| shardy | tripleo.sh --repo-setup | 10:29 |
| shardy | and compare with the repos you configured manually | 10:30 |
| shardy | If they differ we may have an error in the docs | 10:30 |
| *** dtantsur|afk is now known as dtantsur | 10:30 | |
| jprovazn | hi, I'm hitting an ssl verification issue - [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579) when deploying overcloud with ceph-ansible, does this sound familiar? http://paste.openstack.org/show/619155/ | 10:30 |
| *** udesale has quit IRC | 10:31 | |
| *** skramaja has quit IRC | 10:33 | |
| jaosorior | bandini: is the pacemaker agent running in the same node as the redis instance? | 10:37 |
| lvdombrkr | shardy: this script will configure correct repositories and after i can continue configuration step by step? | 10:37 |
| jaosorior | jprovazn: it sure does. do the overcloud nodes trust the certifiacte? | 10:38 |
| jaosorior | *certificate | 10:38 |
| jprovazn | jaosorior: from the error message I would say that not | 10:38 |
| jaosorior | jprovazn: that message could be many things (openssl doesn't give the best error messages) | 10:39 |
| *** Guest91464 is now known as rook | 10:39 | |
| jaosorior | jprovazn: 1) the node doesn't trust the cert 2) the node was accessed with a wrong hostname (needs to match the cert) 3) the cert is malformed | 10:40 |
| jaosorior | jprovazn: so lets go for #1 first. Did you pass the undercloud's CA certificate to the overcloud so it could trust it? | 10:40 |
| *** jkilpatr has quit IRC | 10:42 | |
| jaosorior | jprovazn: that would be done by passing the undercloud's CA certificate (not the undercloud's haproxy server cert) via the SSLRootCertificate parameter as described in SSLRootCertificate | 10:42 |
| *** cshastri has quit IRC | 10:42 | |
| jaosorior | * as described in http://tripleo.org/install/advanced_deployment/ssl.html#certificate-details | 10:43 |
| jprovazn | jaosorior: OC was deployed with the following env files (should be mostly what oooq uses), let me check SSLRootCertificate | 10:43 |
| jprovazn | http://paste.openstack.org/show/619156/ - is the deployment command | 10:43 |
| jaosorior | jprovazn: check /home/stack/enable-tls.yaml | 10:44 |
| jaosorior | jprovazn: sorry, wrong file. Check /home/stack/inject-trust-anchor.yaml | 10:44 |
| jprovazn | jaosorior: regarding certificate - I can't check now - I deleted the stack before your feedback, but iirc yesterday the issue was that CA authority was not trusted by the controller node | 10:44 |
| * jprovazn checks | 10:44 | |
| amoralej | sshnaidm, we are hitting dns issues for tripleo jobs in review.r.o | 10:45 |
| amoralej | it's using an internal dns, right? | 10:45 |
| jprovazn | jaosorior: in the /home/stack/inject-trust-anchor.yaml is SSLRootCertificate included | 10:45 |
| jaosorior | jprovazn: gimme a minute | 10:45 |
| *** _jbadiapa has joined #tripleo | 10:46 | |
| sshnaidm | amoralej, we have problems now, yes, which job are you talking? | 10:46 |
| *** adarazs_lunch is now known as adarazs | 10:46 | |
| jaosorior | sshnaidm: all ovb jobs are having dns issues it seems | 10:46 |
| *** akrivoka has joined #tripleo | 10:47 | |
| sshnaidm | jaosorior, yeah | 10:47 |
| amoralej | sshnaidm, https://logs.rdoproject.org/03/8303/3/check/rdoinfo-tripleo-master-testing-centos-7-multinode-1ctlr-featureset005-nv/Zb4e636fb44cb40b0bf261191b3ed9dba/devstack-gate-setup-host.txt | 10:47 |
| amoralej | we are having issues with google dns in other tenants in rdo-cloud, is it the same issue? | 10:48 |
| *** jbadiapa_ has quit IRC | 10:48 | |
| jaosorior | shardy: is heat-config-notify supposed to be hitting a public endpoint for swift? http://paste.openstack.org/show/619155/ | 10:49 |
| sshnaidm | amoralej, infra image has 127.0.0.1 as dns and unbound server witch pre-cached data | 10:49 |
| bandini | jaosorior: oh gasp ouch | 10:49 |
| sshnaidm | amoralej, and this seems like not working.. no idea what happened there | 10:49 |
| bandini | jaosorior: not necessarily, depends how you deployed it (composable HA, etc) | 10:49 |
| jaosorior | jprovazn: ok, it seems that quickstart using different CA files, thus it doesn't trust the one in the undercloud. For one, I don't think it should be hitting swift's public endpoint. but given that that's the situation now, lets get the overcloud to trust the undercloud's CA: First, lets copy /usr/share/openstack-tripleo-heat-templates/environments/inject-trust-anchor-hiera.yaml and lets fill that up with | 10:52 |
| jaosorior | the contents of '/etc/pki/ca-trust/source/anchors/cm-local-ca.pem' and use it in | 10:52 |
| jaosorior | * and we'll use it as described here http://tripleo.org/install/advanced_deployment/ssl.html#getting-the-overcloud-to-trust-cas | 10:52 |
| jaosorior | jprovazn: ^^ | 10:52 |
| jprovazn | jaosorior: ack, sec | 10:53 |
| dtantsur | mwhahaha: morning! do you think https://github.com/openstack/tripleo-common/commit/53b48a303053fbe5c7308ca2623db4d2c2156be5 is a backportable change? | 10:55 |
| *** cylopez has quit IRC | 10:55 | |
| dtantsur | the reason I ask is because 'openstack baremetal instackenv validate' is semi-broken now, and it should use these new workflows instead | 10:55 |
| dtantsur | I've got https://bugzilla.redhat.com/show_bug.cgi?id=1480699 about it not working with vbmc in ocata | 10:56 |
| openstack | bugzilla.redhat.com bug 1480699 in python-ironicclient "instackenv validate not supporting custom ports" [Unspecified,New] - Assigned to rhos-maint | 10:56 |
| jaosorior | mandre, bandini: Could you check this out https://review.openstack.org/#/c/496492/ ? | 10:56 |
| jaosorior | mandre, bandini: And this one https://review.openstack.org/#/c/496351/ | 10:57 |
| shardy | jaosorior: Hmm, I thought we defaulted to CFN_SIGNAL, so we should poll swift and signal heat-api-cfn | 10:57 |
| jprovazn | jaosorior: so should I copy key from /etc/pki/ca-trust/source/anchors/cm-local-ca.pem under first-ca-name's content? and leave second-ca-name there? | 10:58 |
| jaosorior | jprovazn: so, those values are dynamic, you could remove second-ca-name (since you're not using it) | 10:58 |
| jaosorior | jprovazn: and you could rename first-ca-name to undercloud-ca to make the name more descriptive | 10:59 |
| jaosorior | shardy: right, so from that paste it seems it's calling swift. However, it seems to be using swift's public endpoint for that. | 10:59 |
| jaosorior | shardy: thought it should be using the internal one. | 10:59 |
| shardy | jaosorior: No I think the public endpoint is expected, as heat doesn't expect instances to have access to internal networks | 11:00 |
| shardy | it's configurable in heat.conf tho if we need to use internal for TripleO reasons | 11:00 |
| *** atoth has quit IRC | 11:00 | |
| jprovazn | jaosorior: does this look sane? http://paste.openstack.org/show/619161/ | 11:00 |
| shardy | got to drop for an hour, bbiab | 11:00 |
| *** shardy has quit IRC | 11:01 | |
| jaosorior | jprovazn: oh, I see you used inject-trust-anchor.yaml . if you're planning to use that file for both the undercloud and the overcloud's CA files (which could in theory be the same, but it's not that way in quickstart) then you need to add the overcloud CA there too | 11:02 |
| *** dsariel is now known as dsariel|afk | 11:02 | |
| jaosorior | jprovazn: you'll find the overcloud's CA cert in /etc/pki/ca-trust/source/anchors/overcloud-cacert.pem | 11:03 |
| mhenkel | Hi All. After updating a 7.3/OSP11 undercloud to 7.4/OSP11 I get this: | 11:03 |
| mhenkel | Removing the current plan files | 11:03 |
| mhenkel | Uploading new plan files | 11:03 |
| mhenkel | Started Mistral Workflow tripleo.plan_management.v1.update_deployment_plan. Execution ID: c04784c4-c22e-48e8-84eb-9d9cf528185d | 11:03 |
| mhenkel | Plan updated | 11:03 |
| mhenkel | Deploying templates in the directory /tmp/tripleoclient-C16WLW/tripleo-heat-templates | 11:03 |
| mhenkel | Invalid input [name=tripleo.deployment.v1.deploy_plan, class=DirectWorkflowSpec, unexpected=[u'skip_deploy_identifier'] | 11:03 |
| mhenkel | any idea? I checked recent bugs but couldn't find anything | 11:03 |
| *** Tengu has joined #tripleo | 11:04 | |
| openstackgerrit | Martin André proposed openstack/puppet-tripleo master: Use TLS proxy for Redis' internal TLS https://review.openstack.org/496616 | 11:04 |
| jprovazn | jaosorior: aha, so I should have used both, e.g. like this? http://paste.openstack.org/show/619163/ | 11:04 |
| Tengu | hello! | 11:04 |
| jaosorior | jprovazn: that looks good | 11:05 |
| Tengu | I'm wanting to uupgrade a test infra from ocata to spike. Any advice? We're using a standard tripleO deploy, without QuickStart. | 11:05 |
| Tengu | I was pointed to http://tripleo.org/install/post_deployment/upgrade.html - but it concerns earlier releases, I don't know if the newton->ocata process may apply for ocata->pike. | 11:06 |
| *** florianf has quit IRC | 11:06 | |
| *** pgadiya has joined #tripleo | 11:06 | |
| jprovazn | jaosorior: ok, thanks, I added the second one into deploy command and redeployed | 11:06 |
| jaosorior | jprovazn: you could also move the contents of SSLRootCertificate and add a second field to the CAMap in inject-trust-anchor-hiera.yaml | 11:06 |
| jaosorior | jprovazn: that way you would keep track of only one file. | 11:06 |
| Tengu | also, I stumbled on https://github.com/openstack/tripleo-upgrade - but that seems to be redhat oriented (and won't work on centos) | 11:06 |
| jprovazn | jaosorior: I can try that on next re-deploy | 11:06 |
| jaosorior | jprovazn: sure | 11:07 |
| *** pgadiya has quit IRC | 11:07 | |
| *** jprovazn has quit IRC | 11:08 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 11:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712098 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 11:10 |
| openstack | Launchpad bug 1712098 in tripleo "Periodic master tempest job with 107 failures" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 11:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 11:10 |
| *** ffiore_ has quit IRC | 11:10 | |
| openstackgerrit | Merged openstack/os-net-config master: Delete controller for standalone OVS bridges https://review.openstack.org/495919 | 11:11 |
| *** salmankhan1 has joined #tripleo | 11:11 | |
| *** salmankhan has quit IRC | 11:11 | |
| *** salmankhan1 is now known as salmankhan | 11:11 | |
| openstackgerrit | Merged openstack/tripleo-ui master: Imported Translations from Zanata https://review.openstack.org/496568 | 11:13 |
| *** salmankhan has quit IRC | 11:13 | |
| openstackgerrit | Arundhati Surpur proposed openstack/tripleo-docs master: Update .gitignore https://review.openstack.org/496623 | 11:16 |
| *** salmankhan has joined #tripleo | 11:17 | |
| *** pdeore has quit IRC | 11:18 | |
| *** jkilpatr has joined #tripleo | 11:19 | |
| *** limao has joined #tripleo | 11:20 | |
| *** florianf has joined #tripleo | 11:21 | |
| *** pkovar has joined #tripleo | 11:22 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-quickstart-extras master: Make overcloud trust the undercloud's CA certificate https://review.openstack.org/496639 | 11:22 |
| *** limao_ has joined #tripleo | 11:27 | |
| *** limao has quit IRC | 11:27 | |
| *** bfournie has quit IRC | 11:30 | |
| *** salmankhan1 has joined #tripleo | 11:32 | |
| *** salmankhan1 has quit IRC | 11:32 | |
| *** salmankhan has quit IRC | 11:34 | |
| *** salmankhan has joined #tripleo | 11:34 | |
| chandankumar | sshnaidm: please have a look on this https://review.openstack.org/#/c/493030/ | 11:35 |
| *** salmankhan has quit IRC | 11:37 | |
| *** salmankhan has joined #tripleo | 11:37 | |
| *** salmankhan has quit IRC | 11:38 | |
| *** salmankhan has joined #tripleo | 11:38 | |
| *** salmankhan has quit IRC | 11:40 | |
| *** salmankhan has joined #tripleo | 11:40 | |
| *** salmankhan has quit IRC | 11:41 | |
| *** salmankhan1 has joined #tripleo | 11:41 | |
| *** dpawar has joined #tripleo | 11:42 | |
| *** salmankhan1 has quit IRC | 11:43 | |
| openstackgerrit | Martin André proposed openstack/puppet-tripleo master: [WIP] Use TLS proxy for Redis' internal TLS https://review.openstack.org/496616 | 11:44 |
| *** thrash|g0ne is now known as thrash | 11:50 | |
| *** ratailor has quit IRC | 11:51 | |
| *** _jbadiapa has quit IRC | 11:54 | |
| mandre | jaosorior: wdyt of https://review.openstack.org/496616? we still need to hook the needed hieradata in t-h-t for puppet/services/database/redis | 11:54 |
| jaosorior | uhm | 11:56 |
| jaosorior | mandre: so this is a tricky one. | 11:56 |
| mwhahaha | dtantsur: backport to what? newton? | 11:56 |
| *** shrjoshi has quit IRC | 11:57 | |
| jaosorior | mandre: | 11:57 |
| jaosorior | mandre: The clients are expecting to access Redis on the VIP | 11:57 |
| *** brault has quit IRC | 11:57 | |
| jaosorior | mandre: so we have two options, 1) we make haproxy serve TLS and use a certificate for the VIP 2) we get the redis' TLS proxy to serve the cert for the VIP | 11:58 |
| jaosorior | mandre: httpd usually serves certs with the fqdn for the node no the specific interface, not the VIP. So we would need to request a cert specifically for Redis. | 11:59 |
| *** atoth has joined #tripleo | 11:59 | |
| *** pradk has joined #tripleo | 12:00 | |
| mandre | jaosorior: ack, this is a bit more complex than my single line modification to haproxy.conf | 12:00 |
| jaosorior | mandre: so the haproxy configuration is more or less what you did, I commented on that. The issue is the cert (that will be used by the TLS proxy) | 12:01 |
| *** lucasagomes is now known as lucas-hungry | 12:01 | |
| *** ansmith_ has quit IRC | 12:01 | |
| jaosorior | mandre: And there's another tricky issue in this. And that's pacemaker. It uses redis-cli to access redis and check the status of the service. However, I'm not sure if the cli client detects TLS, and I think it doesn't, cause folks recommend setting up stunnel on the client side | 12:02 |
| *** bfournie has joined #tripleo | 12:02 | |
| jaosorior | mandre: so... we'll need to set stunnel (or a different apache-based TLS proxy for clients), on the pacemaker nodes as well | 12:02 |
| jaosorior | then we need to change all the references to redis (the URLs) to use the rediss:// protocol instead of redis:// | 12:02 |
| jaosorior | mandre: soo... yeah, giving a bit more research to this, it's definitely not straight forward as other services :/ | 12:03 |
| *** dprince has joined #tripleo | 12:04 | |
| *** jpena is now known as jpena|lunch | 12:04 | |
| * jaosorior curses redis | 12:04 | |
| pabelanger | mwhahaha: weshay_PTO_AM: EmilienM okay, OVB job seems to be hitting mirror.centos.org directly: http://logs.openstack.org/28/495328/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024/c7a9a98/console.html#_2017-08-23_11_24_22_406999 This looks like mirrors haven't been setup just yet | 12:05 |
| *** funzo has joined #tripleo | 12:05 | |
| mwhahaha | pabelanger: yea that's from the builder | 12:06 |
| mwhahaha | pabelanger: actually isnt' that the logs fetching, weird | 12:06 |
| *** limao_ has quit IRC | 12:07 | |
| *** limao has joined #tripleo | 12:07 | |
| pabelanger | mwhahaha: I think the issue is, toci_gate_test.sh is installing stuff before repo files have been rewritten | 12:07 |
| mwhahaha | pabelanger: actually that's the end of the run, something else went wrong in that job | 12:08 |
| mwhahaha | i think | 12:08 |
| mwhahaha | i'll take a look at it more indeptb in a bit | 12:08 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Move multinode overcloud user to multinode settings https://review.openstack.org/496675 | 12:08 |
| *** morazi has joined #tripleo | 12:09 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Release extra-args from multinode environment settings https://review.openstack.org/496676 | 12:09 |
| sshnaidm | numans, ^^ | 12:09 |
| *** funzo has quit IRC | 12:09 | |
| *** hewbrocca_afk is now known as hewbrocca | 12:10 | |
| sshnaidm | numans, I release extra-args for you, please depend on my changes | 12:10 |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 12:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 12:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 12:10 |
| *** _jbadiapa has joined #tripleo | 12:10 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/instack-undercloud master: Updated from global requirements https://review.openstack.org/492768 | 12:11 |
| *** pkovar has quit IRC | 12:11 | |
| *** sbrzozow has joined #tripleo | 12:11 | |
| *** pblaho1 has joined #tripleo | 12:12 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo master: HAProxy: Make certmonger bundle the cert and key on renewal https://review.openstack.org/496572 | 12:13 |
| *** ansmith has quit IRC | 12:13 | |
| *** ffiore_ has joined #tripleo | 12:16 | |
| *** brault has joined #tripleo | 12:17 | |
| *** abishop has joined #tripleo | 12:17 | |
| *** pblaho has quit IRC | 12:18 | |
| *** pblaho1 has quit IRC | 12:18 | |
| *** pblaho has joined #tripleo | 12:18 | |
| *** jcoufal has joined #tripleo | 12:19 | |
| d0ugal | mhenkel: DId you find a solution? | 12:21 |
| d0ugal | mhenkel: that error suggests your python-tripleoclient is newer than your tripleo-common | 12:22 |
| *** brault has quit IRC | 12:22 | |
| d0ugal | (or, maybe tripleo-common is updated, but the workflow have not been reloaded into Mistral) | 12:22 |
| *** brault has joined #tripleo | 12:22 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements https://review.openstack.org/487398 | 12:23 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements https://review.openstack.org/488148 | 12:23 |
| mhenkel | d0ugal: python-tripleoclient-6.2.0-1.el7ost.noarch openstack-tripleo-common-6.1.0-2.el7ost.noarch | 12:23 |
| mhenkel | so, yes, there is a version difference | 12:23 |
| d0ugal | mhenkel: They wouldn't have the same version numbers... | 12:24 |
| mhenkel | how can I tell? | 12:24 |
| mhenkel | booth come from the same repo | 12:24 |
| d0ugal | Interesting | 12:25 |
| mhenkel | rhel-7-server-openstack-11-rpms | 12:25 |
| d0ugal | Basically, you need to make sure you have both of these changes... | 12:25 |
| d0ugal | https://github.com/openstack/python-tripleoclient/commit/f3acb69ffce3c630df5b3051779a7af21b96af47 | 12:25 |
| d0ugal | https://github.com/openstack/tripleo-common/commit/9d158874301eaa9444811f2c21cc9d86c2eaad7a | 12:25 |
| d0ugal | The error suggests your have the first one, as it is passing the parameter, but the workflow doesn't understand it so you get an error | 12:26 |
| *** cylopez has joined #tripleo | 12:26 | |
| *** pkovar has joined #tripleo | 12:29 | |
| *** rlandy has joined #tripleo | 12:31 | |
| openstackgerrit | Julien Danjou proposed openstack/tripleo-heat-templates master: Fix Panko public port https://review.openstack.org/496689 | 12:32 |
| numans | sshnaidm, ack. i will rebase the patch accordingly | 12:32 |
| numans | thanks | 12:32 |
| mhenkel | d0ugal: you were right. I downgraded the python-tripleoclient to 6.1.0.6 and deployment started | 12:33 |
| sshnaidm | numans, you can just set "Depends-On" on my two patches, no need to rebase | 12:33 |
| d0ugal | mhenkel: great | 12:33 |
| numans | sshnaidm, ack. i guess i don't need this patch then - https://review.openstack.org/#/c/495258/ | 12:34 |
| mhenkel | so, it looks like that either the python-tripleoclient rpm was pushed too early to the rhel-7-server-openstack-11-rpms or somebody missed to push the tripleo-common rpm | 12:35 |
| *** ecerquei has joined #tripleo | 12:36 | |
| sshnaidm | numans, yep, just try to use extra args | 12:36 |
| *** psahoo has quit IRC | 12:37 | |
| mhenkel | d0ugal: I checked the repos online and it is indeed the case | 12:38 |
| dtantsur | mwhahaha: (sorry, was having coffee) Ocata. this is where we introduces vbmc, which requires custom ports | 12:39 |
| openstackgerrit | Flavio Percoco proposed openstack/tripleo-heat-templates master: WIP: Deploy OpenShift using OOO on the overcloud https://review.openstack.org/494470 | 12:39 |
| mwhahaha | dtantsur: it's a good ux improvement but not sure if it's backportable. We would need a bug to backport it as well. Maybe file a bug and explain a case for a backport | 12:42 |
| dtantsur | mwhahaha: yeah, I'm going to fix it for master first anyway | 12:42 |
| *** rbrady has joined #tripleo | 12:43 | |
| *** rbrady has quit IRC | 12:43 | |
| *** rbrady has joined #tripleo | 12:43 | |
| *** gkadam has quit IRC | 12:43 | |
| *** jlabarre has joined #tripleo | 12:44 | |
| sshnaidm | derekh, do you know whom to talk when there are network issues in rh1 cloud? | 12:46 |
| openstackgerrit | Numan Siddique proposed openstack/tripleo-quickstart master: Add fset030 for OVN with containerized oc https://review.openstack.org/490624 | 12:46 |
| mhenkel | d0ugal: https://bugzilla.redhat.com/show_bug.cgi?id=1484396 | 12:47 |
| openstack | bugzilla.redhat.com bug 1484396 in rhosp-director "the rhel-7-server-openstack-11-rpms repo python-tripleoclient and openstack-tripleo-common are incompatible" [Unspecified,New] - Assigned to athomas | 12:47 |
| *** dmacpher has joined #tripleo | 12:47 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo master: Fix panko port to match tht https://review.openstack.org/496694 | 12:48 |
| *** udesale has joined #tripleo | 12:48 | |
| d0ugal | mhenkel: thanks! | 12:48 |
| openstackgerrit | Numan Siddique proposed openstack/tripleo-heat-templates master: Support deploying OVN as container services https://review.openstack.org/468860 | 12:49 |
| *** atoth has quit IRC | 12:50 | |
| *** jlinkes has quit IRC | 12:50 | |
| jaosorior | pradk: is that something we need to backport? can you file a launchpad bug too? | 12:50 |
| *** dpawar has quit IRC | 12:51 | |
| *** pchavva has joined #tripleo | 12:51 | |
| pradk | i think we changed this recently.. lemme double check | 12:51 |
| pradk | yea looks like we need to backport.. i'll file a bug | 12:52 |
| pradk | jaosorior, i guess this proves no one is using panko .. lol | 12:52 |
| jaosorior | pradk: well, most deployers want to stick with the long term supported releases :/ that might be a reason | 12:53 |
| pradk | yea | 12:53 |
| pradk | k bug on the way | 12:53 |
| jaosorior | pradk: thanks | 12:53 |
| *** ffiore_ has quit IRC | 12:54 | |
| *** shardy has joined #tripleo | 12:55 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo master: Fix panko port to match tht https://review.openstack.org/496694 | 12:55 |
| openstackgerrit | Raoul Scarazzini proposed openstack/tripleo-quickstart-extras master: Restore tags inside the main playbook https://review.openstack.org/473491 | 12:56 |
| *** tzumainn has joined #tripleo | 12:56 | |
| jaosorior | pradk: don't we test panko in some scenario? | 12:57 |
| pradk | jaosorior, we do, perhaps not with ssl | 12:57 |
| jaosorior | aah that would make sense | 12:58 |
| jaosorior | haha well shit | 12:58 |
| pradk | and panko is deprecated in pike .. ;) | 12:59 |
| pradk | hopefully it will die in queen, lets see | 12:59 |
| *** cdearborn has joined #tripleo | 12:59 | |
| egonzalez | trozet, ping | 13:00 |
| trozet | egonzalez: i got to go afk for a bit, will ping you when i get back | 13:01 |
| *** oidgar has quit IRC | 13:01 | |
| egonzalez | trozet, roger | 13:01 |
| *** gbarros has joined #tripleo | 13:02 | |
| *** lucas-hungry is now known as lucasagomes | 13:03 | |
| *** atoth has joined #tripleo | 13:03 | |
| *** fzdarsky_ has quit IRC | 13:08 | |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 13:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 13:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 13:10 |
| *** jpena|lunch is now known as jpena | 13:10 | |
| *** florianf has quit IRC | 13:16 | |
| *** dsariel|afk has quit IRC | 13:16 | |
| *** oidgar has joined #tripleo | 13:18 | |
| *** dparkes has quit IRC | 13:20 | |
| *** florianf has joined #tripleo | 13:21 | |
| *** snecklifter has joined #tripleo | 13:21 | |
| openstackgerrit | Jakub Libosvar proposed openstack/os-net-config stable/pike: Delete controller for standalone OVS bridges https://review.openstack.org/496707 | 13:23 |
| pabelanger | mwhahaha: weshay_PTO_AM: EmilienM: could somebody please add the missing container job scenarios to quickstact-extras? That seemed to be the reason from stevebaker last night for the gate breakage | 13:24 |
| *** bkopilov has joined #tripleo | 13:26 | |
| *** bnemec has joined #tripleo | 13:26 | |
| *** noslzzp has quit IRC | 13:27 | |
| *** amoralej is now known as amoralej|lunch | 13:27 | |
| shardy | dtantsur: Hey can we mark https://blueprints.launchpad.net/tripleo/+spec/node-discovery-by-range implemented? | 13:27 |
| shardy | looks like everything except the UI patch landed now? | 13:27 |
| *** noslzzp has joined #tripleo | 13:28 | |
| *** ansmith has joined #tripleo | 13:28 | |
| dtantsur | shardy: I'm fine with that, dunno about UI folks | 13:30 |
| dtantsur | but yes, the UI patch was the last | 13:30 |
| shardy | honza: ^^ can we declare the BP complete and raise a bug to track adding UI support? | 13:31 |
| *** masco has quit IRC | 13:31 | |
| *** hewbrocca is now known as hewbrocca_afk | 13:33 | |
| *** oidgar has quit IRC | 13:34 | |
| openstackgerrit | Michael Chapman proposed openstack/tripleo-quickstart-extras master: Add opendaylight to collect-logs https://review.openstack.org/494043 | 13:35 |
| openstackgerrit | Michael Chapman proposed openstack/tripleo-quickstart-extras master: Add opendaylight to collect-logs https://review.openstack.org/494043 | 13:37 |
| *** fzdarsky has joined #tripleo | 13:37 | |
| *** links has quit IRC | 13:38 | |
| openstackgerrit | Michael Chapman proposed openstack/tripleo-quickstart-extras master: Add opendaylight to collect-logs https://review.openstack.org/494043 | 13:38 |
| trown | sshnaidm: could I get a review on https://review.openstack.org/475747 ? would be nice for the periodic job to work for container builds instead of needing to run it via "check manual" | 13:39 |
| abishop | shardy: hoping you can take another look at https://review.openstack.org/#/c/480587/ (commit message has been updated) | 13:42 |
| lvdombrkr | shardy: just to let you know for future it was my mistake i was added needless repo | 13:42 |
| lvdombrkr | shardy: now glance works fine | 13:42 |
| *** agurenko has quit IRC | 13:43 | |
| lvdombrkr | shardy: thank you in assist | 13:43 |
| *** ansmith_ has joined #tripleo | 13:43 | |
| shardy | lvdombrkr: ack, well that's good news, glad you got it working :) | 13:43 |
| shardy | abishop: will do | 13:43 |
| *** eck`gone is now known as eck` | 13:45 | |
| openstackgerrit | Jakub Libosvar proposed openstack/os-net-config stable/pike: Delete controller for standalone OVS bridges https://review.openstack.org/496707 | 13:46 |
| *** fpan has joined #tripleo | 13:46 | |
| *** noslzzp has quit IRC | 13:48 | |
| *** noslzzp has joined #tripleo | 13:49 | |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: Add ironic-inspector to the undercloud-containers job https://review.openstack.org/476130 | 13:51 |
| *** Goneri has joined #tripleo | 13:57 | |
| *** limao has quit IRC | 13:58 | |
| abishop | shardy: thank you! once that one merges, my own https://review.openstack.org/#/c/489239/ will needs reviews | 13:58 |
| shardy | abishop: ack - it'd be good if we could land those before the RC causes tripleo-common to branch tomorrow | 13:59 |
| abishop | shardy: whoa, yeah! In that case, can you please review https://review.openstack.org/#/c/489239/ ? | 14:00 |
| *** oidgar has joined #tripleo | 14:01 | |
| abishop | d0ugal: please revisit https://review.openstack.org/#/c/480587/; you approved once before, but commit msg was updated | 14:02 |
| d0ugal | abishop: will do | 14:02 |
| shardy | I just rechecked due to the ovb job failures, but the following patch in the series passed them, so perhaps we should just approve them | 14:03 |
| abishop | d0ugal, shardy, jaosorior: thank you for the fast response! | 14:04 |
| *** hamzy has quit IRC | 14:05 | |
| *** florianf has quit IRC | 14:06 | |
| *** udesale has quit IRC | 14:08 | |
| *** florianf has joined #tripleo | 14:08 | |
| *** cylopez has quit IRC | 14:08 | |
| *** florianf has quit IRC | 14:08 | |
| pabelanger | trown: where will 475747 run, on rdocloud? | 14:09 |
| *** sshnaidm is now known as sshnaidm|afk | 14:09 | |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 14:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 14:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 14:10 |
| openstackgerrit | Merged openstack/tripleo-common master: Add ironic-api healthcheck https://review.openstack.org/490836 | 14:12 |
| *** amoralej|lunch is now known as amoralej | 14:12 | |
| *** bkopilov has quit IRC | 14:15 | |
| *** udesale has joined #tripleo | 14:15 | |
| *** ykarel has quit IRC | 14:17 | |
| *** florianf has joined #tripleo | 14:20 | |
| mwhahaha | gfidente: does ansible-ceph try and install epel? | 14:20 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Add panko publisher in the event pipeline https://review.openstack.org/496723 | 14:22 |
| mwhahaha | gfidente: http://logs.openstack.org/74/495374/1/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/cf08cc9/logs/undercloud/var/log/mistral/executor.log.txt.gz#_2017-08-23_11_51_24_377 seems to have failed while trying to reach out to epel or something | 14:23 |
| mwhahaha | gfidente: changed: [192.168.24.11] => (item=epel-release)\nfailed: [192.168.24.11] (item=python-setuptools) => {"failed": true, "item": "python-setuptools", "msg": "Failure talking to yum: Cannot retrieve metalink for repository: epel/x86_64. Please verify its path and try again"}\nok: [192.168.24.11] => (item=libselinux-python)\n\nPLAY RECAP | 14:23 |
| trown | pabelanger: ya that runs on rdo cloud | 14:23 |
| openstackgerrit | Adriano Petrich proposed openstack/instack-undercloud master: Add an hourly cron trigger for tripleo-ui logging https://review.openstack.org/469608 | 14:24 |
| pabelanger | trown: thanks | 14:25 |
| apetrich | honza, I added the changes to delete cron-triggers on https://review.openstack.org/#/c/469608/ is that alright? it was removed between 9..10 | 14:26 |
| *** itlinux has joined #tripleo | 14:28 | |
| numans | sshnaidm|afk, can you please see this review https://review.openstack.org/#/c/468860/ and the job http://logs.openstack.org/60/468860/25/check/gate-tripleo-ci-centos-7-nonha-multinode-oooq/e4d85bd/logs/undercloud/home/jenkins/overcloud-deploy.sh.txt.gz | 14:29 |
| numans | sshnaidm|afk, something seems to be wrong in the way overcloud-deploy.sh is generated | 14:30 |
| *** udesale has quit IRC | 14:31 | |
| *** udesale__ has joined #tripleo | 14:31 | |
| *** itlinux has quit IRC | 14:34 | |
| lvdombrkr | guys my Stack CREATE completed successfully but there is no overcloudrc generated in /home/stack | 14:36 |
| lvdombrkr | what does thats mean? | 14:36 |
| *** mdnadeem has quit IRC | 14:37 | |
| shardy | lvdombrkr: did you ctrl-C from the client before the stack create completed? | 14:38 |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo stable/ocata: Move gnocchi::api resource to run with wsgi setup https://review.openstack.org/489437 | 14:38 |
| lvdombrkr | shardy: what you mean? | 14:39 |
| *** trown is now known as trown|brb | 14:41 | |
| *** vpickard_ is now known as vpickard | 14:41 | |
| *** hamzy has joined #tripleo | 14:43 | |
| *** ykarel has joined #tripleo | 14:47 | |
| openstackgerrit | Flavio Percoco proposed openstack/tripleo-heat-templates master: WIP: Deploy OpenShift using OOO on the overcloud https://review.openstack.org/494470 | 14:47 |
| shardy | lvdombrkr: If you exit the client early the overcloudrc isnt written, also check which directory you ran the overcloud deploy command in, as IIRC it writes to $PWD/overcloudrc | 14:47 |
| gfidente | mwhahaha yeah I noticed that error in a few runs | 14:47 |
| gfidente | it looks like it tries yes | 14:48 |
| mwhahaha | gfidente: can we supress that? | 14:48 |
| gfidente | we had to allow it do install packages because on deployed-server there isn't everything we need | 14:48 |
| mwhahaha | gfidente: cause that's going to be a problem | 14:48 |
| gfidente | but yes looking into it | 14:48 |
| *** ebarrera has quit IRC | 14:49 | |
| *** ebarrera has joined #tripleo | 14:49 | |
| *** anshul has quit IRC | 14:49 | |
| lvdombrkr | shardy: how can i chack that stack was deployed reallu succees? | 14:50 |
| *** trown|brb is now known as trown | 14:50 | |
| *** oanson has quit IRC | 14:53 | |
| *** oanson has joined #tripleo | 14:54 | |
| *** leitan has joined #tripleo | 14:56 | |
| *** yprokule has quit IRC | 14:57 | |
| *** fpan has quit IRC | 14:58 | |
| jaosorior | shardy: is there an easy way to tell if a deployment is containerized? from a container point of view | 15:00 |
| jaosorior | dprince: ^^ | 15:02 |
| shardy | jaosorior: you mean if any services are running in containers, or a specific service? | 15:03 |
| jaosorior | any service | 15:04 |
| jaosorior | do we have a hiera key that tells that docker is being deployed? or something like that? | 15:04 |
| dprince | jaosorior: I think there is probably something in /proc | 15:05 |
| dprince | jaosorior: but we aren't doing any fancy tricks for that yet | 15:05 |
| *** sshnaidm|afk is now known as sshnaidm | 15:06 | |
| jpena | dprince: about the openstackclient command, I've just tested it on a local container and "openstack overcloud container image prepare" is exposed | 15:06 |
| dprince | jaosorior: as far as hiera goes I don't think we write anything yet to signal that a container is used. This would need to be per-service at this point as our architecture is currently hybrid | 15:06 |
| *** ebarrera has quit IRC | 15:06 | |
| jpena | versions: python2-openstackclient-3.12.0-0.20170818132251.f67ebce.el7.centos.noarch and python-tripleoclient-7.2.1-0.20170823093447.4ae96dd.el7.centos.noarch (from the "/current" directory) | 15:06 |
| dprince | jpena: really. I'm perplex cause it was working for me too | 15:07 |
| openstackgerrit | Michael Bayer proposed openstack/puppet-tripleo stable/ocata: Add option for innodb_flush_log_at_trx_commit = 2 for Galera only https://review.openstack.org/496790 | 15:07 |
| dprince | jpena: I'm building a container with just python-tripleoclient in it | 15:07 |
| shardy | dprince: inside the container can you see the bind mounts? | 15:08 |
| * shardy doesn't have a containerized deployment running atm to check | 15:08 | |
| amoralej | dprince, i see the endpoint "openstack overcloud container image prepare" available in master repo | 15:08 |
| dprince | shardy: if they are exposed. But they just show up as normal dirs | 15:08 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Release extra-args from multinode environment settings https://review.openstack.org/496676 | 15:09 |
| dprince | jpena, amoralej: http://paste.openstack.org/show/619186/ | 15:09 |
| amoralej | dprince, what repo are you using | 15:09 |
| amoralej | ? | 15:09 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Add panko publisher in the event pipeline https://review.openstack.org/496723 | 15:10 |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 15:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| dprince | amoralej: https://trunk.rdoproject.org/centos7/75/a2/75a2e3bde576a18219eddedba9dcc796673f1d4b_a2b1e6ac | 15:10 |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 15:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 15:10 |
| dprince | amoralej: but fwiw I tried current-tripleo too and got similar | 15:10 |
| sshnaidm | numans, please try again, I hope fixed it | 15:10 |
| numans | sshnaidm, sure. thanks. | 15:10 |
| amoralej | dprince, using current-passed-ci http://paste.openstack.org/show/619188/ let me check what's happening with current | 15:11 |
| dprince | amoralej: cool | 15:11 |
| mwhahaha | pradk: https://review.openstack.org/#/c/496751/ do we need this in pike rc1 for puppet-ceilometer? | 15:13 |
| mwhahaha | pradk: nm just saw your update, i'll get it in | 15:13 |
| pradk | mwhahaha, yea just replied | 15:13 |
| pradk | sorry for last minute | 15:13 |
| jpena | amoralej: I used current in my test | 15:14 |
| openstackgerrit | Michael Bayer proposed openstack/puppet-tripleo stable/newton: Add option for innodb_flush_log_at_trx_commit = 2 for Galera only https://review.openstack.org/496794 | 15:14 |
| pabelanger | any suggestions to see why this gate job failed: http://logs.openstack.org/00/495900/3/gate/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/b09cf60/logs/undercloud/home/jenkins/failed_deployment_list.log.txt.gz | 15:15 |
| *** udesale__ has quit IRC | 15:15 | |
| pabelanger | resources.WorkflowTasks_Step2_Execution: ERROR I do not know where to look for that | 15:16 |
| mwhahaha | pabelanger: see my comment about epel to gfidente | 15:16 |
| pabelanger | mwhahaha: thanks, looking | 15:16 |
| pabelanger | oh noes, epel | 15:16 |
| mwhahaha | pabelanger: yea | 15:17 |
| mwhahaha | ceph-ansible is needing to install stuff which is problematic | 15:17 |
| mwhahaha | or ansible-ceph whichever it is | 15:17 |
| pabelanger | k, I don't yet understand all the bit, but is that happening in a docker container? | 15:19 |
| trozet | egonzalez: hey | 15:19 |
| pabelanger | or on the overcloud | 15:19 |
| egonzalez | trozet, is tacker master tested in tripleo or apex? | 15:19 |
| lvdombrkr | shardy: any ideas why overcloudrc no generated? | 15:19 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Make archive policy configurable https://review.openstack.org/496796 | 15:20 |
| pabelanger | http://logs.openstack.org/00/495900/3/gate/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/b09cf60/logs/undercloud/var/log/mistral/ceph-install-workflow.log.txt.gz is a much nicer log | 15:20 |
| trozet | egonzalez: i'm going to be testing it soon with ooo/apex on ocata | 15:20 |
| mwhahaha | pabelanger: i'm not sure where that occurs but yea that log is much nicer :D | 15:20 |
| trozet | egonzalez: why? what's up? | 15:20 |
| egonzalez | trozet, saying this, because tacker requires mistral for vim monitoring, which requires tacker-conductor that as far i can find is not implemented | 15:21 |
| pabelanger | mwhahaha: gfidente: do you happen to know where the epel install happens? In container or on overcloud node? | 15:21 |
| pabelanger | mwhahaha: gfidente: hopefully overcloud | 15:21 |
| mwhahaha | pabelanger: i'd assume overcloud | 15:21 |
| pabelanger | mwhahaha: ya, I am hoping that too | 15:21 |
| gfidente | pabelanger mwhahaha sorry guys haven't really started as I am pulled into an admin issue :D | 15:21 |
| pabelanger | because, if overcloud. python-setuptools is already installed | 15:22 |
| gfidente | but yeah it's installing some packageson the baremetal | 15:22 |
| *** fpan has joined #tripleo | 15:22 | |
| *** pkovar has quit IRC | 15:23 | |
| pabelanger | https://github.com/ceph/ceph-ansible ? | 15:23 |
| pabelanger | is that the playbook it is using | 15:23 |
| mwhahaha | yes | 15:23 |
| pabelanger | https://github.com/ceph/ceph-ansible/blob/master/roles/ceph-defaults/defaults/main.yml#L59 | 15:23 |
| pabelanger | okay, at least they expost it | 15:24 |
| *** jlinkes has joined #tripleo | 15:24 | |
| pabelanger | so, can pass in the package dependencies with epel | 15:24 |
| mwhahaha | i like how epel-release is a dependency | 15:24 |
| trozet | egonzalez: afaik tacker is monolithic and does not have separated services | 15:24 |
| pabelanger | in fact, I would unset both | 15:24 |
| pabelanger | and just have tripleo manage them | 15:24 |
| *** pkovar has joined #tripleo | 15:24 | |
| mwhahaha | indeed | 15:24 |
| egonzalez | trozet, changed in pike https://docs.openstack.org/tacker/latest/contributor/tacker_conductor.html | 15:25 |
| trozet | egonzalez: maybe that changed, I see https://github.com/openstack/tacker/tree/master/tacker/conductor | 15:25 |
| trozet | egonzalez: looks like we have some work to do then in ooo and puppet-tacker | 15:25 |
| egonzalez | trozet, conductor is meant for RPC calls with mistral | 15:25 |
| *** catintheroof has joined #tripleo | 15:25 | |
| egonzalez | trozet, yep. there is not that much config to do, just start a new service with same config file | 15:27 |
| pabelanger | mwhahaha: gfidente: can we create a bug in launchpad to triage? Since it is breaking the gate | 15:27 |
| trozet | egonzalez: https://github.com/openstack/tacker/blob/master/tacker/conductor/conductorrpc/vim_monitor_rpc.py#L30 | 15:27 |
| mwhahaha | pabelanger: gfidente yea let me file a bug | 15:28 |
| trozet | egonzalez: are you going to handle doing those updates? | 15:28 |
| pabelanger | mwhahaha: tyty | 15:28 |
| egonzalez | trozet, -20% puppet skills | 15:28 |
| trozet | egonzalez: haha ok, do you want to create a LP and then assign to me and I will get to it? | 15:29 |
| egonzalez | trozet, sure, blueprint or bug for backport? | 15:29 |
| mwhahaha | pabelanger: gfidente https://bugs.launchpad.net/tripleo/+bug/1712599 | 15:31 |
| openstack | Launchpad bug 1712599 in tripleo "scenario001 fails because ceph-ansible is trying to install epel-release" [Critical,Triaged] | 15:31 |
| egonzalez | trozet, i can target this kolla bug to tripleo too https://bugs.launchpad.net/kolla-ansible/+bug/1710874 | 15:31 |
| openstack | Launchpad bug 1710874 in kolla-ansible "tacker broken in pike, need conductor service" [Undecided,Fix released] - Assigned to Eduardo Gonzalez (egonzalez90) | 15:31 |
| *** dsariel has joined #tripleo | 15:31 | |
| gfidente | mwhahaha thanks guys | 15:31 |
| gfidente | mwhahaha pabelanger though some runs passed | 15:34 |
| gfidente | so it's unclear when causes the issue with the epel repo itself | 15:34 |
| gfidente | *what | 15:34 |
| gfidente | but I agree we should try not to use it anyway | 15:34 |
| mwhahaha | gfidente: network because it's probably pulling remote | 15:34 |
| mwhahaha | gfidente: we're trying to reduce our reliance on external stuff | 15:35 |
| mwhahaha | plus we shouldn't be allowing it to pull in packages anyway | 15:35 |
| gfidente | so we can disable some tagfs | 15:35 |
| gfidente | and that prevents installation of packages | 15:35 |
| gfidente | but we need to install some stuff on the infra nodes | 15:35 |
| pabelanger | gfidente: mwhahaha: yup, it is hitting epel directly, and not using our AFS mirrors. That is why it is intermittent. But bigger thing, depending on epel is bad now. I know a lot of effort has gone into remove that | 15:36 |
| gfidente | right | 15:36 |
| gfidente | we don't want epel anyway | 15:36 |
| pabelanger | you shouldn't need python-setuptool, it should be there | 15:36 |
| *** oidgar has quit IRC | 15:36 | |
| pabelanger | also, I would see why you need python-setuptools | 15:36 |
| pabelanger | is that building a package locally? | 15:36 |
| mwhahaha | https://github.com/ceph/ceph-ansible/blob/master/roles/ceph-defaults/defaults/main.yml#L80 | 15:37 |
| mwhahaha | probably if ceph_origin is upstream | 15:38 |
| mwhahaha | or something | 15:38 |
| amoralej | dprince, it's working to me with current, do you have some log of failure? | 15:38 |
| gfidente | trown so would https://bugs.launchpad.net/tripleo/+bug/1712432/comments/4 unblock us? | 15:38 |
| openstack | Launchpad bug 1712432 in tripleo "mistral fails to start the ceph container" [High,Triaged] - Assigned to John Trowbridge (trown) | 15:38 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Set gnocchi archivepolicy in scenario001 job to high https://review.openstack.org/496800 | 15:39 |
| pabelanger | mwhahaha: okay, I think I see a bigger issue: https://review.openstack.org/#/c/495900/ | 15:40 |
| pabelanger | mwhahaha: Why didn't gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container run in check | 15:40 |
| pabelanger | and only in gate | 15:40 |
| pabelanger | same with gate-tripleo-ci-centos-7-scenario001-multinode-oooq | 15:41 |
| mwhahaha | pabelanger: we added that after i think | 15:41 |
| mwhahaha | pabelanger: ie added yesterday afternoon | 15:41 |
| pabelanger | Ah | 15:41 |
| pabelanger | okay | 15:41 |
| mwhahaha | specifically for this :D | 15:41 |
| pabelanger | ya | 15:41 |
| pabelanger | cool | 15:41 |
| mwhahaha | hrm seems to be happening a bunch more so i'm going to throw an alert on that bug | 15:43 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Separate config_volume for ringbuilder https://review.openstack.org/494008 | 15:43 |
| pabelanger | mwhahaha: what ansible-ceph recently added? | 15:45 |
| mwhahaha | no | 15:46 |
| mwhahaha | well sort of | 15:46 |
| *** bnemec has quit IRC | 15:46 | |
| egonzalez | trozet, cannot find/assign you (probably not having permissions in tripleo-drivers LP group) This is bug, https://bugs.launchpad.net/tripleo/+bug/1710874 | 15:46 |
| openstack | Launchpad bug 1710874 in tripleo "tacker requires conductor service in pike" [Undecided,New] | 15:46 |
| pabelanger | there is also the issue that mistral logs are not getting indexed by logstash | 15:46 |
| pabelanger | so, elastic-recheck didn't catch this | 15:46 |
| *** bnemec has joined #tripleo | 15:47 | |
| vkmc | hey guys, I'm seeing some errors when trying to get container images on the compute node... http://paste.openstack.org/show/619195/ | 15:47 |
| *** itlinux has joined #tripleo | 15:47 | |
| pabelanger | vkmc: is that local or in the gate? | 15:47 |
| trown | gfidente: ya... not really blocked but ya that will fix it | 15:47 |
| vkmc | is there some known problem with the repo or is something on my env? | 15:47 |
| vkmc | pabelanger, local | 15:48 |
| gfidente | trown you have a change to implement that or shall I? | 15:48 |
| pabelanger | vkmc: ya, looks like networking issue to docker.io. If it was the gate, we have a reverse proxy to docker.io | 15:48 |
| gfidente | mwhahaha pabelanger fultonj I might have a fix for the scenarios | 15:49 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: WIP create default settings for environment types https://review.openstack.org/495407 | 15:49 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Change devmode to deploy containerized services by default https://review.openstack.org/472412 | 15:50 |
| Slower | mandre: were you going to build new tripleoupstream images? | 15:50 |
| * Slower checks to see if you did :) | 15:50 | |
| vkmc | pabelanger, k, so... how should I be deploying the overcloud for testing purposes...? should I be using an additional environment file? | 15:51 |
| pabelanger | gfidente: I couldn't mind why python-setuptools is needed. If you have a sucessful run of ansible-cept log, it would be helpful | 15:51 |
| vkmc | pabelanger, this is the overcloud deploy I'm performing "openstack overcloud deploy --templates -e /usr/share/openstack-tripleo-heat-templates/environments/docker.yaml -e $HOME/docker_registry.yaml" | 15:51 |
| pabelanger | vkmc: when you say local, are you just using your local environment or is this in rdocloud for example | 15:52 |
| *** gbarros has quit IRC | 15:52 | |
| larsks | So...it looks like newer versions of hiera (I've got 3.x locally) will break on many of the variables we're using in tripleo (it looks like "." becomes a hierarchy separator, so all the "tripleo.x.y.z" vars stop working). Is that something for which there is/should an open issue? | 15:52 |
| vkmc | pabelanger, local environment | 15:52 |
| pabelanger | vkmc: best you could do in that case would be setup https://docs.docker.com/registry/recipes/mirror/ then configure your docker pulls to hit your local registry first. | 15:53 |
| trown | gfidente: oh... didnt see comment 4 yet... thought we were just missing commits in tripleo-ci | 15:53 |
| pabelanger | vkmc: you'd start caching the requests over going to docker.io everytime | 15:53 |
| *** dixiaoli has joined #tripleo | 15:53 | |
| vkmc | pabelanger, so... building images locally | 15:53 |
| vkmc | right? | 15:53 |
| trown | gfidente: ya I can put up patch for that | 15:54 |
| vkmc | pabelanger, does this always happen? | 15:54 |
| pabelanger | vkmc: that is an option too. The proxy cache does the same thing, without you building | 15:54 |
| gfidente | trown cool, ty! :D | 15:54 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Do not install epel-release with ceph-ansible https://review.openstack.org/496806 | 15:54 |
| pabelanger | vkmc: in the gate it happens a lot. That is why we setup a reverse proxy cache to docker. Now our regional mirrors cache all images from docker.io | 15:54 |
| gfidente | mwhahaha pabelanger fultonj ^^ | 15:54 |
| vkmc | pabelanger, can you point me to some doc explaining on how to configure my docker pulls to hit the local registry first? | 15:55 |
| vkmc | all right | 15:55 |
| pabelanger | vkmc: mirror.dfw.rax.openstack.org:8081/registry-1.docker for exmaple | 15:55 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo master: Use resource collector for the fencing -> stonith ordering https://review.openstack.org/496807 | 15:55 |
| pabelanger | vkmc: 1 sec, I can find the changes we made for the gate | 15:55 |
| openstackgerrit | Steven Hardy proposed openstack/python-tripleoclient master: Consume new RoleConfig data in config download https://review.openstack.org/485732 | 15:55 |
| vkmc | pabelanger, k, thanks | 15:55 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Do not install epel-release with ceph-ansible https://review.openstack.org/496806 | 15:56 |
| fultonj | gfidente: i see, so you override the deps list with a list not containing epel | 15:57 |
| gfidente | fultonj yeah | 15:57 |
| pabelanger | vkmc: https://review.openstack.org/491923/ seeing appears to be undercloud_docker_registry_mirror | 15:57 |
| gfidente | only for scenario jobs | 15:57 |
| pabelanger | setting* | 15:57 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Telemetry specific config for scenario001 https://review.openstack.org/496800 | 15:57 |
| *** dixiaoli has quit IRC | 15:57 | |
| vkmc | pabelanger, thanks | 15:58 |
| pabelanger | vkmc: np | 15:59 |
| *** gbarros has joined #tripleo | 15:59 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates master: Separate config_volume for ringbuilder https://review.openstack.org/494008 | 15:59 |
| *** tongl has joined #tripleo | 15:59 | |
| *** Guest28943 is now known as sdake | 16:00 | |
| *** sdake has joined #tripleo | 16:00 | |
| *** bogdando has quit IRC | 16:00 | |
| *** florianf_ has joined #tripleo | 16:01 | |
| *** florianf has quit IRC | 16:02 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Telemetry specific config for scenario001 https://review.openstack.org/496800 | 16:03 |
| trozet | egonzalez: thanks assigned it to myself | 16:05 |
| *** bnemec has quit IRC | 16:06 | |
| *** funzo has joined #tripleo | 16:07 | |
| *** jkilpatr has quit IRC | 16:08 | |
| *** milan has quit IRC | 16:08 | |
| *** suuuper has quit IRC | 16:09 | |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 16:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712599 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 16:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 16:10 |
| openstack | Launchpad bug 1712599 in tripleo "scenario001 fails because ceph-ansible is trying to install epel-release" [Critical,In progress] - Assigned to Giulio Fidente (gfidente) | 16:10 |
| *** funzo has quit IRC | 16:11 | |
| *** iranzo has quit IRC | 16:11 | |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart master: Add missing ceph-ansible install to appropriate releases https://review.openstack.org/496822 | 16:15 |
| *** florianf_ is now known as florianf | 16:16 | |
| *** shardy has quit IRC | 16:16 | |
| *** jfrancoa has quit IRC | 16:19 | |
| *** jfrancoa has joined #tripleo | 16:20 | |
| *** snecklifter has quit IRC | 16:20 | |
| mandre | Slower: yeah! I pushed new images yesterday | 16:22 |
| mandre | you probably figured this by now :) | 16:22 |
| *** weshay_PTO_AM is now known as weshay | 16:25 | |
| *** ykarel has quit IRC | 16:26 | |
| *** colonwq has quit IRC | 16:26 | |
| *** colonwq has joined #tripleo | 16:27 | |
| weshay | pabelanger, did you get help re: missing container jobs? not sure what we're talking about yet to be honest | 16:27 |
| mwhahaha | weshay: i think it was having the scenario jobs on oooq-extras | 16:28 |
| weshay | to check gate tqe? | 16:28 |
| mwhahaha | yea | 16:28 |
| weshay | k | 16:28 |
| mwhahaha | cause it broke stuff last night | 16:28 |
| *** dtantsur is now known as dtantsur|afk | 16:29 | |
| *** rcernin has quit IRC | 16:29 | |
| tongl | mwhahaha: can you point to some docs that talks about more details about how to deploy Red Hat OpenStack with master code? I checked tripleo.org yesterday and have no clue where to start. Thanks! | 16:31 |
| pabelanger | weshay: mwhahaha: ya, thanks | 16:32 |
| mwhahaha | tongl: not sure what more you want because those docks are the ones to deploy master packages from rdo | 16:32 |
| weshay | mwhahaha, pabelanger k.. thanks for letting me know, doesn't look like there are open reviews | 16:32 |
| weshay | so I'll go take care of it | 16:32 |
| pabelanger | weshay: ya, I haven't proposed anything | 16:32 |
| tongl | mwhahaha: Am I supposed to use instack-undercloud to pull master? http://tripleo.org/install/introduction/components.html#instack-undercloud | 16:34 |
| mwhahaha | tongl: so i'm not sure what you're doing. to use master you would just use the master repos from rdo. then deploy it as needed. | 16:35 |
| tongl | mwhahaha: we added our networking driver to tripleo master/pike. currently the release version of OpenStack Platform 10 is newton and 11 is ocata, which don't have our code. I want to test our neutron driver on master/pike. | 16:37 |
| mwhahaha | tongl: then you would just follow the dev instructions from tripleo.org | 16:37 |
| *** ykarel has joined #tripleo | 16:38 | |
| *** jlinkes has quit IRC | 16:38 | |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-heat-templates master: Containerized Fluentd client https://review.openstack.org/467072 | 16:41 |
| *** mdnadeem has joined #tripleo | 16:42 | |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-common master: Run fluentd container https://review.openstack.org/487038 | 16:42 |
| openstackgerrit | David Sariel proposed openstack/tripleo-validations master: Add containers_sanity_check https://review.openstack.org/483403 | 16:42 |
| openstackgerrit | David Sariel proposed openstack/tripleo-validations master: Add containers_sanity_check https://review.openstack.org/483403 | 16:43 |
| *** itlinux has quit IRC | 16:44 | |
| *** aditya_r has joined #tripleo | 16:45 | |
| jrist | almost ready. please review https://review.openstack.org/#/c/469608/ | 16:47 |
| jrist | jpich, EmilienM, d0ugal | 16:47 |
| jrist | plz | 16:47 |
| *** jpich has quit IRC | 16:48 | |
| *** mcornea has quit IRC | 16:48 | |
| *** tongl has quit IRC | 16:48 | |
| *** yprokule has joined #tripleo | 16:50 | |
| weshay | pabelanger, https://review.openstack.org/#/c/496830/ | 16:50 |
| *** jkilpatr has joined #tripleo | 16:53 | |
| mwhahaha | uh oh | 16:53 |
| mwhahaha | i think ovb just got broken | 16:53 |
| mwhahaha | 2017-08-23 16:35:05 | Error: Package: rubygem-fluent-plugin-secure-forward-0.4.5-1.el7.noarch (quickstart-centos-opstools) | 16:54 |
| mwhahaha | 2017-08-23 16:35:05 | Requires: rubygem(proxifier) | 16:54 |
| mwhahaha | http://logs.openstack.org/00/495900/3/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-oooq/d94fe0b/logs/undercloud/home/jenkins/overcloud_image_build.log.txt.gz#_2017-08-23_16_35_09 | 16:54 |
| *** egonzalez has quit IRC | 16:54 | |
| *** itlinux has joined #tripleo | 16:55 | |
| *** jcoufal has quit IRC | 16:56 | |
| * mwhahaha files a bug and sighs | 16:56 | |
| *** trown is now known as trown|lunch | 16:56 | |
| mwhahaha | https://bugs.launchpad.net/tripleo/+bug/1712621 | 16:57 |
| openstack | Launchpad bug 1712621 in tripleo "ovb overcloud image build broken due to rubygem-fluent-plugin-secure-forward" [Critical,Triaged] | 16:57 |
| *** gbarros has quit IRC | 17:03 | |
| *** fzdarsky is now known as fzdarsky|afk | 17:03 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: allow tqe roles to work in sync with ARA https://review.openstack.org/482237 | 17:05 |
| rook | mwhahaha dprince what is the best way to restart a container? | 17:06 |
| *** thrash is now known as thrash|biab | 17:07 | |
| *** lucasagomes is now known as lucas-afk | 17:07 | |
| *** pcaruana has quit IRC | 17:08 | |
| *** yprokule has quit IRC | 17:08 | |
| *** derekh has quit IRC | 17:10 | |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 17:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712599 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712621 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 17:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 17:10 |
| openstack | Launchpad bug 1712599 in tripleo "scenario001 fails because ceph-ansible is trying to install epel-release" [Critical,In progress] - Assigned to Giulio Fidente (gfidente) | 17:10 |
| openstack | Launchpad bug 1712621 in tripleo "ovb overcloud image build broken due to rubygem-fluent-plugin-secure-forward" [Critical,Triaged] | 17:10 |
| mwhahaha | rook: can't you just do docker restart <container> | 17:11 |
| pabelanger | weshay: mwhahaha: gfidente: left a -1 on 496806. I think the patch, we should apply it to all scenories by default | 17:11 |
| rook | mwhahaha: i dunno... i haven't tried.. didn't know if that would break things | 17:12 |
| rook | ill try it | 17:12 |
| gfidente | pabelanger well, that is the only scenario testing ceph-ansible for now | 17:12 |
| rook | i want to kick the haproxy container | 17:12 |
| mwhahaha | rook: pretty sure that's the way to do it | 17:12 |
| *** tesseract-RH has quit IRC | 17:12 | |
| pabelanger | gfidente: will others in the future? | 17:13 |
| mwhahaha | gfidente: pabelanger: so it doesn't need to be in all scenarios (they don't all deploy ceph) but I'm still concerned why we have any packages being installed by this process | 17:13 |
| pabelanger | gfidente: my fear, is it will get overlooked and we hit the issue again | 17:13 |
| pabelanger | mwhahaha: yes, better understanding that too is helpful | 17:13 |
| pabelanger | I think these are dependencies for ansible | 17:13 |
| mwhahaha | which don't we already have in our images? | 17:14 |
| pabelanger | but really need to see a successful deploy log | 17:14 |
| *** gbarros has joined #tripleo | 17:14 | |
| pabelanger | mwhahaha: I'd expect them to already be in place | 17:14 |
| *** pkovar has quit IRC | 17:14 | |
| gfidente | pabelanger mwhahaha so the default tht templates skip these two tags https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/ceph-ansible/ceph-base.yaml#L60 | 17:14 |
| pabelanger | I am guessing the ceph-ansible playbooks is trying to manage all the things | 17:14 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: docker: Stop all active ceilometer services during compute upgrade https://review.openstack.org/493825 | 17:15 |
| gfidente | pabelanger mwhahaha so the entire task is skipped in regular deployments https://github.com/ceph/ceph-ansible/blob/9da848abaca0798fa627dcb09aefed6636da76ea/roles/ceph-common/tasks/main.yml#L44 | 17:16 |
| mwhahaha | gfidente: so why are we enabling that here? | 17:16 |
| gfidente | fultonj ^^ you know which package in particular was missing on pre-deployed servers which required us to unset skiptags? | 17:17 |
| *** achadha has joined #tripleo | 17:17 | |
| *** gbarros has quit IRC | 17:18 | |
| fultonj | gfidente: yes, it was the ceph-jewel repo | 17:18 |
| mwhahaha | we turned on all package installs because the ceph repo wasn't there? | 17:19 |
| gfidente | mwhahaha fultonj the ceph repo is already there | 17:19 |
| fultonj | mwhahaha: gfidente let's shorten the list to only what we need | 17:19 |
| *** achadha_ has joined #tripleo | 17:19 | |
| *** achadha has quit IRC | 17:19 | |
| *** gbarros has joined #tripleo | 17:19 | |
| gfidente | fultonj no my point is why can't we skip the tags entirely in upstream CI? | 17:19 |
| gfidente | did you find a particular package we needed to install which was not? | 17:20 |
| gfidente | because if not, I'd leave SkipTags to the default instead of overriding the list indeed | 17:20 |
| fultonj | gfidente: yes, i think it was the ceph-ansible pacakge. let me dig up what i found. | 17:20 |
| fultonj | gfidente: agreed | 17:20 |
| gfidente | well we can tentatively clear the parameter and see what fails | 17:21 |
| fultonj | sure | 17:21 |
| fultonj | gfidente: so you're going to update https://review.openstack.org/#/c/496806 to temporarily have the original skip tags and we'll see how it does? | 17:22 |
| gfidente | yeah | 17:22 |
| *** achadha has joined #tripleo | 17:22 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Do not install epel-release with ceph-ansible https://review.openstack.org/496806 | 17:22 |
| gfidente | mwhahaha pabelanger problem is the infra nodes are not pre-installed with tripleo baremetal images | 17:23 |
| gfidente | but let's see if anything is actually needed | 17:23 |
| fultonj | we could have a document to support ceph in a deployed-server scenario | 17:24 |
| mwhahaha | we should for split stack | 17:24 |
| gfidente | fultonj I think the problem is just figuring what package we eventually need | 17:24 |
| *** achadha_ has quit IRC | 17:24 | |
| mwhahaha | we need to know what packages we need to know | 17:24 |
| mwhahaha | because we need to be able to provide a list, we can't rely on ceph-ansible to just do magic | 17:25 |
| fultonj | gfidente: yes, then we put that shortend minimal list in the scenario's test and document it | 17:25 |
| *** achadha_ has joined #tripleo | 17:25 | |
| fultonj | mwhahaha: and we can get that done by passing it the list from tripleo | 17:25 |
| gfidente | fultonj but are you saying it's possible that ceph-mon/-osd wasn't installed at all? | 17:25 |
| *** aufi has quit IRC | 17:25 | |
| fultonj | gfidente: yes, this was deployed-servers IIRC | 17:26 |
| *** achadha_ has quit IRC | 17:26 | |
| fultonj | clarification: i know it's deployed servers, what ISTR is that those ceph packages were not there | 17:26 |
| weshay | any cores avail for an easy review? https://review.openstack.org/#/c/496822/ | 17:26 |
| pabelanger | I'm still confused what is coming from EPEL in this case | 17:26 |
| *** achadha_ has joined #tripleo | 17:26 | |
| gfidente | so that poses an additional problem, we need it to use the centos/sig repo eventually, still not epel | 17:26 |
| gfidente | pabelanger the RPMs from ceph.com require EPEL | 17:26 |
| pabelanger | gfidente: Ya | 17:26 |
| fultonj | pabelanger: agreed, i don't think we need EPEL or setuptools and we'll get them out | 17:27 |
| gfidente | so we need to have that as an alternative to ceph.com and not install epel-release in that case | 17:27 |
| mwhahaha | but we don't suppose the ceph.com packages | 17:27 |
| pabelanger | gfidente: right, but which dependency is getting pulled in? Doesn't RDO have a ceph package? | 17:27 |
| mwhahaha | s/suppose/support | 17:27 |
| *** achadha_ has quit IRC | 17:27 | |
| *** achadha has quit IRC | 17:27 | |
| fultonj | and we need it working with CentOS Storage sig, not ceph.com | 17:27 |
| pabelanger | ya, that is what I'd expect | 17:27 |
| gfidente | pabelanger not RDO explicitly, but in centos we maintain the storage sig repo | 17:27 |
| *** achadha has joined #tripleo | 17:27 | |
| pabelanger | k | 17:27 |
| pabelanger | sounds like it needs to be using http://mirror.dfw.rax.openstack.org/centos/7/storage/x86_64/ceph-jewel/ or http://mirror.dfw.rax.openstack.org/centos/7/storage/x86_64/ceph-hammer/ | 17:28 |
| *** achadha has quit IRC | 17:28 | |
| gfidente | one more thing is | 17:28 |
| fultonj | pabelanger: yes, jewel | 17:28 |
| fultonj | i thought when scneario001 was working it was using centos sig, i probably just failed to do this clean up | 17:28 |
| gfidente | those steps could be made entirely optional on containerized | 17:28 |
| fultonj | gfidente: that's what i thought too, but ceph-ansible needed to run ceph -s at the end that that command failed | 17:29 |
| gfidente | ahahahaa | 17:29 |
| *** aditya_r has quit IRC | 17:29 | |
| gfidente | no thing is simple here | 17:29 |
| fultonj | gfidente: thus, i enabled the repos early on, and then ceph-ansible 'yum installed' it so it could run that command | 17:29 |
| fultonj | ceph-ansible installed it because i dropped skip tags | 17:30 |
| *** jokke_ has joined #tripleo | 17:30 | |
| fultonj | we made 001 work, but we need to clean up the mess now | 17:30 |
| fultonj | the epel mess | 17:30 |
| gfidente | right so I'd say | 17:30 |
| *** aditya_r has joined #tripleo | 17:30 | |
| pabelanger | fultonj: it is much work for epel mess? | 17:31 |
| gfidente | we might actually *need* epel to install some deps for ceph.com | 17:31 |
| gfidente | until it supports centos/sig repo | 17:31 |
| fultonj | yes, but we're not using ceph.com | 17:31 |
| fultonj | it will work with centos sig repos | 17:31 |
| gfidente | well, but it's unsupported for now | 17:31 |
| gfidente | in ceph-ansible | 17:31 |
| pabelanger | Ya, I think the fix is use centos sig repo, which we mirror | 17:31 |
| gfidente | pabelanger do you have any clue if that repo is pre-installed on the infra nodes? | 17:32 |
| gfidente | or if oooq installs it? | 17:32 |
| mwhahaha | oooq installs it | 17:32 |
| pabelanger | ya, somebody installs it | 17:32 |
| fultonj | but this is deployed servers | 17:32 |
| *** dpawar has joined #tripleo | 17:32 | |
| *** mdnadeem has quit IRC | 17:32 | |
| *** itlinux has quit IRC | 17:32 | |
| pabelanger | https://review.openstack.org/496085/ cleans it up a bit | 17:32 |
| pabelanger | fultonj: what is deployed servers in this context? sorry I don't know much about tripleo testing | 17:33 |
| mwhahaha | pabelanger: multinode | 17:33 |
| mwhahaha | so we use split stack to deploy on the provisioned servers | 17:33 |
| fultonj | pabelanger: aka splitstack | 17:33 |
| *** jpena is now known as jpena|off | 17:33 | |
| fultonj | thus the overcloud image, with the packages preinstalled, isn't used so the packages are not there | 17:34 |
| pabelanger | all I know, is non OVB jobs, should have it setup | 17:34 |
| mwhahaha | pabelanger: when using that, tripleo installs the packages it needs where as in ovb we're using images so we skip installing packages | 17:34 |
| fultonj | +1 | 17:34 |
| *** rcernin has joined #tripleo | 17:34 | |
| *** dpawar has quit IRC | 17:34 | |
| *** achadha has joined #tripleo | 17:34 | |
| pabelanger | mwhahaha: k, so that process would need to also setup mirrors for AFS. I am hoping it does | 17:34 |
| fultonj | "tripleo installs the packages it needs" i thought i made a change so it would include ceph-jewel for scenario001, i will try to find it | 17:35 |
| mwhahaha | pabelanger: we do, oooq/tripleo-ci does it | 17:35 |
| pabelanger | k | 17:35 |
| *** achadha has quit IRC | 17:35 | |
| *** tongl has joined #tripleo | 17:35 | |
| mwhahaha | we just need to make sure the ceph repo gets installed (which it should) | 17:35 |
| *** achadha has joined #tripleo | 17:36 | |
| pabelanger | http://logs.openstack.org/82/490082/1/gate/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/0e3413d/logs/undercloud/etc/yum.repos.d/quickstart-centos-ceph-jewel.repo.txt.gz | 17:36 |
| pabelanger | http://logs.openstack.org/82/490082/1/gate/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/0e3413d/logs/subnode-2/etc/yum.repos.d/quickstart-centos-ceph-jewel.repo.txt.gz | 17:36 |
| pabelanger | both nodes have it setup | 17:36 |
| *** ykarel has quit IRC | 17:37 | |
| *** tosky has quit IRC | 17:37 | |
| gfidente | mwhahaha pabelanger yeah and we set ceph_origin: distro in tht | 17:37 |
| gfidente | so ceph-ansible isn't adding the ceph.com repo anyway https://github.com/ceph/ceph-ansible/blob/23f551a05fa948c0c7e291f863ebae2075661f40/roles/ceph-common/tasks/installs/install_on_redhat.yml#L16 | 17:37 |
| pabelanger | okay, cool | 17:38 |
| pabelanger | so dropping epel-release will stop yum.repo for epel getting added to the node | 17:38 |
| gfidente | yeah it brings us back to the initial patchset which overrides the list of packages | 17:38 |
| pabelanger | which, then breaks because yum client tries to get indexes | 17:38 |
| gfidente | we can probably trim down that list further | 17:38 |
| fultonj | +1 | 17:38 |
| pabelanger | but, I think mwhahaha has a good point. Why do we even need any of those dependencies now | 17:38 |
| pabelanger | ++ | 17:39 |
| *** gbarros has quit IRC | 17:39 | |
| gfidente | but we do have to set skiptags empty or it won't install the necessary ceph packages | 17:40 |
| mwhahaha | ok well let's just make sure we don't need extra deps | 17:40 |
| gfidente | also, confirmed the ceph.com repo is not there as intended http://logs.openstack.org/00/495900/3/gate/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/b09cf60/logs/undercloud/etc/yum.repos.d/ | 17:40 |
| mwhahaha | it's better to know what we actually need than just guessing | 17:40 |
| mwhahaha | i have a feeling we shouldn't need any of those deps | 17:41 |
| gfidente | mwhahaha right I assumed someone did that in ceph-ansible for a reason :D | 17:41 |
| *** dpawar has joined #tripleo | 17:41 | |
| mwhahaha | i don't :D | 17:41 |
| gfidente | but we can try a blank list | 17:41 |
| gfidente | ack | 17:41 |
| *** ebarrera has joined #tripleo | 17:42 | |
| fultonj | if we're using deployed-servers, then we expect puppet-nova and similar to install the nova packages | 17:42 |
| mwhahaha | let's try a blank list and add to it | 17:42 |
| mwhahaha | right i get that concept but these packages aren't related to ceph because those should be handled when we yum install the ceph packages | 17:42 |
| fultonj | blank_list++ | 17:42 |
| mwhahaha | the deps seems to be for the dev workflow | 17:42 |
| mwhahaha | not prod | 17:42 |
| fultonj | mwhahaha: yes, i agree | 17:43 |
| mwhahaha | so lets go with a blank list for now and if we need something just document it in the scenario | 17:43 |
| fultonj | my point was that we can't tell ceph-ansible to skip all installs | 17:43 |
| fultonj | sounds good to me | 17:43 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: WIP create default settings for environment types https://review.openstack.org/495407 | 17:43 |
| fultonj | right, we exclude non-ceph installs with a blank list and grow it minimally if at all | 17:44 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: Change devmode to deploy containerized services by default https://review.openstack.org/472412 | 17:44 |
| *** jcoufal has joined #tripleo | 17:46 | |
| *** trown|lunch is now known as trown | 17:46 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Do not install additional CentOS packages via ceph-ansible https://review.openstack.org/496806 | 17:48 |
| gfidente | ok this should do it then | 17:49 |
| gfidente | I think I am going AFK for today | 17:49 |
| gfidente | please pick it up should it fail! | 17:49 |
| mwhahaha | k | 17:49 |
| fultonj | gfidente: ok | 17:49 |
| *** gbarros has joined #tripleo | 17:49 | |
| gfidente | thanks for helping | 17:50 |
| gfidente | we can probably "port" some of it in ceph-ansible | 17:50 |
| gfidente | itself | 17:50 |
| *** itlinux has joined #tripleo | 17:50 | |
| pabelanger | okay, Hmm | 17:54 |
| pabelanger | nm, was looking at wrong location | 17:55 |
| *** thrash|biab is now known as thrash | 17:56 | |
| *** social has quit IRC | 17:56 | |
| pabelanger | weshay: https://review.openstack.org/496830/ is that the review to catch the regression from last night? | 17:57 |
| *** pblaho has quit IRC | 17:57 | |
| *** tosky has joined #tripleo | 17:58 | |
| trown | i missed it... what was the regression last night? | 18:00 |
| pabelanger | trown: https://review.openstack.org/496445/ | 18:01 |
| pabelanger | that broke the gate | 18:01 |
| pabelanger | specifically, scenario container jobs | 18:02 |
| *** gfidente has quit IRC | 18:04 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: WIP create default settings for environment types https://review.openstack.org/495407 | 18:04 |
| trown | pabelanger: ah thanks | 18:04 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Add containers periodic test https://review.openstack.org/475747 | 18:04 |
| trown | pabelanger: then ya weshay's project-config patch should prevent that in the future | 18:05 |
| pabelanger | trown: ya, basically, I am in favor of insta-reverts for things that break gate :) Then figure out why it was allowed to pass check. | 18:05 |
| pabelanger | trown: ++ | 18:05 |
| pabelanger | so far, things have been looking really good this weel | 18:06 |
| trown | ya | 18:06 |
| *** pkovar has joined #tripleo | 18:06 | |
| pabelanger | week* | 18:06 |
| trown | knock on wood just to be safe | 18:06 |
| pabelanger | Ya, I think most of the hard stuff is done | 18:06 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Change devmode to deploy containerized services by default https://review.openstack.org/472412 | 18:06 |
| *** catintheroof has quit IRC | 18:07 | |
| *** funzo has joined #tripleo | 18:07 | |
| *** pkovar has quit IRC | 18:08 | |
| *** gbarros_ has joined #tripleo | 18:09 | |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 18:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712599 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712621 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 18:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 18:10 |
| openstack | Launchpad bug 1712599 in tripleo "scenario001 fails because ceph-ansible is trying to install epel-release" [Critical,In progress] - Assigned to Giulio Fidente (gfidente) | 18:10 |
| openstack | Launchpad bug 1712621 in tripleo "ovb overcloud image build broken due to rubygem-fluent-plugin-secure-forward" [Critical,Triaged] | 18:10 |
| *** gbarros has quit IRC | 18:11 | |
| *** funzo has quit IRC | 18:12 | |
| *** jfrancoa has quit IRC | 18:12 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Add port security option for OVB with no network isolation https://review.openstack.org/496838 | 18:14 |
| *** yamahata has joined #tripleo | 18:16 | |
| *** apetrich has quit IRC | 18:18 | |
| *** ansmith__ has joined #tripleo | 18:19 | |
| itlinux | hi cschwede_: are you around? | 18:19 |
| *** apetrich has joined #tripleo | 18:20 | |
| mwhahaha | pabelanger: did you ever figure out the cause of http://logs.openstack.org/82/490082/1/gate/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container/0e3413d/logs/undercloud/home/jenkins/repo_setup.log.txt.gz#_2017-08-23_16_17_05 | 18:21 |
| *** aditya_r has quit IRC | 18:21 | |
| pabelanger | mwhahaha: ya, my patch will fix that. | 18:22 |
| mwhahaha | pabelanger: ok | 18:22 |
| pabelanger | mwhahaha: I think the curl request is 404 | 18:22 |
| *** aditya_r has joined #tripleo | 18:22 | |
| pabelanger | and we get empty data | 18:22 |
| pabelanger | my patch will now validate and error if empty | 18:22 |
| *** fpan has quit IRC | 18:23 | |
| *** ebarrera has quit IRC | 18:23 | |
| *** dixiaoli has joined #tripleo | 18:24 | |
| *** dixiaoli has quit IRC | 18:28 | |
| *** oidgar has joined #tripleo | 18:29 | |
| *** ansmith has quit IRC | 18:31 | |
| *** ansmith_ has quit IRC | 18:32 | |
| *** aditya_r has quit IRC | 18:34 | |
| *** gbarros_ has quit IRC | 18:35 | |
| *** jcoufal has quit IRC | 18:36 | |
| *** jcoufal has joined #tripleo | 18:38 | |
| *** cschwede_ has quit IRC | 18:40 | |
| *** brault has quit IRC | 18:44 | |
| *** rcernin has quit IRC | 18:47 | |
| *** ansmith_ has joined #tripleo | 18:49 | |
| *** ansmith has joined #tripleo | 18:49 | |
| mwhahaha | owalsh: you around? | 18:49 |
| pabelanger | weshay: trown: do you mind looking at 496085 simplifies mirror setup | 18:51 |
| pabelanger | also adds some error handling | 18:51 |
| *** jtomasek_ has quit IRC | 18:52 | |
| trown | pabelanger: sure | 18:53 |
| *** akrivoka has quit IRC | 18:54 | |
| *** fpan has joined #tripleo | 18:55 | |
| *** shardy has joined #tripleo | 18:57 | |
| tongl | if I want to use upstream master, is "master" the right --release option for quickstart.sh? | 19:05 |
| *** dpawar has quit IRC | 19:05 | |
| tongl | also, if anything goes wrong for quickstart, where can I look at the relevant logs? | 19:06 |
| openstackgerrit | Alex Schultz proposed openstack/instack-undercloud master: Disable compute auto disabling https://review.openstack.org/496851 | 19:07 |
| *** ansmith has quit IRC | 19:09 | |
| *** itlinux has quit IRC | 19:10 | |
| *** ansmith has joined #tripleo | 19:10 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712599 | 19:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712621 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 19:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 19:10 |
| openstack | Launchpad bug 1712599 in tripleo "scenario001 fails because ceph-ansible is trying to install epel-release" [Critical,In progress] - Assigned to Giulio Fidente (gfidente) | 19:10 |
| openstack | Launchpad bug 1712621 in tripleo "ovb overcloud image build broken due to rubygem-fluent-plugin-secure-forward" [Critical,Triaged] | 19:10 |
| *** oidgar has quit IRC | 19:11 | |
| *** ansmith_ has quit IRC | 19:13 | |
| *** eck` is now known as eck`gone | 19:16 | |
| pabelanger | weshay: 496830 merged, but no containers-scenario003 jobs being run. I believe that is because you need to update the files regex in zuul/layouts.yaml | 19:25 |
| weshay | hrm. | 19:26 |
| * weshay looks | 19:26 | |
| weshay | in 1-1, so slowly looking | 19:26 |
| *** ansmith_ has joined #tripleo | 19:26 | |
| pabelanger | weshay: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n1799 will need to atleast match quickstart-extras files | 19:27 |
| pabelanger | likely something like roles/.* | 19:27 |
| *** ebarrera has joined #tripleo | 19:28 | |
| *** paramite has quit IRC | 19:30 | |
| weshay | pabelanger, juse roles/.* would work? | 19:32 |
| pabelanger | weshay: likely roles and playbooks directories | 19:34 |
| pabelanger | weshay: same for ci-scripts and config, what ever files apply to scenerio3 need to be listed | 19:35 |
| pabelanger | you can see why I am not a fan, you end up building a large static list | 19:35 |
| weshay | k | 19:36 |
| weshay | looks a bit more | 19:36 |
| weshay | thanks for the hand | 19:36 |
| mwhahaha | roles/ would also trigger on THT which might not be ideal | 19:37 |
| pabelanger | ya, you'd have to make the regex smarter to skip tripleo-heat-templates | 19:38 |
| weshay | pabelanger, w/ can you use a skip-if w/ project and files just from that project? | 19:39 |
| pabelanger | weshay: you'd need a unique job name for aside project | 19:41 |
| pabelanger | said* | 19:41 |
| *** pchavva has quit IRC | 19:41 | |
| weshay | trown, do you have any suggestions here? | 19:41 |
| pabelanger | zuulv3 changes is actually. Really not sure how much still applies | 19:41 |
| weshay | an alternative is to list out each tqe role/foo role/bar | 19:42 |
| *** itlinux has joined #tripleo | 19:43 | |
| weshay | seems sloppy, but it's the only thing that is clear to me atm | 19:43 |
| pabelanger | k, looks like we do still support it | 19:43 |
| weshay | pabelanger, and btw.. I'd need to do the same for https://review.openstack.org/#/c/496831/ | 19:44 |
| pabelanger | you'd need to match the / in the sub directory name for roles | 19:44 |
| pabelanger | tripleo-heat-templates doesn't have any sub directories | 19:45 |
| weshay | ah cool | 19:45 |
| mwhahaha | yea | 19:45 |
| weshay | pretty lucky | 19:46 |
| pabelanger | roles\/(.*\/) | 19:46 |
| pabelanger | or something like that | 19:46 |
| *** itlinux has quit IRC | 19:47 | |
| weshay | ya.. playing w/ it now.. /me tests that | 19:49 |
| weshay | pabelanger, that passes | 19:50 |
| *** eck`gone is now known as eck` | 19:51 | |
| *** sbrzozow has quit IRC | 19:55 | |
| *** shardy has quit IRC | 20:00 | |
| *** itlinux has joined #tripleo | 20:01 | |
| *** tongl has quit IRC | 20:01 | |
| *** funzo has joined #tripleo | 20:05 | |
| owalsh | mwhahaha, what's up? | 20:08 |
| mwhahaha | owalsh: nm, was going to ask about a bug but I found consecutive_build_service_disable_threshold | 20:09 |
| *** funzo has quit IRC | 20:10 | |
| mwhahaha | owalsh: just fyi, https://bugs.launchpad.net/tripleo/+bug/1712632 which explains why i've randomly seen the undercloud's nova-compute disabled. I've proposed disabling that for our undercloud | 20:10 |
| openstack | Launchpad bug 1712632 in tripleo "Check that enough nodes are in "available" state with maintenance mode off error" [High,In progress] - Assigned to Alex Schultz (alex-schultz) | 20:10 |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712599 | 20:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712621 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 20:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 20:10 |
| openstack | Launchpad bug 1712599 in tripleo "scenario001 fails because ceph-ansible is trying to install epel-release" [Critical,In progress] - Assigned to Giulio Fidente (gfidente) | 20:10 |
| openstack | Launchpad bug 1712621 in tripleo "ovb overcloud image build broken due to rubygem-fluent-plugin-secure-forward" [Critical,Triaged] | 20:10 |
| owalsh | mwhahaha, yes, vkmc hit that the other day when the undercloud ran out of disk space | 20:11 |
| mwhahaha | it's a really bad UX when that happens on the undercloud :/ | 20:12 |
| owalsh | mwhahaha, I think it would be better if the validation reported that the compute service had been auto-disabled | 20:12 |
| mwhahaha | owalsh: we should have both but I don't think we should auto disable in the undercloud case | 20:12 |
| mwhahaha | aka ComputeFilter failures are really crappy | 20:13 |
| weshay | pabelanger, mwhahaha https://review.openstack.org/496862 | 20:13 |
| weshay | and btw.. this is a cool site http://regexr.com/ | 20:13 |
| owalsh | mwhahaha, that was my initial reaction but if we get 10 consecutive build failures then something ain't right | 20:14 |
| *** itlinux has quit IRC | 20:14 | |
| mwhahaha | owalsh: yea but i've had it happen without build filures | 20:14 |
| owalsh | mwhahaha, then I think it's broken | 20:14 |
| mwhahaha | owalsh: it's possible, so given current cycle i'd like to disable it for now. so i've proposed patches for that now to puppet-nova/instack-undercloud | 20:15 |
| *** Goneri has quit IRC | 20:15 | |
| *** ansmith has quit IRC | 20:17 | |
| *** ansmith_ has quit IRC | 20:17 | |
| *** itlinux has joined #tripleo | 20:17 | |
| owalsh | mwhahaha, ah, that bug came from vkmc... it think we are doomed when the undercloud runs out of space so it's not going to help much but I agree the usability sucks and it took us quite a while to figure that one out | 20:19 |
| mwhahaha | yea it took me a few hours the first time it happened too | 20:19 |
| mwhahaha | so i'd like not to subject end users or some poor support folks to that | 20:19 |
| owalsh | mwhahaha, ok with me if it's disabled, deploy will fail eventually in any case.. but maybe a tech debt bug to catch this condition in the pre deploy validation for queens? | 20:20 |
| mwhahaha | owalsh: yea that would make sense | 20:20 |
| *** jkilpatr has quit IRC | 20:22 | |
| *** toure is now known as toure_biab | 20:26 | |
| *** dprince has quit IRC | 20:26 | |
| *** rcernin has joined #tripleo | 20:36 | |
| *** achadha_ has joined #tripleo | 20:38 | |
| *** achadha has quit IRC | 20:38 | |
| rcernin | shadower: got minute? | 20:39 |
| rcernin | shadower: about ooo-validations | 20:39 |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Add missing ceph-ansible install to appropriate releases https://review.openstack.org/496822 | 20:40 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Configure listen_address for libvirtd when TLS is enabled https://review.openstack.org/496492 | 20:40 |
| *** itlinux has quit IRC | 20:41 | |
| *** ecerquei has quit IRC | 20:41 | |
| *** achadha_ has quit IRC | 20:42 | |
| *** jcoufal has quit IRC | 20:43 | |
| *** achadha has joined #tripleo | 20:47 | |
| *** ffiore has joined #tripleo | 20:57 | |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: Add option to build $RELEASE version of containers images https://review.openstack.org/496920 | 20:58 |
| *** trown is now known as trown|outtypewww | 21:01 | |
| *** ffiore has quit IRC | 21:06 | |
| *** fpan has quit IRC | 21:08 | |
| radez | mwhahaha: is the control vip on a single control node deploy brought up by os-net-config? | 21:08 |
| radez | my more recent issue is the ip on the port labeled control vip doesn't come up. the db is listening there so nothing can connect to the db | 21:09 |
| openstackgerrit | Alan Bishop proposed openstack/tripleo-heat-templates master: WIP: Resolve dependency issues prior to doing yum update https://review.openstack.org/496921 | 21:09 |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 21:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712599 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712621 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 21:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 21:10 |
| openstack | Launchpad bug 1712599 in tripleo "scenario001 fails because ceph-ansible is trying to install epel-release" [Critical,In progress] - Assigned to Giulio Fidente (gfidente) | 21:10 |
| openstack | Launchpad bug 1712621 in tripleo "ovb overcloud image build broken due to rubygem-fluent-plugin-secure-forward" [Critical,Triaged] | 21:10 |
| mwhahaha | radez: i don't know | 21:11 |
| radez | kk | 21:12 |
| *** SlickNik has left #tripleo | 21:17 | |
| *** SlickNik has joined #tripleo | 21:18 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart master: Enable opstools-testing repo https://review.openstack.org/496923 | 21:20 |
| mwhahaha | larsks, dmsimard -^ | 21:21 |
| *** leitan has quit IRC | 21:22 | |
| *** itlinux has joined #tripleo | 21:25 | |
| *** ffiore has joined #tripleo | 21:25 | |
| *** bfournie has quit IRC | 21:25 | |
| itlinux | is the tripleo default selinux now permissive? I thought it was set to enforced | 21:28 |
| *** cdearborn has quit IRC | 21:28 | |
| mwhahaha | upstream has always been permissive, downstream is enforced | 21:28 |
| *** atoth has quit IRC | 21:29 | |
| mwhahaha | because openstack-selinux isn't always up to date with the upstream changes | 21:29 |
| *** jkilpatr has joined #tripleo | 21:29 | |
| itlinux | ahh ok.. thanks mwhahaha.:. so this can be changed once deployed TY | 21:33 |
| *** abishop has quit IRC | 21:37 | |
| *** fpan has joined #tripleo | 21:37 | |
| *** morazi has quit IRC | 21:42 | |
| *** tongl has joined #tripleo | 21:43 | |
| *** ffiore_ has joined #tripleo | 21:51 | |
| *** ffiore has quit IRC | 21:52 | |
| *** jlabarre has quit IRC | 21:53 | |
| *** ffiore_ has quit IRC | 21:54 | |
| openstackgerrit | Rajini Karthik proposed openstack/tripleo-heat-templates master: Add support for Dell EMC Isilon Manila backend https://review.openstack.org/488896 | 22:04 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-quickstart-extras master: Filter image list by deployed services https://review.openstack.org/496950 | 22:04 |
| *** ebarrera has quit IRC | 22:10 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 22:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712599 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712621 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 22:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 22:10 |
| openstack | Launchpad bug 1712599 in tripleo "scenario001 fails because ceph-ansible is trying to install epel-release" [Critical,In progress] - Assigned to Giulio Fidente (gfidente) | 22:10 |
| openstack | Launchpad bug 1712621 in tripleo "ovb overcloud image build broken due to rubygem-fluent-plugin-secure-forward" [Critical,Triaged] | 22:10 |
| openstackgerrit | Lukas Bezdicka proposed openstack/puppet-tripleo master: HAproxy should get full response from ironic-inspector https://review.openstack.org/496954 | 22:17 |
| openstackgerrit | Rajini Karthik proposed openstack/puppet-tripleo master: Support for Dell EMC VNX Manila Driver https://review.openstack.org/490642 | 22:19 |
| *** achadha_ has joined #tripleo | 22:26 | |
| *** limao has joined #tripleo | 22:27 | |
| *** achadha__ has joined #tripleo | 22:28 | |
| *** rlandy is now known as rlandy|bbl | 22:28 | |
| *** achadha has quit IRC | 22:29 | |
| *** achadha__ has quit IRC | 22:29 | |
| *** itlinux has quit IRC | 22:29 | |
| *** achadha has joined #tripleo | 22:30 | |
| *** achadha_ has quit IRC | 22:30 | |
| *** leitan has joined #tripleo | 22:32 | |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Clean up repo_cmd_before command for setup-repo https://review.openstack.org/496085 | 22:33 |
| *** leitan has quit IRC | 22:36 | |
| *** dparkes has joined #tripleo | 22:37 | |
| *** dparkes has quit IRC | 22:45 | |
| *** ebarrera has joined #tripleo | 22:49 | |
| *** achadha_ has joined #tripleo | 22:51 | |
| *** achadha_ has quit IRC | 22:53 | |
| *** achadha_ has joined #tripleo | 22:53 | |
| *** achadha has quit IRC | 22:54 | |
| *** achadha has joined #tripleo | 22:55 | |
| *** achadha_ has quit IRC | 22:57 | |
| *** achadha has quit IRC | 23:00 | |
| *** achadha has joined #tripleo | 23:00 | |
| *** achadha_ has joined #tripleo | 23:06 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1691403 | 23:10 |
| openstack | Launchpad bug 1691403 in tripleo "containerized overcloud - neutron-openvswitchagent fails to start" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1702955 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712436 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712599 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1712621 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1702955 in tripleo "tripleo upgrade jobs timeout on stable/ocata" [Critical,Triaged] | 23:10 |
| openstack | Launchpad bug 1712436 in tripleo "stable/newton gate-tripleo-ci-centos-7-nonha-multinode-oooq is broken due to Controller create timeout" [Critical,Triaged] | 23:10 |
| openstack | Launchpad bug 1712599 in tripleo "scenario001 fails because ceph-ansible is trying to install epel-release" [Critical,In progress] - Assigned to Giulio Fidente (gfidente) | 23:10 |
| openstack | Launchpad bug 1712621 in tripleo "ovb overcloud image build broken due to rubygem-fluent-plugin-secure-forward" [Critical,Triaged] | 23:10 |
| *** achadha has quit IRC | 23:10 | |
| *** achadha_ has quit IRC | 23:11 | |
| *** athomas has quit IRC | 23:11 | |
| *** limao_ has joined #tripleo | 23:16 | |
| *** limao has quit IRC | 23:17 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: More fixes for the Ceph docker images url parsing https://review.openstack.org/495374 | 23:24 |
| *** tosky has quit IRC | 23:25 | |
| *** rbowen has quit IRC | 23:28 | |
| *** tzumainn has quit IRC | 23:32 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements https://review.openstack.org/487398 | 23:37 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements https://review.openstack.org/488148 | 23:37 |
| *** rbowen has joined #tripleo | 23:45 | |
| *** tongl has quit IRC | 23:52 | |
| *** bnemec has joined #tripleo | 23:54 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!