| ayoung | OK...quickstart with --extra-vars @config/general_config/devmode.yml lead to a undercloud install failuer...RPM versions. What is the straightest path to get a Newton-based deployment setup? I assume removing the installed RPMS and setting up the proper repos....any guidance? | 00:08 |
|---|---|---|
| ayoung | ah...should have done | 00:09 |
| ayoung | --release master-tripleo \ | 00:09 |
| ayoung | hmmm | 00:09 |
| sshnaidm|afk | ayoung, https://github.com/openstack/tripleo-quickstart/blob/master/ci-scripts/devmode_tempest.sh#L19-L27 | 00:11 |
| *** maeca1 has quit IRC | 00:16 | |
| *** limao has joined #tripleo | 00:28 | |
| ayoung | sshnaidm|afk, thanks, but I think the problem is with the yum repos | 00:30 |
| ayoung | I think I am getting a mismatched set. I got this error again: | 00:30 |
| ayoung | Error: Package: openstack-swift-proxy-2.7.1-0.20160613173619.d75e6d3.el7.centos.noarch (@delorean) | 00:31 |
| ayoung | Requires: openstack-swift = 2.7.1-0.20160613173619.d75e6d3.el7.centos | 00:31 |
| ayoung | Removing: openstack-swift-2.7.1-0.20160613173619.d75e6d3.el7.centos.noarch (@delorean) | 00:31 |
| ayoung | openstack-swift = 2.7.1-0.20160613173619.d75e6d3.el7.centos | 00:31 |
| ayoung | Obsoleted By: python-swift-2.9.1-0.20160819204243.29d13b7.el7.centos.noarch (delorean-tripleo) | 00:31 |
| ayoung | openstack-swift = 2.9.1-0.20160819204243.29d13b7.el7.centos | 00:31 |
| sshnaidm|afk | hmm.. repos are constructed here: https://github.com/openstack/tripleo-quickstart/blob/master/roles/libvirt/setup/undercloud/tasks/update_image.yml#L9-L14 | 00:32 |
| sshnaidm|afk | check that you really have the right ones | 00:32 |
| sshnaidm|afk | ayoung, oh, check if you have delorean.repo file, most likely it's from old image and should be deleted | 00:34 |
| sshnaidm|afk | /etc/yum.repos.d/delorean.repo | 00:34 |
| ayoung | sshnaidm|afk, yep, they are different. Thanks | 00:38 |
| *** mgarciam has left #tripleo | 00:39 | |
| *** mgarciam has quit IRC | 00:39 | |
| ayoung | sshnaidm|afk, I think it was that a delorean.repo file was left behind. I removed that. It looks like that script only creates delorean-*.repo file names... | 00:44 |
| ayoung | and, now that I reread what you wrote, yes, that is what i saw | 00:44 |
| sshnaidm|afk | ayoung, you can vote for the fix :) https://review.openstack.org/#/c/335953/ | 00:46 |
| ayoung | sshnaidm|afk, does that leave the delorean.repo file name alone? | 00:48 |
| sshnaidm|afk | ayoung, the fix will overwrite it | 00:49 |
| ayoung | KeyError: 'GENERATE_SERVICE_CERTIFICATE' | 00:49 |
| *** bana_k has quit IRC | 00:51 | |
| sshnaidm|afk | ayoung, in undercloud? | 00:51 |
| ayoung | sshnaidm|afk, yep | 00:59 |
| ayoung | ERROR: 2016-08-23 00:48:12,756 -- Hook FAILED. | 00:59 |
| ayoung | ERROR: 2016-08-23 00:48:12,757 -- Failed running command ['dib-run-parts', u'/tmp/tmp4KY3dE/install.d'] | 00:59 |
| sshnaidm|afk | ayoung, can you paste all log? | 01:00 |
| ayoung | sure | 01:00 |
| ayoung | http://paste.openstack.org/show/562321/ sshnaidm|afk | 01:00 |
| sshnaidm|afk | hmm.. is it from tripleo-quickstart run? | 01:02 |
| *** saneax is now known as saneax-_-|AFK | 01:02 | |
| *** chem has quit IRC | 01:02 | |
| ayoung | sshnaidm|afk, yes | 01:07 |
| ayoung | sshnaidm|afk, Certs makes me suspect jaosorio's stuff. | 01:08 |
| sshnaidm|afk | ayoung, weird, just now ran it, was installing ok | 01:08 |
| ayoung | sshnaidm|afk, I wiped out the old VMs, so it was not a vestige. But maybe I still have some RPMs from the early run. Let me try killing it and applying your patch | 01:09 |
| sshnaidm|afk | ayoung, afaik ssl on undercloud is not merged yet: https://review.openstack.org/#/c/356431/ shouldn't be a problem.. | 01:10 |
| sshnaidm|afk | ayoung, for using "from scratch" images, you can use --extra-vars force_cached_images=False | 01:12 |
| *** chem has joined #tripleo | 01:12 | |
| sshnaidm|afk | it will take them from server | 01:12 |
| ayoung | sshnaidm|afk, is that what you use? | 01:12 |
| sshnaidm|afk | ayoung, usually after promotion | 01:12 |
| ayoung | sshnaidm|afk, Congrats, you are now a Colonel | 01:13 |
| sshnaidm|afk | :)) | 01:13 |
| *** Ryjedo_ has joined #tripleo | 01:22 | |
| *** Ryjedo has quit IRC | 01:24 | |
| *** Ryjedo_ is now known as Ryjedo | 01:24 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Add memory to overcloud vms up to 6144 https://review.openstack.org/357532 | 01:25 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: POC: WIP: oooq undercloud install https://review.openstack.org/358919 | 01:26 |
| *** alee has joined #tripleo | 01:37 | |
| alee | ayoung, hey -- so magic incantations for quickstart | 01:37 |
| ayoung | alee, here is what I have so far | 01:38 |
| alee | ayoung, I'm running whats on master in tripleo-quickstart and just pointing to my machine on which to install | 01:38 |
| alee | its failing in "Get undercloud vm ip address" | 01:38 |
| ayoung | bash quickstart.sh --release master-tripleo --extra-vars @config/general_config/devmode.yml --extra-vars force_cached_images=False ayoung-dell-t1700.test | 01:39 |
| ayoung | alee, localhost install? | 01:39 |
| ayoung | There was something special for that. | 01:39 |
| alee | ayoung, no remote .. | 01:39 |
| ayoung | OK, so, I have my remote machine's IP addres in /etc/hosts | 01:39 |
| *** coolsvap has quit IRC | 01:39 | |
| ayoung | that is ayoung-dell-t1700.test | 01:39 |
| alee | ayoung, well I'm using a beaker lab machine .. | 01:39 |
| ayoung | --release master-tripleo gets you working with master. Otherwise defaults to Mitala | 01:40 |
| ayoung | Mitaka | 01:40 |
| alee | but I can put it in /etc/hosts to see if that makes a difference | 01:40 |
| ayoung | alee, nah, so long as it resolves | 01:40 |
| ayoung | I did it with a Beaker machine, too. warp.... | 01:40 |
| alee | yeah - its resolving .. | 01:40 |
| alee | ayoung, what does "force_cached_images=False" do? | 01:41 |
| ayoung | alee, according to sshnaidm|afk who told me to do that: "for using "from scratch" images, you can use --extra-vars force_cached_images=False. it will take them from server. " | 01:43 |
| ayoung | alee, and then I promoted him to Colonel | 01:44 |
| alee | ayoung, ok -trying with your magic incarnation | 01:45 |
| ayoung | alee, I would like to point out that my install is still running, and I have not gotten a successful run yet, either. So you have now caught up to me | 01:45 |
| *** dmacpher has quit IRC | 01:46 | |
| alee | ayoung, :/ | 01:49 |
| ayoung | alee, undercloud install underway... | 01:49 |
| *** saneax-_-|AFK is now known as saneax | 01:51 | |
| *** dmacpher has joined #tripleo | 01:59 | |
| *** hk_ has joined #tripleo | 02:14 | |
| *** yamahata has quit IRC | 02:15 | |
| hk_ | hello all | 02:19 |
| hk_ | exit | 02:23 |
| hk_ | sorry | 02:23 |
| *** hk_ has quit IRC | 02:23 | |
| ayoung | alee, I missed a big one | 02:26 |
| ayoung | I applied this patch | 02:27 |
| ayoung | https://review.openstack.org/#/c/335953/4 | 02:27 |
| ayoung | without that, my undercloud install failed | 02:27 |
| alee | ayoung, ah | 02:27 |
| alee | ayoung, did it succeed this time? | 02:28 |
| ayoung | alee, I think so. | 02:28 |
| ayoung | I have flavors, anyway | 02:28 |
| ayoung | and it looks like the undercloud-post-install.sh ran | 02:28 |
| alee | mine hasn't failed yet -- but I'm sure it will .. | 02:28 |
| ayoung | alee, the issue was that there was two Yum repos, both with the sameish priorities | 02:29 |
| ayoung | one name delorean.repo, the other delorean-tripleo.repo | 02:29 |
| ayoung | but the first one was an old one, and some old packages got installed. | 02:29 |
| *** saneax is now known as saneax-_-|AFK | 02:30 | |
| ayoung | alee, I think I am going to try an overcloud install, see what happens | 02:30 |
| alee | :) | 02:30 |
| *** hk_ has joined #tripleo | 02:32 | |
| *** hk_ has quit IRC | 02:38 | |
| *** ramishra has quit IRC | 02:40 | |
| ayoung | alee, OK, so one other thing. THe setup only creates 2 nodes. I think in order to have an overcloud deploy I need to tell it to only try and user those two | 02:46 |
| ayoung | it was... | 02:47 |
| ayoung | parameters: | 02:47 |
| ayoung | ControllerCount: 1 | 02:47 |
| ayoung | at the end of deploy-env.yml | 02:47 |
| ayoung | and I think I am just going to sneak that into the overcloud deploy script | 02:47 |
| ayoung | actually added an additional -e $PWD/deploy-env.yml param to the call and added ^^ to that (new) file | 02:49 |
| *** jdob has quit IRC | 02:54 | |
| *** bana_k has joined #tripleo | 03:06 | |
| *** ayoung has quit IRC | 03:14 | |
| *** ebalduf has joined #tripleo | 03:20 | |
| *** tzumainn has quit IRC | 03:26 | |
| *** Goneri has quit IRC | 03:31 | |
| *** yamahata has joined #tripleo | 03:40 | |
| *** bana_k has quit IRC | 03:42 | |
| *** Jaison has joined #tripleo | 04:15 | |
| *** Jaison has quit IRC | 04:24 | |
| *** jraju has joined #tripleo | 04:26 | |
| *** jobewan has joined #tripleo | 04:26 | |
| *** assassin has quit IRC | 04:32 | |
| *** jobewan has quit IRC | 04:34 | |
| *** jtomasek has quit IRC | 04:42 | |
| *** fragatina has quit IRC | 04:59 | |
| *** jprovazn has joined #tripleo | 05:02 | |
| *** jprovazn has quit IRC | 05:02 | |
| *** jprovazn has joined #tripleo | 05:03 | |
| *** masco has joined #tripleo | 05:06 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts https://review.openstack.org/357765 | 05:09 |
| *** jaosorior has joined #tripleo | 05:10 | |
| *** aqkhan__ has joined #tripleo | 05:23 | |
| *** aqkhan_ has quit IRC | 05:24 | |
| *** aqkhan_ has joined #tripleo | 05:24 | |
| *** aqkhan__ has quit IRC | 05:29 | |
| *** bkopilov has joined #tripleo | 05:29 | |
| *** numans has joined #tripleo | 05:40 | |
| *** r-mibu has quit IRC | 05:52 | |
| *** bana_k has joined #tripleo | 05:53 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Move resource registry override to enable-tls.yaml https://review.openstack.org/358971 | 05:56 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: DO NOT MERGE - TESTING BARBICAN https://review.openstack.org/353262 | 06:01 |
| *** fragatina has joined #tripleo | 06:04 | |
| *** fragatina has quit IRC | 06:05 | |
| *** fragatina has joined #tripleo | 06:07 | |
| *** fragatina has quit IRC | 06:07 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: DO NOT MERGE - TESTING BARBICAN https://review.openstack.org/353262 | 06:08 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add Barbican to the overcloud - WIP https://review.openstack.org/352458 | 06:08 |
| *** fragatina has joined #tripleo | 06:09 | |
| *** numans has quit IRC | 06:11 | |
| *** r-mibu has joined #tripleo | 06:12 | |
| *** fragatina has quit IRC | 06:14 | |
| *** pcaruana has joined #tripleo | 06:14 | |
| *** tobias_fiberdata has joined #tripleo | 06:20 | |
| jaosorior | ccamacho: ping | 06:22 |
| *** jbadiapa has joined #tripleo | 06:24 | |
| openstackgerrit | Saravanan KR proposed openstack/os-net-config: Add support for OVS DPDK Bridge and Port https://review.openstack.org/338694 | 06:32 |
| *** mcornea has joined #tripleo | 06:32 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add HAProxy TLS handled by certmonger as composable service https://review.openstack.org/356430 | 06:33 |
| *** abregman has joined #tripleo | 06:37 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add HAProxy TLS handled by certmonger as composable service https://review.openstack.org/356430 | 06:39 |
| *** ubijtsa has joined #tripleo | 06:40 | |
| *** tvignaud has quit IRC | 06:40 | |
| *** ubijtsa is now known as Guest11182 | 06:40 | |
| *** tvignaud has joined #tripleo | 06:40 | |
| *** Guest11182 has quit IRC | 06:41 | |
| *** Guest11182 has joined #tripleo | 06:41 | |
| *** Guest11182 is now known as assassin | 06:42 | |
| *** saneax-_-|AFK is now known as saneax | 06:45 | |
| *** mhenkel has joined #tripleo | 06:49 | |
| *** florianf has joined #tripleo | 06:55 | |
| *** yolanda has quit IRC | 06:56 | |
| *** tremble has joined #tripleo | 06:58 | |
| *** tremble has joined #tripleo | 06:58 | |
| jaosorior | bandini: it seems that when the haproxy configuration is updated in an HA deployment. HAProxy is not reloading the configuration :/ | 07:00 |
| bandini | jaosorior: how do you update the configuration? i.e. is pacemaker_resource_restart not triggering or something else? | 07:02 |
| *** TSCHAK has joined #tripleo | 07:02 | |
| jaosorior | bandini: so, from a non-ssl I updated to an SSL configuration | 07:03 |
| jaosorior | and haproxy is still serving the non-ssl configuration. So Step 3 of the post-deploy (puppet) fails | 07:03 |
| *** yolanda has joined #tripleo | 07:04 | |
| bandini | how do you update a non-ssl conf to an ssl config? just change some environment variables and launch overcloud redeploy? | 07:04 |
| jaosorior | bandini: yep | 07:04 |
| *** dsariel has joined #tripleo | 07:04 | |
| *** bana_k has quit IRC | 07:05 | |
| bandini | jaosorior: and pacemaker_resource_restart.sh is only triggered if we do a stack update, not when we redeploy against an existing overcloud, correct? | 07:05 |
| *** vijaykc4 has joined #tripleo | 07:06 | |
| jaosorior | bandini: I think it's also triggered on stack update | 07:06 |
| jaosorior | bandini: but I think it might be triggered too late | 07:07 |
| bandini | ah I see | 07:07 |
| jaosorior | bandini: indeed | 07:08 |
| jaosorior | bandini: it's ran after all the puppet stuff is ran | 07:08 |
| jaosorior | bandini: so in my case it doesn't get triggered cause puppet fails itself | 07:09 |
| bandini | jaosorior: what are our options? move it earlier on? add another script that driggers early on? | 07:09 |
| jaosorior | bandini: I'm not sure. I tried to manually run pcs resource restart haproxy-clone | 07:10 |
| jaosorior | and that didn't do anything | 07:10 |
| bandini | mmh that does not sound right | 07:11 |
| jaosorior | bandini: however, doing manually systemd restart helped | 07:12 |
| jaosorior | pcs resource restart just had a success but didn't do crap :( | 07:12 |
| *** bootsha has joined #tripleo | 07:12 | |
| *** milan has joined #tripleo | 07:12 | |
| jaosorior | bandini: do we disable refreshes to haproxy in puppet? Can we set it do systemctl reloads instead? | 07:13 |
| jaosorior | jistr: are youa round? | 07:15 |
| bandini | jaosorior: so pcs resource restart haproxy-clone works fine here on a newton system | 07:15 |
| jaosorior | bandini: I'm on a newton system | 07:16 |
| bandini | jaosorior: this is what I get in the logs after restarting via pcs : https://paste.fedoraproject.org/412735/36562147/ | 07:16 |
| bandini | jaosorior: do you have some logs, more info? | 07:16 |
| jaosorior | bandini: no logs | 07:16 |
| jaosorior | bandini: nothing went on the logs | 07:16 |
| bandini | yes we do inhibit the restart of pacemaker resources via puppet | 07:16 |
| jaosorior | bandini: it just printed success and that's it | 07:17 |
| bandini | jaosorior: so you launch "pcs resource restart haproxy-clone" via shell and it does nothing? | 07:17 |
| jaosorior | no restarting of haprox | 07:17 |
| jaosorior | as fucked up as that sounds | 07:17 |
| bandini | can you get me the "pcs cluster cib /tmp/cib.xml" (I need /tmp/cib.xml) | 07:17 |
| openstackgerrit | Saravanan KR proposed openstack/os-net-config: Add support for OVS DPDK Bond https://review.openstack.org/349183 | 07:18 |
| jaosorior | bandini: will do | 07:18 |
| *** tesseract- has joined #tripleo | 07:18 | |
| bandini | jaosorior: thanks, happy to take a look there | 07:18 |
| jaosorior | bandini: re-running the deploy now (after manually restarting haproxy with systemd) | 07:18 |
| openstackgerrit | Merged openstack/tripleo-ui: Move plan upload errors from notifications to formerrors https://review.openstack.org/332181 | 07:19 |
| bandini | jaosorior: the only quick explanation I can think of is that "pcs property |grep maintenance-mode" returns "true" | 07:21 |
| bandini | anything else is rather weird | 07:21 |
| jaosorior | bandini: I see | 07:22 |
| jaosorior | bandini: didn't we set maintenance-mode for updates? | 07:23 |
| bandini | jaosorior: I don't think we do. (we will do for the migration from Full HA to NG HA) | 07:23 |
| bandini | jaosorior: do you have it set to true? | 07:23 |
| jaosorior | bandini: deploy is still on-going | 07:24 |
| jaosorior | don't wanna mess much with it cause I constantly get OOMs if I do small extra things... | 07:24 |
| jaosorior | my deployment is fragile as fuck | 07:24 |
| bandini | si que horror ;) | 07:25 |
| *** bana_k has joined #tripleo | 07:25 | |
| *** ohamada has joined #tripleo | 07:26 | |
| jaosorior | de-la-chinagada | 07:26 |
| *** jpich has joined #tripleo | 07:27 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts https://review.openstack.org/357765 | 07:29 |
| jaosorior | bandini: it was indeed in maintenance mode | 07:31 |
| bandini | jaosorior: ah okay at least we have a sensible explanation. any ideas why? | 07:33 |
| jaosorior | bandini: I'm not entirely sure man | 07:33 |
| bandini | let me dig around | 07:34 |
| jaosorior | bandini: we do set it to maintenance mode | 07:35 |
| jaosorior | in the standard pacemaker deployment | 07:35 |
| bandini | in extraconfig/tasks/pacemaker_maintenance_mode.sh you mean? | 07:35 |
| jaosorior | yeah | 07:36 |
| jaosorior | bandini: that's included in extraconfig/tasks/pre_puppet_pacemaker.yaml | 07:36 |
| jaosorior | which is included in environments/puppet-pacemaker.yaml | 07:36 |
| bandini | I see, yeah. Well then during that phase systemd restart/reload is the only way that will work | 07:36 |
| jaosorior | bandini: that seems to be the case... now we need to figure out how to do that | 07:37 |
| ccamacho | jaosorior good morning man, just waking up, Im a little feberish today :( | 07:39 |
| ccamacho | tell me :) | 07:39 |
| jaosorior | ccamacho: haha I don't remember what the deal was | 07:40 |
| ccamacho | :P | 07:40 |
| *** hewbrocca-afk is now known as hewbrocca | 07:41 | |
| *** ifarkas_afk is now known as ifarkas | 07:42 | |
| jaosorior | bandini: I guess setting service_manage for haproxy when we're in maintenance mode wouldn't be too bad | 07:43 |
| bandini | jaosorior: that is one way to do it I guess, if it overrides the restart disabling that we do somewhere else | 07:44 |
| jaosorior | bandini: which restart desabling? | 07:44 |
| jaosorior | bandini: I guess the restart-disabling (that I mean) IS the service_manage flag that we give to puppet | 07:45 |
| *** dbecker has joined #tripleo | 07:45 | |
| hewbrocca | Morning folks | 07:47 |
| bandini | jaosorior: we used to have a bunch of these https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/pacemaker/neutron.pp#L55 | 07:48 |
| *** mpsairam has quit IRC | 07:48 | |
| bandini | I can't see it for haproxy though, maybe I misremembered | 07:48 |
| bandini | morning hewbrocca | 07:48 |
| *** sshnaidm|afk is now known as sshnaidm | 07:48 | |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-ui: Use Mistral API for plan deletion https://review.openstack.org/356458 | 07:48 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: POC: WIP: oooq undercloud install https://review.openstack.org/358919 | 07:48 |
| jaosorior | ccamacho: now I remember. Did you dig further into reducing the number of workers in the undercloud>? | 07:49 |
| jaosorior | bandini: for haproxy we have https://github.com/openstack/puppet-tripleo/blob/master/manifests/haproxy.pp#L460 | 07:50 |
| *** zoli_gone-proxy is now known as zoliXXL | 07:50 | |
| bandini | jaosorior: ah righto | 07:50 |
| ccamacho | yes and no, :) I didnt realized than in my virtual environment there was no swap, so, what I did yesterday was to set the workers to 4 and add a 2GB swap file to the undercloud. And worked fine | 07:51 |
| ccamacho | jaosorior ^ | 07:51 |
| ccamacho | so, Im doing more deployments to see how it goes | 07:51 |
| *** zoliXXL is now known as zoli|trng | 07:51 | |
| ccamacho | but for now its working fine | 07:51 |
| jaosorior | bandini: and this is where it ends https://github.com/puppetlabs/puppetlabs-haproxy/blob/master/manifests/service.pp#L14 | 07:51 |
| jaosorior | ccamacho: I see. | 07:51 |
| ccamacho | I want to propose a small 1GB swap file for UC | 07:51 |
| ccamacho | just testing it now | 07:52 |
| *** ebalduf has quit IRC | 07:54 | |
| *** rcernin has quit IRC | 07:56 | |
| *** nyechiel has joined #tripleo | 07:56 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: DO NOT MERGE - TESTING BARBICAN https://review.openstack.org/353262 | 08:02 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add Barbican to the overcloud - WIP https://review.openstack.org/352458 | 08:02 |
| *** bana_k has quit IRC | 08:02 | |
| *** openstackgerrit has quit IRC | 08:03 | |
| *** openstackgerrit has joined #tripleo | 08:04 | |
| *** skramaja has quit IRC | 08:07 | |
| *** jtomasek has joined #tripleo | 08:07 | |
| *** Goneri has joined #tripleo | 08:13 | |
| *** athomas has joined #tripleo | 08:14 | |
| *** lucas-dinner is now known as lucasagomes | 08:15 | |
| *** derekh has joined #tripleo | 08:18 | |
| *** apetrich has quit IRC | 08:19 | |
| *** dsariel has quit IRC | 08:20 | |
| *** ohamada_ has joined #tripleo | 08:20 | |
| *** ohamada has quit IRC | 08:20 | |
| *** apetrich has joined #tripleo | 08:21 | |
| *** pgadiya has joined #tripleo | 08:22 | |
| *** fragatina has joined #tripleo | 08:22 | |
| *** vijaykc4 has quit IRC | 08:22 | |
| *** vijaykc4 has joined #tripleo | 08:23 | |
| openstackgerrit | sandhya proposed openstack/diskimage-builder: Add support for building images capable of UEFI https://review.openstack.org/287784 | 08:24 |
| *** coolsvap has joined #tripleo | 08:25 | |
| *** dsariel has joined #tripleo | 08:26 | |
| jaosorior | easy commit https://review.openstack.org/#/c/358971/ :D | 08:31 |
| *** maeca1 has joined #tripleo | 08:31 | |
| *** vijaykc4 has quit IRC | 08:31 | |
| *** adarazs has quit IRC | 08:32 | |
| *** adarazs has joined #tripleo | 08:32 | |
| openstackgerrit | Carlos Camacho proposed openstack-infra/tripleo-ci: Adding a 1GB swap file to the undercloud. https://review.openstack.org/359035 | 08:35 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Reload HAProxy on refresh https://review.openstack.org/359038 | 08:40 |
| jaosorior | bandini: ^^ | 08:40 |
| bandini | looking | 08:40 |
| *** ifarkas has quit IRC | 08:41 | |
| *** ifarkas has joined #tripleo | 08:42 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart: Reproducing specific CI run https://review.openstack.org/340156 | 08:45 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart: Devmode: rename delorean current so any existing repo is overwritten https://review.openstack.org/335953 | 08:45 |
| *** akrivoka has joined #tripleo | 08:45 | |
| *** TSCHAK has quit IRC | 08:47 | |
| jaosorior | bandini: you think https://review.openstack.org/359038 will be alright? | 08:48 |
| bandini | jaosorior: just added a comment | 08:48 |
| *** TSCHAK has joined #tripleo | 08:48 | |
| jaosorior | bandini: aha! one thing I could do is add an onlyif | 08:50 |
| jaosorior | and check for | 08:50 |
| jaosorior | pcs property | grep -q maintenance-mode | 08:50 |
| bandini | jaosorior: ah yes! | 08:50 |
| jaosorior | but... | 08:50 |
| jaosorior | then we need to know if pacemaker is enabled | 08:50 |
| jaosorior | cause that runs even on nonha | 08:50 |
| bandini | in the short term yes, in the mid-term I hope we can just kill nonha now that the new HA stuff has landed | 08:50 |
| jaosorior | bandini: it also runs in the undercloud | 08:51 |
| bandini | meh | 08:52 |
| bandini | jaosorior: I guess a if [[ $(systemctl is-active pacemaker) = "active" ]] should do it though? | 08:53 |
| jaosorior | bandini: I have an easier way | 08:53 |
| jaosorior | jsut figured it out :P | 08:53 |
| bandini | ah cool | 08:57 |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-common: Use Openstack cli instead in order to get the instance status https://review.openstack.org/353996 | 08:57 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Reload HAProxy on refresh https://review.openstack.org/359038 | 08:58 |
| *** electrofelix has joined #tripleo | 09:00 | |
| openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: [NO MERGE] Test write performances https://review.openstack.org/359054 | 09:01 |
| *** gfidente has joined #tripleo | 09:02 | |
| *** gfidente has quit IRC | 09:02 | |
| *** gfidente has joined #tripleo | 09:02 | |
| *** ifarkas_ has joined #tripleo | 09:02 | |
| jaosorior | gfidente: let me know when you're up for a cal | 09:03 |
| jaosorior | *call | 09:03 |
| *** mcornea has quit IRC | 09:04 | |
| *** mcornea has joined #tripleo | 09:04 | |
| jaosorior | bandini: uploaded another version. What do you think? | 09:04 |
| jaosorior | jistr: are you around? | 09:05 |
| jistr | jaosorior: hi, yes but i have 0 extra bandwidth :/ | 09:05 |
| jaosorior | jistr: no problem. Was gonna have a call with gfidente to explain the certmonger stuff. But if you don't have time it's fine | 09:06 |
| jistr | jaosorior, gfidente: ok guys please ping me when you are starting and i'll decide then. I need to get to test one fix as soon as possible, perhaps i could be on a call in parallel. | 09:09 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Use separated SSL endpoint environment file https://review.openstack.org/356488 | 09:09 |
| *** abehl has joined #tripleo | 09:11 | |
| gfidente | jistr can you ping us when it's best for you? | 09:11 |
| *** abehl has quit IRC | 09:11 | |
| *** abehl has joined #tripleo | 09:11 | |
| jaosorior | ccamacho: are you interested in joining also? Would sure use reviewers. | 09:12 |
| ccamacho | hey! thanks sure :) | 09:12 |
| jistr | gfidente, jaosorior: ack thanks :) i'm hoping in 20 minutes or so, i just need to start the upgrade run | 09:12 |
| jaosorior | jistr: it can be in 20 minutes. It's nothing with a hard-schedule | 09:13 |
| jistr | jaosorior: cool, thanks :) | 09:13 |
| ccamacho | +1 | 09:13 |
| ccamacho | thanks! | 09:13 |
| jaosorior | jistr: gives me time to make more coffee :P | 09:13 |
| jistr | hehe | 09:13 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: WIP DO NOT MERGE - Enable the HA deployment by default https://review.openstack.org/359060 | 09:13 |
| gfidente | jistr ccamacho cool guys :) | 09:14 |
| *** d0ugal has quit IRC | 09:15 | |
| *** skramaja has joined #tripleo | 09:15 | |
| *** zoli|trng is now known as zoli_gone-proxy | 09:15 | |
| *** akuznetsov has joined #tripleo | 09:16 | |
| *** d0ugal has joined #tripleo | 09:16 | |
| *** milan has quit IRC | 09:17 | |
| *** ebarrera has joined #tripleo | 09:17 | |
| ccamacho | Quick question, do you remember which is the file from CI logs that shows the node status (RAM, disk usage, etc) ? | 09:21 |
| openstackgerrit | Ana Krivokapic proposed openstack/instack-undercloud: Add default plan creation https://review.openstack.org/349532 | 09:22 |
| *** ebarrera has quit IRC | 09:23 | |
| jaosorior | don't remember :/ | 09:23 |
| ccamacho | ajajaj I know its there but cant remember neither | 09:25 |
| ccamacho | jaosorior, found it http://logs.openstack.org/94/355394/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/0abf721/logs/undercloud/var/log/host_info.txt.gz | 09:26 |
| jaosorior | nice | 09:28 |
| *** Goneri has quit IRC | 09:30 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates: Refactor upgrade checks. https://review.openstack.org/357750 | 09:32 |
| *** pgadiya_ has joined #tripleo | 09:33 | |
| *** Goneri has joined #tripleo | 09:33 | |
| *** spredzy has quit IRC | 09:33 | |
| *** maeca1 has quit IRC | 09:33 | |
| *** asbishop has joined #tripleo | 09:34 | |
| *** pgadiya has quit IRC | 09:34 | |
| *** athomas has quit IRC | 09:34 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Upgrade ceph-mon https://review.openstack.org/357130 | 09:36 |
| jistr | jaosorior, gfidente, ccamacho: upgrade running, i'm ready for a call if you are :) | 09:36 |
| jistr | thanks for waiting :) | 09:36 |
| ccamacho | yeahp | 09:37 |
| ccamacho | :) | 09:38 |
| gfidente | https://bluejeans.com/u/gfidente/ ? | 09:38 |
| ccamacho | Bluejeans? | 09:38 |
| ccamacho | cool | 09:38 |
| gfidente | jaosorior ^^ ? | 09:38 |
| jaosorior | could be google hangouts if people prefer that | 09:38 |
| jaosorior | I'm in gfidente's bluejeans | 09:39 |
| jaosorior | so yeah | 09:39 |
| jaosorior | https://bluejeans.com/u/gfidente/ is cool | 09:39 |
| gfidente | except I don't hear you guys | 09:39 |
| jistr | gfidente: it says wait for moderator to start the meeting. Are you the moderator? :) | 09:39 |
| *** lucasagomes is now known as lucas-afk | 09:40 | |
| *** athomas has joined #tripleo | 09:40 | |
| *** zoli_gone-proxy is now known as zoliXXL | 09:40 | |
| jaosorior | wow | 09:41 |
| *** zoliXXL is now known as zoli|trng | 09:41 | |
| jaosorior | there are three instances of gfidente in that meeting | 09:41 |
| gfidente | right | 09:41 |
| *** tosky has joined #tripleo | 09:41 | |
| gfidente | let's use something else | 09:41 |
| gfidente | I have no clue guys :) | 09:41 |
| jaosorior | what about this? https://bluejeans.com/2527414409 | 09:42 |
| jaosorior | gfidente: that seems to work ^^ | 09:42 |
| *** dtantsur|afk is now known as dtantsur | 09:42 | |
| gfidente | jaosorior much better thanks | 09:42 |
| *** spredzy has joined #tripleo | 09:45 | |
| *** ebarrera has joined #tripleo | 09:46 | |
| hewbrocca | marios: OMG THESE GUYS ARE CLONING GFIDENTE | 09:48 |
| Jokke_ | is this known issue or did I hit something new http://logs.openstack.org/94/355394/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/0abf721/console.html#_2016-08-22_14_57_37_192964 | 09:51 |
| *** bootsha has quit IRC | 09:53 | |
| Jokke_ | oh darn it failed here http://logs.openstack.org/94/355394/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/0abf721/console.html#_2016-08-22_14_47_49_233311 still have no idea | 09:53 |
| *** ifarkas has quit IRC | 09:54 | |
| *** ifarkas_ is now known as ifarkas | 09:54 | |
| *** bootsha has joined #tripleo | 09:56 | |
| bandini | Jokke_: I have seen that one too | 09:59 |
| *** ebarrera has quit IRC | 10:01 | |
| *** asbishop has quit IRC | 10:01 | |
| Jokke_ | bandini: should I just recheck again or is there something else I actually can do about it? | 10:03 |
| bandini | Jokke_: I haven't investigated much but a recheck seemed to fix it | 10:04 |
| Jokke_ | thnx, trying again | 10:05 |
| *** limao has quit IRC | 10:10 | |
| *** ccamacho is now known as ccamacho|afk | 10:15 | |
| gfidente | jaosorior topic is bp/tls-via-certmonger ? | 10:15 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add HAProxy TLS handled by certmonger as composable service https://review.openstack.org/356430 | 10:18 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts https://review.openstack.org/357765 | 10:18 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable usage of FQDNs for the endpoints https://review.openstack.org/356416 | 10:18 |
| jaosorior | gfidente: yeah. I'm updating the reviews | 10:19 |
| gfidente | jaosorior ah nice | 10:20 |
| d0ugal | jpich: Hey, do you have plans to write an openstack overcloud plan create? | 10:21 |
| jpich | d0ugal: No plans | 10:22 |
| d0ugal | jpich: nice pun! | 10:22 |
| jpich | I totally did that on purpose | 10:22 |
| jpich | (I didn't) | 10:22 |
| d0ugal | lol | 10:22 |
| jaosorior | gfidente, ccamacho|afk, jistr so it's updated now https://review.openstack.org/#/q/status:open+topic:bp/tls-via-certmonger | 10:22 |
| d0ugal | jpich: Okay, in the process of documenting stuff, I realise we really need a plan create and a way to deploy the plan. | 10:22 |
| d0ugal | jpich: openstack overcloud deploy --plan $NAME | 10:23 |
| * d0ugal will create bugs | 10:23 | |
| shadower | d0ugal: why would you create bugs??? | 10:24 |
| d0ugal | shadower: I don't know how to do anything else :( | 10:25 |
| d0ugal | shadower: I don't always report them tho' | 10:25 |
| jpich | d0ugal: From your experience in previous releases, what's the deadline for getting stuff into the client? I see the Newton schedule says Feature Freeze is also the final release for client libraries but.... :/ | 10:25 |
| d0ugal | jpich: clients are usually a bit more relaxed | 10:25 |
| jpich | d0ugal: I thought the plan deploy stuff was what you were working on? Is that different? | 10:25 |
| d0ugal | jpich: Yeah, it's different - subtly. | 10:26 |
| d0ugal | jpich: I am making the existing command use mistral. | 10:26 |
| d0ugal | jpich: it creates a plan on the fly and deploys it | 10:26 |
| jpich | I see | 10:26 |
| d0ugal | jpich: we need to break that out so users can do the individual steps | 10:27 |
| d0ugal | jpich: for example, you added a delete plan - but you can't create a plan without deploying it yet. | 10:27 |
| d0ugal | jpich: all the code is in my patch, so it should be quite easy - in theory | 10:27 |
| jpich | d0ugal: Right... I've been calling the create default plan action but I don't think there's a CLI command for it either yet | 10:28 |
| jpich | d0ugal: "In theory" :P | 10:28 |
| d0ugal | jpich: True, the create plan command should probably allow users to select the default or provide their own templates. | 10:28 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts https://review.openstack.org/357765 | 10:28 |
| d0ugal | jpich: the version I am working on always provides templates, because that is the current behaviour - often they will be the same as the default templates, but I don't think there is a good way to check. | 10:29 |
| derekh | dtantsur: did that tftp/iptables thing get unstuck yesterday? | 10:31 |
| derekh | *get you unstuck | 10:31 |
| jpich | d0ugal: Ok, thank you for the explanations! I'll try to give a hand with the missing plan commands | 10:31 |
| d0ugal | jpich: https://bugs.launchpad.net/tripleo/+bug/1616014 | 10:32 |
| openstack | Launchpad bug 1616014 in tripleo "Provide an `openstack overcloud plan create` command" [High,Confirmed] | 10:32 |
| d0ugal | jpich: k, I can help with it too probably | 10:32 |
| dtantsur | derekh, yes, you're amazing (with bnemec who fixed my subnets misunderstanding) | 10:32 |
| derekh | ;-) | 10:32 |
| dtantsur | derekh, now I got as far as passing out-of-band cleaning, so we're ready for deploy | 10:32 |
| jpich | d0ugal: Excellent :D | 10:32 |
| d0ugal | jpich: One more incoming. | 10:33 |
| dtantsur | derekh, do you mind reviewing https://review.openstack.org/356539 please? just adds a couple of missing bits? | 10:33 |
| * jpich dodges | 10:33 | |
| derekh | dtantsur: will do | 10:33 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Upgrade ceph-osd https://review.openstack.org/357312 | 10:34 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Upgrade ceph-mon https://review.openstack.org/357130 | 10:34 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Pass TRIPLEO_ROOT directory to heat_deploy_times.sh https://review.openstack.org/356946 | 10:35 |
| *** bootsha has quit IRC | 10:35 | |
| jpich | d0ugal: I'm spending some time on reviews/testing patches at the moment, if you were about to write these don't mind me and go ahead - I'll give a hand reviewing them instead :) | 10:36 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Upgrade ceph-osd https://review.openstack.org/357312 | 10:36 |
| d0ugal | jpich: Sure, I'll do the create plan command. FYI, https://bugs.launchpad.net/tripleo/+bug/1616015 | 10:37 |
| openstack | Launchpad bug 1616015 in tripleo "Provide a way to deploy a plan" [High,Confirmed] | 10:37 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-docs: [WIP] Mistral API Documentation https://review.openstack.org/358685 | 10:38 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Reload HAProxy on refresh https://review.openstack.org/359038 | 10:38 |
| jpich | d0ugal: Roger. I'll look at whatever's left unassigned later on then | 10:38 |
| jaosorior | bandini: thanks for the review | 10:38 |
| *** ramishra has joined #tripleo | 10:39 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-docs: [WIP] Mistral API Documentation https://review.openstack.org/358685 | 10:41 |
| bandini | jaosorior: no problem dude ;) | 10:44 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Move out of roles any Ceph setting https://review.openstack.org/356657 | 10:45 |
| d0ugal | https://review.openstack.org/#/c/298732/ <- needs some attention if anyone has time for reviews :) | 10:48 |
| d0ugal | jaosorior: Thanks! | 10:50 |
| *** coolsvap is now known as coolsvap_ | 10:53 | |
| *** TSCHAK has quit IRC | 10:56 | |
| *** TSCHAK has joined #tripleo | 10:58 | |
| *** rodrigods has quit IRC | 10:59 | |
| *** rodrigods has joined #tripleo | 10:59 | |
| *** pkovar has joined #tripleo | 11:03 | |
| *** milan has joined #tripleo | 11:04 | |
| *** jpena is now known as jpena|lunch | 11:06 | |
| *** bootsha has joined #tripleo | 11:07 | |
| *** ccamacho|afk is now known as ccamacho | 11:09 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Enable SSL for ironic inspector https://review.openstack.org/332641 | 11:10 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Refactor endpoint env variable population https://review.openstack.org/350396 | 11:10 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Enable TLS for Zaqar API https://review.openstack.org/351576 | 11:10 |
| *** vijaykc4 has joined #tripleo | 11:12 | |
| openstackgerrit | Ana Krivokapic proposed openstack/instack-undercloud: Add default plan creation https://review.openstack.org/349532 | 11:13 |
| jaosorior | gfidente: hey dude, could you help me out a bit figuring out what's missing here? https://review.openstack.org/#/c/352458/12 | 11:14 |
| jaosorior | gfidente: also, if you jistr and ccamacho have some time to check out again the submission for FQDNs for the overcloud VIPs https://review.openstack.org/#/c/356416/ would appreciate it | 11:15 |
| *** pkovar has quit IRC | 11:19 | |
| *** bkopilov has quit IRC | 11:19 | |
| *** ramishra has quit IRC | 11:20 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Enable SSL for undercloud-only job https://review.openstack.org/359131 | 11:20 |
| *** ramishra has joined #tripleo | 11:22 | |
| Jokke_ | jaosorior: looks like the common error from the different runs is http://logs.openstack.org/58/352458/12/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/ebf551e/console.html#_2016-08-23_10_31_36_034557 | 11:22 |
| jaosorior | Jokke_: which commit is that | 11:22 |
| Jokke_ | jaosorior: 12 | 11:23 |
| jaosorior | Jokke_: that's the error I'm seeing in the barbican-for-the-overcloud submissions. But I haven't figured out what's missing there exactly | 11:23 |
| Jokke_ | the one you mentioned above | 11:23 |
| jaosorior | aah | 11:23 |
| jaosorior | yeah | 11:23 |
| jaosorior | I know what the error is | 11:23 |
| jaosorior | but it is not entirely apparent to me what did I actually miss | 11:23 |
| Jokke_ | aha | 11:23 |
| jaosorior | Jokke_: since I'm not changing the VipDeployment's input. Just adding barbican there | 11:24 |
| *** pkovar has joined #tripleo | 11:26 | |
| *** mburned_out is now known as mburned | 11:28 | |
| *** dprince has joined #tripleo | 11:29 | |
| *** adarazs is now known as adarazs_lunc | 11:29 | |
| *** adarazs_lunc is now known as adarazs_lunch | 11:29 | |
| *** numans has joined #tripleo | 11:30 | |
| *** limao has joined #tripleo | 11:31 | |
| *** limao_ has joined #tripleo | 11:32 | |
| *** rbowen has joined #tripleo | 11:33 | |
| *** rbowen is now known as DrBacchus | 11:34 | |
| d0ugal | slagle: Thanks for the deploy workflow review! | 11:35 |
| *** limao has quit IRC | 11:35 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Migrate the deploy command to use the Mistral workflows https://review.openstack.org/342261 | 11:36 |
| jaosorior | d0ugal: wow | 11:36 |
| jaosorior | lets see how that goes :O | 11:36 |
| jaosorior | cool stuff | 11:36 |
| d0ugal | jaosorior: It wont actually be used anywhere until the CLI patch lands - but at least that can test against CI soon. | 11:37 |
| *** jtomasek has quit IRC | 11:37 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: POC: WIP: oooq undercloud install https://review.openstack.org/358919 | 11:37 |
| d0ugal | ... and I need to do a few updates to the CLI deploy patch still, should be easier now. | 11:37 |
| thrash | d0ugal: awesome. I'm still trying to track down exactly what was failing for me yesterday. | 11:38 |
| d0ugal | thrash: Yeah, did you send me your envs? | 11:38 |
| d0ugal | I kinda hope that CI will fail and reveal issues... | 11:39 |
| d0ugal | but I also kinda hope CI just works ;) | 11:39 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE Testin OOOQ job https://review.openstack.org/359146 | 11:39 |
| *** bfournie has quit IRC | 11:40 | |
| *** vipulnayyar has joined #tripleo | 11:40 | |
| *** ccamacho is now known as ccamacho|lunch | 11:41 | |
| vipulnayyar | Hey there, while setting up packages in undercloud, I'm getting following yum dependency issues: | 11:42 |
| vipulnayyar | Error: Package: python-tripleoclient-5.0.0-0.20160819210400.f666339.el7.centos.noarch (delorean-current) | 11:42 |
| vipulnayyar | Requires: python-osc-lib >= 0.3.0 | 11:42 |
| Jokke_ | jaosorior: could it be because you did not add it to overcloud-without-mergepy.yaml? | 11:42 |
| vipulnayyar | Apparently, python-osc-lib rpm is not present in delorean repo only src.rpm | 11:42 |
| vipulnayyar | Can someone guide me what would be the workaround for this? | 11:43 |
| dtantsur | vipulnayyar, please raise it on #rdo | 11:44 |
| *** rcernin has joined #tripleo | 11:44 | |
| sshnaidm | vipulnayyar, which repos do you have? | 11:47 |
| jaosorior | Jokke_: we're not using overcloud-without-mergepy.yaml. Actually I'm not sure if we can remove that file already :D that would be cool | 11:47 |
| jaosorior | slagle: ?? ^^ | 11:47 |
| vipulnayyar | sshnaidm: This one: http://trunk.rdoproject.org/centos7/aa/bf/aabf94f4cf5d5d3973e76b4b1b274ccfcdd29300_586add66 | 11:48 |
| *** jtomasek has joined #tripleo | 11:49 | |
| EmilienM | good morning folks | 11:49 |
| Jokke_ | jaosorior: ok, just thought as VipDeployment was defined there as well, but noticed that you're not touching it ;) | 11:50 |
| Jokke_ | hi EmilienM | 11:50 |
| *** jaosorior has quit IRC | 11:50 | |
| thrash | d0ugal: I sent you my list. They're all readily available. :) | 11:50 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Enable SSL for ironic inspector https://review.openstack.org/332641 | 11:51 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Refactor endpoint env variable population https://review.openstack.org/350396 | 11:51 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Enable TLS for Zaqar API https://review.openstack.org/351576 | 11:51 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Mistral db-populate depends on inspector's endpoint https://review.openstack.org/359153 | 11:51 |
| *** jaosorior has joined #tripleo | 11:51 | |
| thrash | d0ugal: puppet-pacemaker.yaml, tripleo-ci/test-environments/workers.yaml, and a puppet-modules.yaml from using the latest puppet modules. | 11:51 |
| jaosorior | EmilienM: sup dude | 11:51 |
| *** abehl has quit IRC | 11:52 | |
| thrash | d0ugal: so yeah... just enabling puppet-pacemaker.yaml on master is doing it. | 11:52 |
| *** limao_ has quit IRC | 11:52 | |
| thrash | d0ugal: it didn't trigger on a packaged version of tht (I don't think I have the latest promote installed) | 11:52 |
| thrash | let me try that. | 11:52 |
| *** limao has joined #tripleo | 11:52 | |
| sshnaidm | vipulnayyar, weird, it has such: https://trunk.rdoproject.org/centos7/aa/bf/aabf94f4cf5d5d3973e76b4b1b274ccfcdd29300_586add66/python2-osc-lib-0.4.1-0.20160802185554.ef4a4f1.el7.centos.noarch.rpm | 11:54 |
| thrash | d0ugal: works with [stack@instack ~]$ rpm -q openstack-tripleo-heat-templates | 11:54 |
| thrash | openstack-tripleo-heat-templates-5.0.0-0.20160818151351.6758686.el7.centos.noarch | 11:54 |
| thrash | d0ugal: does not work with master. | 11:54 |
| vipulnayyar | sshnaidm: I tried manually installing that package, but it further brings up another dependency issue with python-oslo-utils | 11:55 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Permit usage of raw disks as backends for Swift storage https://review.openstack.org/350841 | 11:55 |
| d0ugal | thrash: k, thanks - just going for lunch but I'll take a look when I get back. Must have missed the list yesterday, sorry :) | 11:55 |
| sshnaidm | vipulnayyar, did you try yum install python2-osc-lib? | 11:55 |
| thrash | d0ugal: no worries. I'm troubleshooting on my end. | 11:55 |
| sshnaidm | vipulnayyar, if it's reproducible, like dtantsur said it's better to raise it in #rdo | 11:56 |
| *** Goneri has quit IRC | 11:57 | |
| vipulnayyar | sshnaidm: All this was working fine for me a couple weeks ago, by following the TripleO docs. Suddenly, it seems to be broken | 11:57 |
| vipulnayyar | sshnaidm: the package python2-osc-lib is somehow not available in any repo provided by TripleO | 11:57 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: WIP DO NOT MERGE - Enable the HA deployment by default https://review.openstack.org/359060 | 11:57 |
| vipulnayyar | sshnaidm: Also, I have raised this in #rdo | 11:57 |
| *** dsariel has quit IRC | 11:58 | |
| sshnaidm | vipulnayyar, I'm not sure, but it seems like yum issue, because python2-osc-lib is definitely in http://trunk.rdoproject.org/centos7/aa/bf/aabf94f4cf5d5d3973e76b4b1b274ccfcdd29300_586add66 . Maybe try "yum clean ll", rebuilding rpmdb and again | 11:58 |
| *** adarazs_lunch is now known as adarazs | 12:00 | |
| dtantsur | I've left my overcloud to work for some time, and now all my API services fail with broken pipe | 12:00 |
| dtantsur | I wonder if something is wrong around haproxy... | 12:00 |
| *** yamahata has quit IRC | 12:00 | |
| openstackgerrit | Merged openstack/python-tripleoclient: Fix unbracket_ipv6() https://review.openstack.org/358729 | 12:01 |
| EmilienM | can anyone summarize 1) how ovb jobs are back to green (specially HA) and 2) how promotion did happen? | 12:01 |
| jaosorior | gfidente: I guess we need to backport this too https://review.openstack.org/#/c/358729/ | 12:02 |
| *** pgadiya_ is now known as pgadiya | 12:03 | |
| gfidente | jaosorior yeah I don't know if akrivoka is pushing it maybe? | 12:03 |
| jaosorior | akrivoka: can you backport https://review.openstack.org/#/c/358729/ ? | 12:03 |
| akrivoka | jaosorior: sure | 12:03 |
| *** Goneri has joined #tripleo | 12:04 | |
| sshnaidm | slagle, hi | 12:04 |
| jaosorior | akrivoka: thanks | 12:04 |
| vipulnayyar | sshnaidm: That worked for me :-) Thanks a lot | 12:04 |
| slagle | sshnaidm: hello | 12:05 |
| sshnaidm | vipulnayyar, np | 12:05 |
| sshnaidm | slagle, can I run periodic jobs again today, based on your patch? https://review.openstack.org/#/c/346949/ | 12:05 |
| sshnaidm | slagle, of course setting cashupload to 1 and everything that needs.. | 12:06 |
| slagle | sshnaidm: did the jobs pass last night? | 12:06 |
| sshnaidm | slagle, ha passed, nonha failed, I suspect it's casual fail | 12:07 |
| slagle | and if not, why. and have we fixed that problem yet | 12:07 |
| *** abehl has joined #tripleo | 12:07 | |
| slagle | sshnaidm: i think we should start filing some bugs for all these casual failures | 12:07 |
| slagle | and get some eyes on them | 12:08 |
| dtantsur | for the record: I had to restart httpd to bring my overcloud make to life | 12:08 |
| slagle | sshnaidm: but if you want to rerun the periodic test, you can. we got a promote yesterday though | 12:08 |
| sshnaidm | slagle, ok, sure | 12:08 |
| *** limao has quit IRC | 12:08 | |
| slagle | sshnaidm: the other thing that needs doing is reporting on the status of the periodic jobs on tripleo.org | 12:09 |
| sshnaidm | slagle, yes, the second question is the mechanism to run it on demand, not waiting a day, is this patch is acceptable way to do it? | 12:09 |
| sshnaidm | slagle, we have this reporting in my page: http://status-tripleoci.rhcloud.com/ | 12:10 |
| slagle | sshnaidm: yes, but we need something on tripleo.org | 12:10 |
| slagle | sshnaidm: it looks like both the jobs failed due to timeout this morning? | 12:11 |
| slagle | sshnaidm: we need to figure out why jobs are running so slowly in CI | 12:12 |
| *** vipulnayyar has quit IRC | 12:12 | |
| slagle | i'd focus on that instead of trying to get another promote | 12:12 |
| sshnaidm | slagle, ha job succeeded to upload images and report its success to mirror server before it was cut by timeout | 12:12 |
| slagle | it will probably just time out again though | 12:12 |
| slagle | b/c regular HA jobs are taking more than 2h40m | 12:13 |
| sshnaidm | slagle, right, it's a big problem now | 12:13 |
| sshnaidm | slagle, I thought maybe return to use cached images, but I'm not sure about status of puppet modules in overcloud images: https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/common_functions.sh#L128 | 12:15 |
| slagle | sshnaidm: we should have that now if a new image got uploaded on monday | 12:16 |
| slagle | since the promotes passed on monday | 12:16 |
| sshnaidm | slagle, ok, then I'll try to use cache | 12:16 |
| slagle | ok, great. you can confirm if the new image is there on the mirror server | 12:16 |
| sshnaidm | I'll check it\ | 12:17 |
| openstackgerrit | Saravanan KR proposed openstack/os-net-config: Add support for OVS DPDK Bridge and Port https://review.openstack.org/338694 | 12:18 |
| openstackgerrit | Saravanan KR proposed openstack/os-net-config: Add support for OVS DPDK Bond https://review.openstack.org/349183 | 12:20 |
| *** DrBacchus has quit IRC | 12:22 | |
| *** rcernin has quit IRC | 12:23 | |
| *** asbishop has joined #tripleo | 12:23 | |
| *** jdob has joined #tripleo | 12:27 | |
| *** rhallisey_ has joined #tripleo | 12:28 | |
| *** morazi has joined #tripleo | 12:28 | |
| *** abregman has quit IRC | 12:28 | |
| *** rcernin has joined #tripleo | 12:29 | |
| *** cdearborn has joined #tripleo | 12:29 | |
| *** masco has quit IRC | 12:30 | |
| openstackgerrit | Merged openstack/tripleo-common: Adds Deployment Action & Workflow https://review.openstack.org/298732 | 12:30 |
| hewbrocca | slagle: if we're not using a cached image then CI is certainly going to be show... | 12:32 |
| hewbrocca | er slow | 12:32 |
| pradk | can i get some reviews on these please - https://review.openstack.org/#/c/352476/ & https://review.openstack.org/#/c/293642/ | 12:32 |
| *** jcoufal has joined #tripleo | 12:32 | |
| *** abehl has quit IRC | 12:32 | |
| hewbrocca | right? | 12:33 |
| slagle | hewbrocca: relative to using a cached image, of course it will be slower | 12:35 |
| *** coolsvap_ is now known as coolsvap | 12:36 | |
| slagle | but it's not just image building | 12:36 |
| hewbrocca | OK, I see | 12:36 |
| slagle | jobs are slow across the board | 12:36 |
| slagle | ha is taking 30-40 mins longer than it was just a month or 2 ago | 12:37 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: DO NOT MERGE Add undercloud reboot - Testing https://review.openstack.org/350489 | 12:37 |
| slagle | anecdotally it seems it's just b/c we keep making everything slower | 12:37 |
| slagle | and not an actual infrastructure issue | 12:37 |
| hewbrocca | slagle: don't talk about gfidente that way | 12:38 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Migrate the deploy command to use the Mistral workflows https://review.openstack.org/342261 | 12:38 |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Add a big HA deployment configuration https://review.openstack.org/359173 | 12:38 |
| *** bfournie has joined #tripleo | 12:39 | |
| *** ebarrera has joined #tripleo | 12:39 | |
| hewbrocca | All BS aside... did we make any progress not activating some of the new overcloud services? | 12:39 |
| *** alee has quit IRC | 12:39 | |
| hewbrocca | Or is that not even the issue? | 12:39 |
| slagle | yes, sahara is now off | 12:39 |
| slagle | as is cinder backup | 12:39 |
| slagle | so now we are just squeaking through on ram and timeout | 12:40 |
| *** jpena|lunch is now known as jpena | 12:40 | |
| hewbrocca | Anything else we can easily turn off? | 12:40 |
| gfidente | I noticed the nonha job is also taking 2h40min | 12:40 |
| gfidente | so it doesn't look like pcmk scenario is any better or worse | 12:41 |
| *** vijaykc4 has quit IRC | 12:41 | |
| gfidente | here for example nonha is slower https://review.openstack.org/#/c/354613/ | 12:41 |
| akrivoka | jaosorior: https://review.openstack.org/#/c/358729/ backport needs to go into mitaka and liberty, right? | 12:41 |
| slagle | gfidente: yep | 12:42 |
| openstackgerrit | Ana Krivokapic proposed openstack/python-tripleoclient: Fix unbracket_ipv6() https://review.openstack.org/359174 | 12:42 |
| gfidente | akrivoka both yes but liberty version needs to be on top of https://review.openstack.org/#/c/355882/ | 12:42 |
| slagle | gfidente: the overcloud deploy took an hour in that job | 12:42 |
| akrivoka | gfidente: ok, there's the mitaka version ^ and I'll wait for your patch to land into liberty first | 12:43 |
| *** vijaykc4 has joined #tripleo | 12:43 | |
| jaosorior | akrivoka: yes | 12:43 |
| slagle | gfidente: might want to check the logs to see if maybe the nova instances got rescheduled a bunch of times, or what caused it to take so long | 12:43 |
| *** vijaykc4 has quit IRC | 12:43 | |
| *** ebarrera has quit IRC | 12:43 | |
| *** abregman has joined #tripleo | 12:44 | |
| *** rlandy has joined #tripleo | 12:45 | |
| *** abehl has joined #tripleo | 12:45 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Add manifests to inject and trust CA certificates https://review.openstack.org/353021 | 12:45 |
| *** bootsha has quit IRC | 12:47 | |
| *** jubapa has joined #tripleo | 12:48 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-docs: Documentation for installing and using Ironic in overcloud https://review.openstack.org/354016 | 12:49 |
| *** dtantsur is now known as dtantsur|mtg | 12:50 | |
| jaosorior | EmilienM: Hey dude, before you went on PTO you mentioned that there might be an issue with deploying the keystone endpoints via puppet. Did you get time to ellaborate on what's up? | 12:51 |
| EmilienM | jaosorior: yes, chem and I commented on one of your patches | 12:51 |
| EmilienM | I -2'ed it | 12:51 |
| *** lblanchard has joined #tripleo | 12:52 | |
| jaosorior | EmilienM: ah | 12:54 |
| jaosorior | now I see | 12:54 |
| jaosorior | EmilienM: so we need to make sure that all the *::keystone::auth parameters get into the hieradata in a node where keystone is running | 12:55 |
| jaosorior | EmilienM: how do we do it for the firewall rules? | 12:55 |
| EmilienM | jaosorior: yes | 12:55 |
| EmilienM | firewall rules are declared where api services are running | 12:55 |
| jaosorior | EmilienM: seems to me that those are only executed in the specific nodes where those services are running | 12:56 |
| *** morazi has quit IRC | 12:56 | |
| jaosorior | so that approach doesn't address this case | 12:56 |
| *** chlong has quit IRC | 12:56 | |
| *** mburned is now known as mburned_out | 12:56 | |
| *** tzumainn has joined #tripleo | 12:56 | |
| *** tzumainn has quit IRC | 12:57 | |
| *** tzumainn has joined #tripleo | 12:57 | |
| *** tobias_fiberdata has quit IRC | 13:00 | |
| *** coolsvap is now known as coolsvap_ | 13:00 | |
| *** rhallisey_ is now known as rhallisey | 13:03 | |
| *** noslzzp has joined #tripleo | 13:03 | |
| *** rcernin has quit IRC | 13:03 | |
| sshnaidm | slagle, images weren't uploaded, because mirrors server is 100% used space, arrgh | 13:04 |
| gfidente | slagle have any clue why I can't find this output in the console https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/get_host_info.sh#L37 ? | 13:07 |
| gfidente | though reality is that in https://review.openstack.org/#/c/354613/ same code took 2h45m on ovb and 1h on multinode job | 13:08 |
| sshnaidm | gfidente, it should be in separate file host_info.txt | 13:08 |
| gfidente | sshnaidm yeah I can't find it there | 13:09 |
| gfidente | also that command is not given as part of host_info so I think it should be in the console log no? | 13:09 |
| tzumainn | d0ugal, your mistral documentation is really great so far, big thumbs up from me | 13:09 |
| *** akuznetsov has quit IRC | 13:10 | |
| *** akuznetsov has joined #tripleo | 13:10 | |
| *** akuznetsov has quit IRC | 13:11 | |
| gfidente | sshnaidm looks like it's looking for stackrc in wrong location? | 13:11 |
| slagle | gfidente: in that nonha job, ControllerOvercloudServicesDeployment_Step4 took 18 minutes | 13:12 |
| slagle | gfidente: and ControllerOvercloudServicesDeployment_Step5 took 15 minutes | 13:12 |
| slagle | that is insane | 13:12 |
| *** chlong has joined #tripleo | 13:12 | |
| gfidente | slagle right I was looking for the event-list output as well, where is it? | 13:12 |
| sshnaidm | gfidente, it seems right location | 13:12 |
| *** lucas-afk is now known as lucas-hungry | 13:12 | |
| slagle | gfidente: it's possibly in the postci log | 13:12 |
| sshnaidm | gfidente, lemme see where this output is going to.. | 13:12 |
| gfidente | slagle++ postci :) | 13:13 |
| gfidente | sshnaidm thanks for checking as well | 13:13 |
| slagle | gfidente: http://logs.openstack.org/13/354613/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/8ed156c/logs/postci.txt.gz | 13:13 |
| sshnaidm | slagle, can I delete some folder in mirror server then? | 13:13 |
| sshnaidm | although 40G is not too much for server that hosting images.. | 13:14 |
| slagle | sshnaidm: yes, you can delete some of the older uploads under /var/www/html/builds | 13:18 |
| sshnaidm | slagle, I think we need to consider to extend volume size there for long range usage.. | 13:19 |
| EmilienM | jaosorior: can you land https://review.openstack.org/#/c/356754/ please? | 13:19 |
| *** julim has joined #tripleo | 13:21 | |
| *** ccamacho|lunch is now known as ccamacho | 13:21 | |
| *** egafford has joined #tripleo | 13:21 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Add a big HA deployment configuration https://review.openstack.org/359173 | 13:21 |
| jaosorior | EmilienM: do we have in hiera which services are enabled globally? | 13:21 |
| jaosorior | gfidente: ^^ | 13:21 |
| EmilienM | jaosorior: I don't recall | 13:21 |
| *** ioggstream has joined #tripleo | 13:22 | |
| jaosorior | dprince: ^^ | 13:22 |
| ioggstream | gfidente: hi | 13:22 |
| *** pgadiya has quit IRC | 13:23 | |
| gfidente | slagle EmilienM I collected some data from the logs | 13:27 |
| gfidente | I do not think this is a problem with the overcloud per-se | 13:27 |
| gfidente | nor with any of the steps in particular | 13:27 |
| gfidente | slagle EmilienM look at these numbers https://paste.fedoraproject.org/412868/ | 13:29 |
| EmilienM | gfidente: nonha is longer than ha? | 13:31 |
| gfidente | EmilienM exactly, that reinforces the point | 13:31 |
| gfidente | and multinode nonha takes 1h3m *less* than ovb nonha | 13:31 |
| gfidente | but in general, looking at the ovb ha | 13:31 |
| EmilienM | gfidente: do you want me to +2 the patch? | 13:31 |
| *** bkopilov has joined #tripleo | 13:32 | |
| gfidente | EmilienM no the patch is not actually needed, I used those CI logs to collect the numbers | 13:32 |
| gfidente | *yes* overcloud deploy runs for 50mins (out of 2h25mins) | 13:34 |
| yolanda | hi, i'm on the need of overriding ipxe template, with pxe_append_params in the undercloud. I see that this is possible on newton, i look at puppet-stack-config and i see support for drivers::pxe there, but not on liberty or mitaka, how can i handle this? | 13:34 |
| gfidente | it takes *10mins* to go from *nova please give me 4vms* to *here are your 4vms* | 13:34 |
| *** tobias_fiberdata has joined #tripleo | 13:34 | |
| hewbrocca | I guess the cloud is busy... | 13:35 |
| gfidente | and 40mins to do nodes configuration, maybe we can speed that up a little | 13:35 |
| gfidente | but undercloud configuration takes ~25mins | 13:35 |
| jaosorior | gfidente: well, doesn't nonha do introspection also? | 13:35 |
| slagle | gfidente: i just don't understand how steps 4 and 5 can take almost 35-40 minutes | 13:35 |
| gfidente | slagle right it's still a bit slower than undercloud in doing the puppet part | 13:37 |
| gfidente | but some nodes depend on others, if you take controllers only | 13:37 |
| gfidente | nodes are ready at 8.45 and deployed at 9.33 | 13:37 |
| *** jcoufal_ has joined #tripleo | 13:38 | |
| gfidente | if you remove validations and networking | 13:38 |
| gfidente | it goes from 8.56 to 9.33 | 13:39 |
| gfidente | and it makes 5 puppet steps | 13:39 |
| slagle | i wonder if it's all those openstackclient calls | 13:40 |
| gfidente | we could make it a bit faster I suppose but it's not really the 2h25mins | 13:40 |
| *** [1]cdearborn has joined #tripleo | 13:40 | |
| *** dprince has quit IRC | 13:40 | |
| *** jraju has quit IRC | 13:40 | |
| *** jcoufal has quit IRC | 13:40 | |
| slagle | yea, there is no one single thing | 13:40 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: Add undercloud reboot https://review.openstack.org/350489 | 13:40 |
| *** dprince has joined #tripleo | 13:41 | |
| slagle | gfidente: we've just made everything slower | 13:41 |
| *** ayoung has joined #tripleo | 13:42 | |
| hewbrocca | how excellent of us | 13:42 |
| gfidente | for example, it takes 40mins to build the image | 13:42 |
| hewbrocca | are we taking advantage of the SSDs at all, or is that no longer possible with OVB | 13:42 |
| gfidente | but I am unsure how we could speed that up | 13:43 |
| *** akuznetsov has joined #tripleo | 13:43 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Create entries for overcloud VIPs in /etc/hosts https://review.openstack.org/357765 | 13:43 |
| ayoung | jaosorior, ok...so got through some of the Newton based issues last night, ut can't deploy overcloud. Some reason it could not find enoughhosts. | 13:43 |
| *** abregman has quit IRC | 13:44 | |
| gfidente | if not by going with cached images and skip build entirely | 13:44 |
| jaosorior | ayoung: what steps did you take? | 13:44 |
| jaosorior | ayoung: and, are you using the latest quickstart? | 13:44 |
| ayoung | jaosorior, so ran quickstart like this... | 13:44 |
| ayoung | actually, starte by applying a patch | 13:45 |
| ayoung | jaosorior, this one https://review.openstack.org/#/c/335953/4 | 13:45 |
| ayoung | pretty sure it is the latest plus that | 13:45 |
| jaosorior | ayoung: Do you have this patch included? https://review.openstack.org/#/c/351683/ | 13:46 |
| jaosorior | ayoung: and, did you update the undercloud? | 13:46 |
| ayoung | jaosorior, OK, I don't think I have that one. Let me rebase and try again | 13:47 |
| *** akuznetsov has quit IRC | 13:47 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Don't trigger mariadb upgrade dump/restore when not needed https://review.openstack.org/358755 | 13:48 |
| EmilienM | tripleo meeting in 10 min | 13:48 |
| ayoung | jaosorior, I've been tearing down the undercloud manually...doing so now | 13:48 |
| EmilienM | please add off items here: https://etherpad.openstack.org/p/tripleo-meeting-items | 13:48 |
| ayoung | jaosorior, then | 13:49 |
| ayoung | bash quickstart.sh --release master-tripleo --extra-vars @config/general_config/devmode.yml --extra-vars force_cached_images=False ayoung-dell-t1700.test | 13:49 |
| saneax | ayoung, https://bugzilla.redhat.com/show_bug.cgi?id=1228862 | 13:49 |
| openstack | bugzilla.redhat.com bug 1228862 in instack-undercloud "Can `openstack undercloud install` have a --force-clean option so an error doesn't require restarting?" [Urgent,Closed: errata] - Assigned to jslagle | 13:49 |
| *** mburned_out is now known as mburned | 13:49 | |
| jaosorior | ayoung: alright, that makes sense. I use master (not master-tripleo) but I guess that shouldn't make much difference | 13:51 |
| ccamacho | hey jaosorior, re the OOM issues, at the end by adding some swap (not done by default in undercloud or overcloud) the OOM issues went away was able even to deploy mistral and sahara... This is awesome! This is how im doing it.. http://www.anstack.com/blog/2016/08/23/oom-swap-fix-in-tripleo.html now im trying to get different CI jobs as some of them dont have the host_info.txt.gz to see if they are using swap | 13:52 |
| gfidente | bnemec curiosity, do you have iowait of the ovb hoss? | 13:53 |
| gfidente | *hosts | 13:53 |
| *** cdearborn has quit IRC | 13:53 | |
| hewbrocca | and also the ovb hoss | 13:53 |
| hewbrocca | b/c it makes me think of Bonanza | 13:53 |
| jaosorior | ccamacho: nice! | 13:54 |
| jaosorior | ayoung: This is exactly what I do http://jaormx.github.io/2016/deploying-master-tripleo-quickstart/ | 13:54 |
| ccamacho | its a simple fix but makes the deployments work better, im even using now 6GB overcloud nodes!!! :P | 13:54 |
| ayoung | jaosorior, thanks. need to pass that on to alee and jdennis | 13:54 |
| jaosorior | ayoung: And what I do for FreeIPA http://jaormx.github.io/2016/freeipa-tripleo-integration-notes/ | 13:55 |
| ayoung | jaosorior, need to get you your own TLD. github should not get credit for your work. | 13:56 |
| ayoung | Both look like they are spectacularly useful. Thanks | 13:56 |
| slagle | derekh: which are the sdd drives in rh1? /dev/sd<what> | 13:56 |
| jaosorior | ayoung: adarazs mentioned that they're thinking on deprecating the master-tripleo option | 13:57 |
| ayoung | jaosorior, ok...so long as it works this time, I'll be happy | 13:57 |
| *** dprince has quit IRC | 13:58 | |
| adarazs | yeah, not sure if it makes sense to use the upstream images vs. building it with the image building role in centos ci. | 13:58 |
| ayoung | but I'll move over to master next. It seems righter. | 13:58 |
| jaosorior | ayoung: I use just master, and then go ahead and rebuild the images | 13:58 |
| adarazs | as we're not testing it extensively. | 13:58 |
| derekh | slagle: it depends on the host your on, iirc some have 2 spinning hd's and others have 4, but its usually the last one on the list | 13:58 |
| derekh | slagle: iirc I think its also the smallest one | 13:58 |
| *** dprince has joined #tripleo | 13:58 | |
| EmilienM | meeting is about to start in a minute, get ready! | 13:58 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Set the default gnocchi driver to swift https://review.openstack.org/354385 | 13:59 |
| *** zaneb has quit IRC | 13:59 | |
| derekh | slagle: when it was deployed the compute nodes just started using them for / (at least the ones I checked) | 13:59 |
| slagle | derekh: ok, i was wondering b/c lsblk is saying all the disks are rotational | 14:00 |
| slagle | the but last disk is mounted at / | 14:00 |
| slagle | so, i guess | 14:00 |
| *** zaneb has joined #tripleo | 14:01 | |
| *** fragatina has quit IRC | 14:01 | |
| gfidente | derekh do we have iowait from the hosts? | 14:02 |
| *** pcaruana has quit IRC | 14:02 | |
| derekh | gfidente: do you mean are we monitoring it? | 14:02 |
| gfidente | no just sample data | 14:02 |
| gfidente | like a 1minute sample | 14:03 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: TEST: DONT RECHECK: periodic jobs https://review.openstack.org/359215 | 14:03 |
| derekh | gfidente: I'm sure we could get it, | 14:03 |
| gfidente | longer better, ideally on a loaded host but I supposed load is spread on all of them in a similar way | 14:03 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: Don't trigger mariadb upgrade dump/restore when not needed https://review.openstack.org/359218 | 14:04 |
| *** tbarron|gone is now known as tbarron | 14:05 | |
| *** coolsvap_ is now known as coolsvap | 14:05 | |
| social | what ironic devs do we have here? | 14:05 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: WIP Add a function to upgrade from full HA to NG HA https://review.openstack.org/358626 | 14:06 |
| derekh | slagle: I'm gonna guess its reported as rotational because we added it as a volume in PERC | 14:07 |
| slagle | derekh: yea, agreed | 14:07 |
| slagle | the speed seems about right for an ssd | 14:07 |
| slagle | i'm getting around 400MB/s | 14:08 |
| slagle | with my scientific dd tests | 14:08 |
| slagle | i don't actually think there's anything going on here other than we just keep making shit slower | 14:08 |
| slagle | we lost 10 minutes on the undercloud install alone by just moving it all to puppet | 14:09 |
| * bnemec wonders if we're still taking a hit on node registration and introspection too | 14:10 | |
| *** lucas-hungry is now known as lucasagomes | 14:10 | |
| bnemec | Also worth noting that I found a few compute hosts in rh1 that don't appear to have ssds. | 14:10 |
| slagle | bnemec: what's the node registration problem? | 14:11 |
| slagle | besides just the general mistral slowness | 14:11 |
| bnemec | slagle: When we went to the Mistral version of node registration it added 2-3 minutes to that step. | 14:12 |
| *** ebalduf has joined #tripleo | 14:12 | |
| bnemec | I was told that it was because we were setting the provisioning state on the nodes unnecessarily for some reason. | 14:12 |
| d0ugal | bnemec: Really? That's odd :/ | 14:12 |
| slagle | bnemec: k, yea, that's the same thing i was thinking | 14:13 |
| d0ugal | ah :/ | 14:13 |
| d0ugal | Is there a bug for this? I could look into it | 14:13 |
| bnemec | I mean, it doesn't explain why our jobs went from taking 1:40 to 2:30 alone. | 14:13 |
| bnemec | But enough of those sorts of things and you might get there. | 14:13 |
| bnemec | d0ugal: I don't know. I believe I heard this in an IRC conversation when I brought up the sudden slowness in that step. | 14:14 |
| gfidente | fwiw, in https://review.openstack.org/#/c/354613/ the --register-nodes took only 2mins | 14:15 |
| *** hewbrocca is now known as hewbrocca-afk | 14:15 | |
| d0ugal | bnemec: k, thanks - I'll take a look later anyway when I get a moment. | 14:16 |
| social | dtantsur|mtg: https://bugs.launchpad.net/ironic/+bug/1616092 but if someone from ironic pokes me I think I should be able to track it down | 14:16 |
| openstack | Launchpad bug 1616092 in Ironic "code path where capabilities aren't converted" [Undecided,New] | 14:16 |
| slagle | gfidente: yea i see that | 14:16 |
| *** pcaruana has joined #tripleo | 14:17 | |
| gfidente | slagle I am trying to figure how much it took to puppet to compile the overcloud manifest, which should be cpu-bound | 14:17 |
| gfidente | don't ask me why but it's only loggin up to step 3 ;P | 14:19 |
| slagle | gfidente: 4, 7, 18, 56, 120 | 14:19 |
| *** rajinir has joined #tripleo | 14:19 | |
| slagle | in seconds, for each step | 14:19 |
| slagle | i did: journalctl --file ./journal/b7177a4eb7d9df34f0b10265f669e835/system.journal -u os-collect-config | grep -i 'compiled catalog' | 14:20 |
| gfidente | slagle cool thanks ;) | 14:20 |
| slagle | so 2 minutes just to compile the catalog in step 5 | 14:20 |
| derekh | gfidente: looks ok on a random compute node http://paste.openstack.org/show/562479/ | 14:21 |
| gfidente | slagle right and on my VMs it takes 40secs | 14:21 |
| gfidente | which means cpu is probably about three times slower | 14:21 |
| slagle | gfidente: what do your vm's have for cpu/ram? | 14:21 |
| gfidente | qemu virtual cpu | 14:22 |
| gfidente | on a xeon | 14:22 |
| slagle | vcpus? | 14:22 |
| gfidente | 1 vcpu | 14:22 |
| gfidente | let's check multinode | 14:22 |
| slagle | ok, same as CI then | 14:22 |
| gfidente | as it only takes 1h ! | 14:22 |
| dtantsur|mtg | social, moving to tripleo, this has nothing really to do with ironic | 14:22 |
| slagle | multinode vm's have 4 vcpu and 8GB ram | 14:23 |
| slagle | i wonder if we shouldnt just make ovb match that | 14:23 |
| slagle | it would at least match what infra uses | 14:23 |
| dtantsur|mtg | jpich, I think you've faced https://bugs.launchpad.net/tripleo/+bug/1616092 already, haven't you? | 14:23 |
| openstack | Launchpad bug 1616092 in tripleo "code path where capabilities aren't converted" [Undecided,New] | 14:23 |
| *** dsariel has joined #tripleo | 14:23 | |
| gfidente | slagle multinode makes step5 in 27secs | 14:24 |
| jpich | dtantsur|mtg: I have | 14:24 |
| sshnaidm | slagle, bnemec do you know what is cache mode on virtual disks on ci? | 14:24 |
| jpich | dtantsur|mtg: But it was using my Brand New Code paths I think. Maybe it's better to fix everywhere though | 14:24 |
| dtantsur|mtg | yeah | 14:25 |
| gfidente | derekh nice so iowait is pretty much 0 so it's not disk issues it seems | 14:25 |
| jpich | dtantsur|mtg: I'll add a comment with what I saw and how I fixed it, in case it turns out handy | 14:25 |
| dtantsur|mtg | thanks! | 14:25 |
| gfidente | slagle bnemec ^^ instead looks like CPU has much bigger impact | 14:25 |
| slagle | let me see if i can run a multinode job in rh1 | 14:25 |
| slagle | i'll need to set it up manually, but that will give us something to compare to | 14:26 |
| bnemec | sshnaidm: I believe we're using unsafe caching in CI, so it should be the fast, but you lose everything if a system goes down setting. | 14:26 |
| *** dtantsur|mtg is now known as dtantsur | 14:26 | |
| derekh | gfidente: at least on that random host over a minute | 14:26 |
| bnemec | We were having CPU throttling issues back before the move to OVB. I wonder if those could be back. | 14:27 |
| gfidente | derekh yeah are there reasons to think it changes much on a different node? | 14:27 |
| gfidente | first output from iowait are stats from boot, and it still says 0.01, 95% idel | 14:27 |
| gfidente | *idel | 14:27 |
| * bnemec needs to get his rh1 monitoring set up on his new laptop again | 14:27 | |
| derekh | gfidente: well bnemec mentioned that a couple of the compute nodes don't have the SSD (we may not have configure them in perc properly) | 14:28 |
| bnemec | Yeah, the list I have is .53, .55, .57 | 14:28 |
| *** maeca1 has joined #tripleo | 14:28 | |
| bnemec | I think those are the provisioning addresses for the compute nodes that appeared to have rotational disks. | 14:29 |
| *** adam_g has quit IRC | 14:29 | |
| bnemec | I only checked ~half of them though. | 14:29 |
| asbishop | bnemec gfidente slagle: hoping you can look at https://review.openstack.org/#/c/354839/ | 14:29 |
| *** dhill_ has joined #tripleo | 14:31 | |
| openstackgerrit | Merged openstack/puppet-tripleo: Clean up unused params from aodh profiles https://review.openstack.org/356523 | 14:31 |
| openstackgerrit | Merged openstack/instack-undercloud: (mitaka only) Fix sysctl_settings configuration https://review.openstack.org/356754 | 14:31 |
| *** jaosorior is now known as jaosorior_away | 14:32 | |
| jpeeler | rbrady: i see your review is also having difficulty with the nonha gate | 14:33 |
| jpeeler | slagle: after rechecking yesterday the nonha gate still didn't pass for me either https://review.openstack.org/#/c/348537/ | 14:34 |
| *** adam_g has joined #tripleo | 14:34 | |
| *** adam_g has quit IRC | 14:34 | |
| *** adam_g has joined #tripleo | 14:34 | |
| rbrady | jpeeler: neither of our patches are being run in CI that I know of, so these are unrelated failures | 14:35 |
| jpeeler | exactly, but nobody wants to approve them until they pass | 14:35 |
| slagle | jpeeler: exactly, so work on making them pass | 14:35 |
| slagle | or don't, and they won't get approved | 14:36 |
| *** davidlenwell has quit IRC | 14:36 | |
| openstackgerrit | Merged openstack/puppet-tripleo: Move ceilometer api to run under apache wsgi https://review.openstack.org/352476 | 14:36 |
| openstackgerrit | Jiri Stransky proposed openstack/puppet-tripleo: Add manifests to inject and trust CA certificates https://review.openstack.org/353021 | 14:36 |
| *** davidlenwell has joined #tripleo | 14:37 | |
| *** ChanServ sets mode: +v davidlenwell | 14:37 | |
| jpeeler | slagle: is there not anybody else to work with on this? i'm not asking you to do the work for me | 14:39 |
| jpeeler | i don't think i have the necessary knowledge of tripleo to debug these failures | 14:41 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: DO NOT MERGE TESTING CI JOBS https://review.openstack.org/359237 | 14:43 |
| *** fragatina has joined #tripleo | 14:43 | |
| *** akuznetsov has joined #tripleo | 14:43 | |
| slagle | jpeeler: i dont know who can debug them for you | 14:43 |
| *** zoli|trng is now known as zoli | 14:44 | |
| *** zoli is now known as zoliXXL | 14:44 | |
| rbrady | slagle: do we still have a CI team? | 14:45 |
| slagle | rbrady: to do what exactly? | 14:46 |
| rbrady | slagle: I thought we used to have a group of people who had knowledge/experience and access to CI to fix when issues arose. | 14:47 |
| rbrady | slagle: not sure if that group still exists | 14:47 |
| slagle | we have such a group of people | 14:48 |
| *** akuznetsov has quit IRC | 14:48 | |
| slagle | i'm not hearing about an issue for that group to fix (i don't think) | 14:48 |
| jpeeler | it was just briefly mentioned in the meeting | 14:48 |
| rbrady | slagle: the heat stack create is failing, but the patches in question aren't tested in CI - they shouldn't be the cause of the failure | 14:49 |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-docs: Documentation for installing and using Ironic in overcloud https://review.openstack.org/354016 | 14:49 |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-docs: [WIP] Document hybrid virtual and bare metal overcloud https://review.openstack.org/359240 | 14:49 |
| gfidente | bnemec if we can install sysstat | 14:49 |
| gfidente | it'll start collecting data with sar | 14:49 |
| rbrady | slagle, jpeeler: I guess we need to file a bug first? | 14:50 |
| gfidente | maybe we can check the iowait in a longer timeframe then | 14:50 |
| slagle | rbrady: jpeeler : to be clear on my stance, it is everyone's job to work on CI | 14:50 |
| gfidente | including CPU as things seem cpu-limited given puppet compile times | 14:50 |
| slagle | rbrady: jpeeler : yes, i'd file a bug and help on finding the fix. the maybe the job will pass on your patch | 14:50 |
| ccamacho | there was a repo promote, is the submission rebased? http://logs.openstack.org/37/348537/4/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/c2b5efe/logs/postci.txt.gz | 14:51 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates: [WIP] Add fluentd client service https://review.openstack.org/353506 | 14:51 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/puppet-tripleo: [WIP] updates to fluentd support https://review.openstack.org/353507 | 14:51 |
| rbrady | slagle: I get that we have to be good citizens and pitch in. It's just inefficient to tell someone who works in a small subset of tripleo to go tackle debugging a CI job alone | 14:51 |
| ayoung | jaosorior_away, that looked better. When I ran in the past I didn't get the Virtual Environment Setup Complete screen. This time I did. | 14:51 |
| slagle | rbrady: jpeeler : but i dont feel that there is "another group" to assign it to | 14:51 |
| ccamacho | sometimes I have these issues and rebasing all as there were jobs passing CI i.e. https://review.openstack.org/#/c/357312/ | 14:52 |
| slagle | if people get stuck, we can try and get help or experts to help get them unstuck | 14:52 |
| *** Goneri has quit IRC | 14:52 | |
| slagle | rbrady: i agree that would be inefficient | 14:53 |
| slagle | but no one has told anyone that | 14:53 |
| slagle | rbrady: to be clear, this convo started with "CI job is failing, it's not my patch" | 14:53 |
| slagle | i would not expect someone else to take over at that point :) | 14:54 |
| jpeeler | apologies for that, that's my fault | 14:54 |
| slagle | let's get a bug (if there isn't one), some triage done, some sufficient log diving, etc to see if anyone knows what's going on | 14:55 |
| jpeeler | part of the problem is it hasn't been failing consistently | 14:56 |
| jpeeler | i'll do one more recheck to see if yesterday's problem is confusing things | 14:56 |
| ccamacho | jpeeler, not recheck, mind to rebase if not rebased? | 14:57 |
| ccamacho | just to be sure | 14:57 |
| colonwq | jpeeler, slagle Rabbitmq-server failed to start on controller 0. | 14:57 |
| colonwq | I have this happen 1/5 of the times I test locally on my laptop. | 14:58 |
| *** maeca1 has quit IRC | 14:58 | |
| *** weshay is now known as weshay_afk | 14:58 | |
| jpeeler | ccamacho: force a rebase? i can try that. | 14:59 |
| openstackgerrit | Jeff Peeler proposed openstack/tripleo-common: Change output for GetCapabilitiesAction https://review.openstack.org/348537 | 14:59 |
| colonwq | entry in Aug 22 23:21:12 of http://logs.openstack.org/37/348537/4/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/c2b5efe/logs/overcloud-controller-0/var/log/messages | 15:00 |
| sshnaidm | zoliXXL, hi | 15:01 |
| zoliXXL | hi sshnaidm - I am dialing in | 15:01 |
| jpeeler | colonwq: thanks, rabbit is failing on mine as well. did you already file a bug? | 15:02 |
| zoliXXL | sshnaidm, just need to struggle with BJN | 15:02 |
| ccamacho | jpeeler, nice lets wait for a fresh CI | 15:02 |
| colonwq | jpeeler, I was thinking it was my environment since I bootlegged it locally. I run the integration stack but my VM and network config may be different. | 15:03 |
| jpeeler | ccamacho: to be clear though, wouldn't a rebase only change the reviewed code. that is a recheck wouldn't use anything the latest CI correct? | 15:03 |
| colonwq | rabbitmq connects to epmd but failes some times. epmd is running by the time I look at it. Rerunning the puppet config a second time does not help. | 15:04 |
| *** Goneri has joined #tripleo | 15:07 | |
| *** weshay_afk is now known as weshay | 15:07 | |
| ccamacho | you are rebasing your patchset to latest master, right? then will run latest tht master (with your changes) against all the other latest master repos, right? I had sometimes issues that having old puppet-tripleo submissions using latest tht in which case.. Had some module dependencies issues, and rebasing everything up to HEAD made work correctly. | 15:08 |
| ccamacho | jpeeler ^ | 15:08 |
| jpeeler | ah i see what you're saying | 15:08 |
| *** hewbrocca-afk is now known as hewbrocca | 15:08 | |
| *** jtomasek has quit IRC | 15:08 | |
| ccamacho | jpeeler I'm saying what I usually do :P "But it's not in any case a must" | 15:09 |
| gfidente | chem do you have submissions still up for review to enable the updates/upgrades jobs? | 15:11 |
| chem | gfidente: you mean gerrit review for the upgrade ? | 15:13 |
| gfidente | chem yep, upgrade and update | 15:13 |
| gfidente | I remember we moved update in the experimental queue, correct? | 15:13 |
| chem | gfidente: yep a lot :) | 15:14 |
| chem | gfidente: https://review.openstack.org/358022 , https://review.openstack.org/358006 , https://review.openstack.org/358005 , https://review.openstack.org/357022 , https://review.openstack.org/354713 , https://review.openstack.org/357750 | 15:14 |
| dtantsur | folks, do THT allow to create some overcloud objects as part of the installation? e.g. networks, host aggregates, etc? | 15:14 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/puppet-tripleo: [WIP] updates to fluentd support https://review.openstack.org/353507 | 15:14 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates: [WIP] Add fluentd client service https://review.openstack.org/353506 | 15:15 |
| chem | gfidente: maybe you know the answer to this one. Is there a way to send back and forth information from the overcloud during deployment to the undercloud like writing/reading in the undercloud swift storage ? | 15:16 |
| gfidente | chem no, we thought about adding a tempurl in undercloud swift but never built on it | 15:17 |
| hewbrocca | dtantsur: we create keystone endpoints | 15:17 |
| hewbrocca | and I think that's done by puppet execing a script that talks to keystone (if I'm not mistaken) | 15:18 |
| dtantsur | hewbrocca, IIRC it's not in THT, rather some specific python code in os-cloud-config, no? | 15:18 |
| dtantsur | hmm, maybe | 15:18 |
| hewbrocca | no b/c os-cloud-config.die.die.die | 15:18 |
| hewbrocca | the *new* way IIUC is do it with puppet | 15:18 |
| gfidente | dtantsur it's not in tht no, you'd have to do it in puppet | 15:18 |
| EmilienM | gfidente, dprince, slagle: I would be super happy to have feedback on https://review.openstack.org/#/c/356675/ | 15:19 |
| dtantsur | anyway, we have an unpleasant chicken-and-egg problem with ironic requiring a network UUID to *start* in case of multi-tenant | 15:19 |
| chem | gfidente: I asked as it could be a easy way to sync the overcloud node during upgrade, but maybe there is another way ? | 15:19 |
| dtantsur | gfidente, so e.g. for networks I need support in puppet-neutron, right? | 15:19 |
| gfidente | chem you mean steps syncing? | 15:20 |
| chem | gfidente: no within script, like non boostrap node waiting for success on the boostrap node or the like | 15:20 |
| gfidente | dtantsur yeah, or you can pull in puppet-neutron in puppet-ironic and make it create the network you need if you know the other parameters | 15:20 |
| *** ifarkas is now known as ifarkas_afk | 15:21 | |
| chem | gfidente: a node pooling a undercloud swift file like a lock and waiting for the right output with a curl in a script | 15:21 |
| gfidente | chem right this is what we usually need the steps for | 15:22 |
| gfidente | bootstrp does it in step-1 | 15:22 |
| chem | gfidente: ack, oki, will look closer into it, thanks | 15:23 |
| gfidente | chem point is that heat gives guarantee about the steps | 15:23 |
| gfidente | chem so you can make bootstrap do things before the others | 15:23 |
| thrash | d0ugal: ok. it was user error/misunderstanding. Once the correct templates were uploaded to swift, it worked. | 15:25 |
| *** abregman has joined #tripleo | 15:25 | |
| hewbrocca | gfidente: dtantsur wants to create an *overcloud* neutron network though | 15:27 |
| *** davidlenwell has quit IRC | 15:27 | |
| hewbrocca | gfidente: not an undercloud neutron network... | 15:27 |
| gfidente | hewbrocca right if it was undercloud we could have used THT | 15:27 |
| hewbrocca | OK | 15:28 |
| hewbrocca | just making sure that part was clear | 15:28 |
| *** larstobi has quit IRC | 15:28 | |
| *** yamahata has joined #tripleo | 15:29 | |
| gfidente | dtantsur so this guy https://github.com/openstack/puppet-neutron/blob/master/lib/puppet/provider/neutron_network/neutron.rb from puppet-neutron can create neutron networks | 15:29 |
| gfidente | though we don't create any network in overcloud neutron | 15:30 |
| d0ugal | thrash: phew :) | 15:30 |
| *** dprince has quit IRC | 15:30 | |
| d0ugal | thrash: it will be easier when we have a "create plan" command that does the container create and upload. | 15:30 |
| dtantsur | gfidente, interesting, thanks! | 15:30 |
| *** abregman is now known as abregman|mtg | 15:31 | |
| *** dprince has joined #tripleo | 15:32 | |
| gfidente | dtantsur it will get keuystone credentials from neutron.conf https://github.com/openstack/puppet-neutron/blob/master/lib/puppet/provider/neutron.rb#L31 it seems | 15:33 |
| gfidente | dtantsur so if you try a neutron_network after overcloud keystone is ready it might get you somewhere | 15:33 |
| dtantsur | yeah, probably I'll have to pick one node to run on it... (or will it run correctly on several nodes in parallel?) | 15:34 |
| *** larstobi has joined #tripleo | 15:34 | |
| gfidente | right, bootstrap_nodeid that is | 15:34 |
| gfidente | you can pick that in puppet-tripleo ironic role | 15:34 |
| dtantsur | okie. I'm still trying to solve it on ironic side, but will fallback to this approach if I fail there. thanks! | 15:37 |
| thrash | d0ugal: gotcha | 15:37 |
| *** davidlenwell has joined #tripleo | 15:40 | |
| *** ChanServ sets mode: +v davidlenwell | 15:40 | |
| *** abehl has quit IRC | 15:42 | |
| *** tremble has quit IRC | 15:42 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Migrate the deploy command to use the Mistral workflows https://review.openstack.org/342261 | 15:43 |
| gfidente | chem EmilienM but is 'updates' meant to become 'experimental' ? | 15:44 |
| gfidente | or should it be gating? | 15:44 |
| EmilienM | updates should be gating probably | 15:44 |
| EmilienM | but upgrades are really exp | 15:44 |
| gfidente | slagle can we do netiso in multinode? | 15:45 |
| slagle | gfidente: yes, i think we could | 15:45 |
| slagle | gfidente: the job does not yet use os-net-config to configure networking though | 15:45 |
| slagle | would need to do add that first | 15:45 |
| chem | gfidente: not sure what you mean here ? | 15:46 |
| *** tesseract- has quit IRC | 15:46 | |
| gfidente | cause I am trying to bring back ipv6 in ci | 15:46 |
| gfidente | ideally together with tls | 15:47 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Better namespace the list of TripleO actions https://review.openstack.org/357682 | 15:47 |
| gfidente | and nonha looked a good fit | 15:47 |
| gfidente | but previously we had it for updates job | 15:47 |
| gfidente | which will be based on multinode though | 15:47 |
| *** saneax is now known as saneax-_-|AFK | 15:51 | |
| slagle | gfidente: i think we just need to get it switched over to use os-net-config, and we could do that | 15:51 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Remove "type: direct" from workflows as it is the default https://review.openstack.org/341617 | 15:51 |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-common: Exclude more unneeded files from default plan https://review.openstack.org/355472 | 15:52 |
| *** pcaruana has quit IRC | 15:56 | |
| *** [1]cdearborn has quit IRC | 15:58 | |
| *** maeca1 has joined #tripleo | 15:58 | |
| *** [1]cdearborn has joined #tripleo | 15:59 | |
| gfidente | slagle like this guy? single-nic-linux-bridge-vlans/ceph-storage.yaml | 15:59 |
| gfidente | this one actually net-single-nic-linux-bridge-with-vlans.yaml | 16:00 |
| *** jaosorior_away is now known as jaosorior | 16:00 | |
| jaosorior | ayoung: nice! | 16:00 |
| slagle | gfidente: it's not going to be that simple | 16:00 |
| *** ohamada_ has quit IRC | 16:01 | |
| slagle | gfidente: you'll have to dig into how the vxlan tunnels are configured among the subnodes | 16:01 |
| jaosorior | jistr: ping | 16:01 |
| jistr | jaosorior: pong | 16:01 |
| slagle | gfidente: and come up with an os-net-config template that configures the tunnels the same way the function does from devstack-gate | 16:01 |
| slagle | gfidente: and we'll also need a step to bootstrap the initial networking via os-net-config so that we can start the polling from heat | 16:02 |
| jaosorior | jistr: by the way, I'm testing the CA profile stuff in CI here https://review.openstack.org/#/c/357791/3 | 16:02 |
| jistr | jaosorior: ah cool thanks for the info. Btw not sure if you noticed yet, i did a mini tweak to https://review.openstack.org/#/c/353021/ as i noticed a typo. | 16:03 |
| jistr | +2'd also | 16:03 |
| jistr | thanks for updating that one | 16:03 |
| gfidente | slagle so the bridge I see in net-config-multinode.yaml I can't just add vlans in there? | 16:03 |
| jaosorior | jistr: ah damn, where was the typo? | 16:03 |
| jaosorior | jistr: you mean the lack of .pem? | 16:04 |
| jistr | jaosorior: yea | 16:04 |
| jaosorior | jistr: thanks for updating that, dude | 16:04 |
| jistr | hehe sure thing | 16:04 |
| hewbrocca | dtantsur: do you have outstanding reviews you need for Ironic-in-overcloud etc.? | 16:04 |
| jistr | jaosorior: and btw kudos on your blog posts, just read 2 so far but helpful things, more in the queue after i get unleashed from downstream a bit more | 16:05 |
| dtantsur | hewbrocca, yes, but it's blocked by gate issues, I guess | 16:05 |
| dtantsur | hewbrocca, https://review.openstack.org/356539 | 16:05 |
| hewbrocca | that the only one? | 16:05 |
| jaosorior | jistr: if you find any info on the blog posts you can submit a bug report on github haha. Glad they're useful | 16:05 |
| dtantsur | hewbrocca, we've landed the majority :) there are some smaller improvements here and there, but this is the most critical. also docs https://review.openstack.org/354016, but here I use newer ironicclient commands, so I put it on hold. still worth reviewing. | 16:07 |
| ayoung | jaosorior, success | 16:07 |
| hewbrocca | dtantsur: thank you sir | 16:07 |
| ayoung | OK...now I am up to speed with Newton development.... | 16:07 |
| ayoung | let me make sure I can tear down and redeploy | 16:08 |
| *** pcaruana has joined #tripleo | 16:09 | |
| dsneddon | Does anyone know if the job "gate-tripleo-ci-centos-7-ovb-nonha" is having issues? I have been seeing patches fail with this error message: "Unable to establish connection to https://192.0.2.2:13004/v1/b41d5d6a89e44336ad06ccd7cbadacd2/stacks/overcloud/ac55763a-95e4-4001-8b11-103315bc050d" | 16:09 |
| dsneddon | Same failure for 4 or 5 days. | 16:10 |
| dtantsur | folks, what to do if stack deletion hangs? | 16:10 |
| dsneddon | dtantsur, Sometimes you need to restart the Ironic conductor and issue the delete command again. | 16:12 |
| dtantsur | dsneddon, well, it hasn't even started deleting nova instances | 16:12 |
| dtantsur | I've interrupted stack creation in the middle, that's why.. still, I don't think it should just hang | 16:12 |
| dsneddon | dtantsur, Yeah, I've had that happen, too. I think it happens when the actual state of the overcloud does not match what Heat expects. | 16:13 |
| dsneddon | dtantsur, Sometimes I have had luck by manually deleting nova instances, ports, etc. Sometimes I've even had to go into the DB to delete ports/routers/etc. | 16:13 |
| tzumainn | dtantsur, deleting the nova instances from the CLI works for me sometimes, I dunno if there's some reason I shouldn't do that though | 16:13 |
| dtantsur | lemme check | 16:14 |
| jaosorior | ayoung: aawww yeah! | 16:15 |
| *** jobewan has joined #tripleo | 16:16 | |
| dsneddon | dtantsur, When I've had to delete Neutron ports from the DB it is because the manual Nova delete didn't work, just FYI. | 16:16 |
| *** jpich has quit IRC | 16:16 | |
| dtantsur | nova delete worked for me fwiw | 16:16 |
| dtantsur | stack is still stuck, I guess it's easier to kill the whole env with fire | 16:17 |
| ebalduf | I'm trying to get a development environment setup. What am I missing here with TripleO-Quickstart? I can send it off with ./quickstart.sh -T all tripleo-testing.pm.solidfire.net and get a working version based on the RDO images, but when I send it off with ./quickstart.sh -T all --config config/general_config/devmode.yml --release master-tripleo tripleo-testing.pm.solidfire.net I keep getting dependency problems with the | 16:18 |
| ebalduf | undercloud deployment. I have tried to wipe out all caches and update everything with no luck. What am I missing here? | 16:18 |
| *** jaosorior has quit IRC | 16:18 | |
| *** zoliXXL is now known as zoli|gone | 16:19 | |
| *** morazi has joined #tripleo | 16:19 | |
| *** zoli|gone is now known as zoli_gone-proxy | 16:20 | |
| *** numans has quit IRC | 16:22 | |
| *** jbadiapa has quit IRC | 16:30 | |
| *** jubapa has quit IRC | 16:30 | |
| *** florianf has quit IRC | 16:32 | |
| *** jraju has joined #tripleo | 16:38 | |
| *** jraju has quit IRC | 16:39 | |
| *** bana_k has joined #tripleo | 16:40 | |
| *** ioggstream has quit IRC | 16:40 | |
| *** maeca1 has quit IRC | 16:42 | |
| dtantsur | hewbrocca, forgot to mention https://review.openstack.org/354125 (in progress by social) | 16:43 |
| dtantsur | it's optional, but we have a few manual steps without it | 16:43 |
| openstackgerrit | Richard Su proposed openstack/tripleo-common: Add List Roles Action https://review.openstack.org/330283 | 16:43 |
| dtantsur | not sure if we'll have enough time to wire it in before feature freeze though | 16:43 |
| hewbrocca | FFEs are going to get very expensive :) | 16:44 |
| *** nyechiel has quit IRC | 16:46 | |
| dtantsur | well, everything, except for the HAProxy patch, can be done manually, I even have it on my docs patch | 16:46 |
| slagle | sshnaidm: did you end up freeing up some space on the mirror server? | 16:47 |
| sshnaidm | slagle, yes | 16:47 |
| slagle | sshnaidm: thanks | 16:47 |
| sshnaidm | slagle, 13G is available now\ | 16:47 |
| *** yamahata has quit IRC | 16:48 | |
| derekh | slagle: once this merges, freeing up space wont be manual https://review.openstack.org/#/c/348806/ | 16:49 |
| slagle | cool | 16:49 |
| *** awayne has quit IRC | 16:49 | |
| hewbrocca | we still don't know why CI is slow though, right? | 16:49 |
| slagle | hopefully we'll get a pass tonight and will have some images | 16:50 |
| hewbrocca | other than we keep making it Slower ? | 16:50 |
| slagle | nothing definitive yet | 16:50 |
| slagle | but yes, we do keep making it slower | 16:51 |
| slagle | dprince: i think this is the most frequent erorr happening in nonha: http://logs.openstack.org/53/359153/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/4dd74c5/console.html | 16:52 |
| dtantsur | is there anyone here who knows a thing or two about host aggregates? | 16:53 |
| dtantsur | I'd like to check if https://review.openstack.org/#/c/359240/1/doc/source/advanced_deployment/baremetal_overcloud.rst makes any sense whatsoever | 16:53 |
| dprince | slagle: looking | 16:53 |
| gfidente | though I am suspicious about cpu being 3/5 times slower than I see in local test env | 16:54 |
| gfidente | slagle hewbrocca ^^ | 16:54 |
| ayoung | sshnaidm, thanks for your help last night. Got Overcloud deployment working this morning. | 16:55 |
| slagle | dprince: https://bugs.launchpad.net/tripleo/+bug/1616144 | 16:55 |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 16:55 |
| sshnaidm | ayoung, great! | 16:55 |
| sshnaidm | ayoung, using devmode? | 16:55 |
| ayoung | Next up is to figure out how to add an additional RPM/Apache module prior to brining up the HTTPD server | 16:55 |
| slagle | there is a connection reset by peer error in the heat api log | 16:55 |
| ayoung | sshnaidm, yeah | 16:55 |
| *** penick has joined #tripleo | 16:55 | |
| sshnaidm | ayoung, cool, so it works | 16:56 |
| ayoung | sshnaidm, I was a bit out of date, and had to rebase. | 16:56 |
| sshnaidm | weshay, ^^ | 16:56 |
| sshnaidm | ayoung, yeah, it's fast changing | 16:56 |
| weshay | sshnaidm, k cool.. I still think we need to evolve devmode. | 16:57 |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient: WIP wire in jinja templating for custom roles https://review.openstack.org/336625 | 16:57 |
| sshnaidm | weshay, maybe to arrange some mtg or ml about it? if removing it, then we need to think what to use instead | 16:58 |
| ayoung | sshnaidm, So, there is a feature we want to use a Heat Artifact to deploy, but it adds in a config file for Apache (something htat Puppet does not know about yet) and I need to have apache run with the module right up front. Is there some sort of pre-deploy hook I can use to do that? | 16:58 |
| weshay | sshnaidm, agree. thanks for keeping me in the loop w/ some examples | 16:58 |
| *** derekh has quit IRC | 16:59 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Convert overcloud.yaml to support jinja2 templating https://review.openstack.org/315679 | 16:59 |
| sshnaidm | ayoung, sorry.. I don't think I can help you with this, not heat expert yet :) | 17:00 |
| dprince | slagle: there is a heat-api failure there http://logs.openstack.org/53/359153/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/4dd74c5/logs/undercloud/var/log/heat/heat-api.txt.gz | 17:00 |
| slagle | dprince: yea, i added that in the bug | 17:00 |
| slagle | dprince: i wonder if that just due to the client closing the connection though? | 17:00 |
| dprince | slagle: yep, could be unrelated | 17:01 |
| *** dtantsur is now known as dtantsur|afk | 17:01 | |
| slagle | i wonder if it is load related, and the client attempts to connect and times out or something | 17:01 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: POC: WIP: oooq undercloud install https://review.openstack.org/358919 | 17:04 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Convert overcloud.yaml to support jinja2 templating https://review.openstack.org/315679 | 17:05 |
| *** anshul has joined #tripleo | 17:06 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Move AllNodesDeployments into jinja template loop https://review.openstack.org/337267 | 17:06 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Move AllNodesValidationDeployments into jinja template loop https://review.openstack.org/337587 | 17:07 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE Testin OOOQ job https://review.openstack.org/359146 | 17:07 |
| *** fragatina has quit IRC | 17:08 | |
| *** lucasagomes is now known as lucas-afk | 17:08 | |
| *** bana_k has quit IRC | 17:09 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Create per-role node_names in hieradata for all nodes https://review.openstack.org/348973 | 17:09 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Create composable mapping between enabled services and role ips https://review.openstack.org/348974 | 17:09 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Add service_ips in csv format to allNodes hieradata https://review.openstack.org/355066 | 17:10 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Move ceph cluster config out of allNodesConfig https://review.openstack.org/355067 | 17:10 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Generate composable service node_names lists https://review.openstack.org/355068 | 17:10 |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 17:10 |
| *** abregman|mtg is now known as abregman | 17:10 | |
| ayoung | EmilienM, is there a way to tell the Apache Puppet module to add in mod_auth_mellon? | 17:10 |
| EmilienM | ayoung: yes, https://github.com/puppetlabs/puppetlabs-apache#installing-apache-modules | 17:12 |
| *** abregman has quit IRC | 17:12 | |
| ayoung | EmilienM, I see in /usr/share/openstack-tripleo-heat-templates/puppet/controller.yaml there is an apache::mod | 17:12 |
| ayoung | # Horizon | 17:13 |
| ayoung | apache::mod::remoteip::proxy_ips: {get_input: horizon_subnet} | 17:13 |
| *** dsariel has quit IRC | 17:13 | |
| ayoung | EmilienM, does the puppet module need to know about the specific Apache module? If Puppet does not know about mod_auth_mellon, is there still a way to get it to install? | 17:13 |
| ayoung | ah apache::mod { 'mod_authn_mellon': } | 17:15 |
| ayoung | So I want OS::TripleO::ControllerExtraConfigPre | 17:16 |
| ayoung | and that should have | 17:16 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 17:17 |
| ayoung | Hmmm | 17:17 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/puppet-tripleo: updates to fluentd support https://review.openstack.org/353507 | 17:18 |
| *** jrist has quit IRC | 17:18 | |
| ayoung | EmilienM, (or anyone else) is it possible to add additional information on to: ControllerConfig: properties.config.hiera.controller.mapped_data without editing /usr/share/openstack-tripleo-heat-templates/puppet/controller.yaml | 17:19 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-specs: Spec: Centralized logging support https://review.openstack.org/356691 | 17:20 |
| *** bana_k has joined #tripleo | 17:24 | |
| *** fragatina has joined #tripleo | 17:24 | |
| *** dsariel has joined #tripleo | 17:27 | |
| *** yamahata has joined #tripleo | 17:28 | |
| *** Ryjedo has quit IRC | 17:29 | |
| *** Ryjedo has joined #tripleo | 17:31 | |
| *** mcornea has quit IRC | 17:32 | |
| *** sshnaidm is now known as sshnaidm|afk | 17:34 | |
| *** maeca1 has joined #tripleo | 17:36 | |
| *** jprovazn has quit IRC | 17:36 | |
| *** cdearborn has joined #tripleo | 17:38 | |
| *** milan has quit IRC | 17:41 | |
| *** tosky has quit IRC | 17:42 | |
| *** jcoufal_ has quit IRC | 17:45 | |
| *** jrist has joined #tripleo | 17:46 | |
| *** jmiu_ has quit IRC | 17:49 | |
| *** [1]cdearborn has quit IRC | 17:52 | |
| *** gfidente has quit IRC | 17:52 | |
| *** fragatina has quit IRC | 17:53 | |
| *** fragatina has joined #tripleo | 17:53 | |
| jobewan | Curious, should nova->ironic "node:" scheduler hints should work in OSP-7? I've updated my ironic capabilities and added the hostnamemap, and the hostname mapping is correct - although the ironic capability "node:" doesn't seem to be used in regard to the scheduler_hints . Any info as to where I can debug that? | 17:58 |
| tzumainn | hi! I have a dumb question - does tripleo support using glance v2 on the overcloud? | 18:02 |
| *** paramite has quit IRC | 18:03 | |
| *** akrivoka has quit IRC | 18:04 | |
| bnemec | jobewan: The predictable placement/hostname functionality wasn't in OSP 7. | 18:04 |
| *** jcoufal has joined #tripleo | 18:06 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: DO NOT MERGE - Periodic test. https://review.openstack.org/346949 | 18:08 |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 18:10 |
| jobewan | well, part of it is though right... So, to get predictive IPs, I added IPPool and NodeIndex to my compute templates, and created the ips-from-poo-all.yaml to use the existing "from_pool" yaml files in the network/ports - which works fine. The predictive naming is the hiccup really... Due to the str_replace functionality in 2015-04-30, I added the "node: hostname" params directly to the template - and "that" seems to work as well, to a degree... It's | 18:12 |
| jobewan | just randomly grabbing a hostname and not considering the hints. | 18:12 |
| *** cdearborn has quit IRC | 18:12 | |
| *** cdearborn has joined #tripleo | 18:13 | |
| jobewan | hints in ironic that is | 18:13 |
| *** hewbrocca is now known as hewbrocca-afk | 18:13 | |
| *** athomas has quit IRC | 18:14 | |
| *** ioggstream has joined #tripleo | 18:14 | |
| *** jcoufal has quit IRC | 18:17 | |
| *** jbadiapa has joined #tripleo | 18:17 | |
| *** jubapa has joined #tripleo | 18:17 | |
| *** bana_k has quit IRC | 18:19 | |
| *** electrofelix has quit IRC | 18:21 | |
| bnemec | jobewan: That's because there's a scheduler filter needed in order for the node hints to take effect. It didn't exist until Liberty (OSP 8). | 18:25 |
| *** jcoufal has joined #tripleo | 18:26 | |
| jobewan | bnemec: yea, that's what I'm looking for really. Is it simply a matter of enabling a filter in nova.conf, or rather actual coding differences? | 18:29 |
| *** pkovar has quit IRC | 18:30 | |
| bnemec | jobewan: It's new code | 18:30 |
| *** bana_k has joined #tripleo | 18:31 | |
| jobewan | So, is it safe to assume that I could use the augmented osp7 templates on an osp8 UC and achieve that functionality? | 18:31 |
| bnemec | jobewan: I would not call it "safe", no. :-) | 18:34 |
| bnemec | Possible? Sure. | 18:34 |
| bnemec | But at that point, why not just use OSP 8 templates too? | 18:34 |
| bnemec | *Liberty, since this is an upstream channel :-) | 18:34 |
| jobewan | osp8 templates req liberty right?, I can't upgrade from kilo yet | 18:34 |
| *** bana_k has quit IRC | 18:36 | |
| bnemec | Well, backporting Liberty features to Kilo is going to be unsupported anyway, but yes you could theoretically do an older deployment from a newer undercloud. | 18:38 |
| bnemec | It is a thing we support, but only for the features in the older release. | 18:38 |
| *** dtantsur|afk has quit IRC | 18:41 | |
| *** mhenkel has quit IRC | 18:46 | |
| *** bana_k has joined #tripleo | 18:55 | |
| *** maeca1 has quit IRC | 18:59 | |
| *** dsariel has quit IRC | 19:02 | |
| *** milan has joined #tripleo | 19:03 | |
| *** penick has quit IRC | 19:05 | |
| *** anshul has quit IRC | 19:05 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement non-ovb overcloud update job - Newton -> Newton https://review.openstack.org/351330 | 19:06 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 19:06 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 19:06 |
| *** nyechiel has joined #tripleo | 19:09 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 19:10 |
| *** trozet has quit IRC | 19:11 | |
| *** trozet has joined #tripleo | 19:12 | |
| *** ioggstream has quit IRC | 19:14 | |
| *** jtomasek has joined #tripleo | 19:18 | |
| *** jtomasek has quit IRC | 19:20 | |
| *** jtomasek_ has joined #tripleo | 19:20 | |
| *** pcaruana has quit IRC | 19:20 | |
| *** nyechiel has quit IRC | 19:24 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo: Expose enable_combination_alarm setting https://review.openstack.org/359387 | 19:31 |
| jobewan | so, is there a way to specify ctlplane IP addressing per node? | 19:33 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 19:34 |
| jobewan | err rather a "preferred method"... | 19:34 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: ensure the OPT_WORKDIR is set appropriately when executed from any directory https://review.openstack.org/333643 | 19:35 |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder: Refactor: block-device handling (local loop) https://review.openstack.org/319591 | 19:45 |
| *** dhill_ has quit IRC | 19:47 | |
| *** dhill_ has joined #tripleo | 19:47 | |
| *** dhill_ has quit IRC | 19:49 | |
| *** dhill_ has joined #tripleo | 19:49 | |
| openstackgerrit | imain proposed openstack/tripleo-heat-templates: WIP: Containerized Services for Composable Roles https://review.openstack.org/330659 | 19:53 |
| openstackgerrit | imain proposed openstack/tripleo-heat-templates: Bind mount files to run DiD in latest atomic host https://review.openstack.org/347218 | 19:53 |
| *** rasca has quit IRC | 20:02 | |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Update authtoken parameters to match recent changes https://review.openstack.org/359400 | 20:08 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Update authtoken parameters to match recent changes https://review.openstack.org/359401 | 20:09 |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 20:10 |
| *** anshul has joined #tripleo | 20:12 | |
| EmilienM | ok tripleo CI just broke | 20:15 |
| EmilienM | http://logs.openstack.org/91/354891/2/check/gate-tripleo-heat-templates-puppet-syntax-3-centos-7/c2a0d1b/console.html#_2016-08-23_18_17_32_703963 | 20:15 |
| EmilienM | I'm working on afix | 20:15 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Pin puppetlabs_spec_helper to 1.1.1 https://review.openstack.org/359406 | 20:19 |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Pin puppetlabs_spec_helper to 1.1.1 https://review.openstack.org/359407 | 20:20 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Pin puppetlabs_spec_helper to 1.1.1 https://review.openstack.org/359408 | 20:21 |
| EmilienM | dprince, slagle: we'll need that ^ asap | 20:21 |
| *** penick has joined #tripleo | 20:25 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add Ceph cluster health validation on upgrade https://review.openstack.org/359410 | 20:27 |
| *** [1]cdearborn has joined #tripleo | 20:29 | |
| *** NobodyCam has quit IRC | 20:29 | |
| *** NobodyCam has joined #tripleo | 20:30 | |
| *** rhefner has quit IRC | 20:31 | |
| bnemec | EmilienM: Is this going to be a problem going forward? Are we going to start seeing more and more puppet modules requiring newer Ruby? | 20:31 |
| *** coolsvap has quit IRC | 20:31 | |
| EmilienM | bnemec: it has been a problem since we have lint syntax and unit test jobs | 20:32 |
| EmilienM | fwiw, most of the problems are also affecting puppet CI so we detect & fix them early | 20:32 |
| *** rhefner has joined #tripleo | 20:33 | |
| bnemec | EmilienM: I guess I'm just wondering if pinning is a sustainable fix. Do we need to look into getting newer Ruby into CentOS so we can continue moving forward with the puppet modules? | 20:35 |
| EmilienM | bnemec: it's clearly a bug in the gem | 20:35 |
| EmilienM | bnemec: puppet community is working on it | 20:35 |
| EmilienM | but yeah having ruby 2.2 would be awesome | 20:35 |
| bnemec | EmilienM: Oh, so this is a temporary thing. | 20:35 |
| EmilienM | bnemec: what would be the process if we want to update ruby on centos? | 20:36 |
| EmilienM | I guess we need to contact the maintainer? | 20:36 |
| bnemec | EmilienM: I have no idea. I don't even know if we can bump a version like that. I suppose it's possible there's a Ruby SIG that maintains newer versions. | 20:37 |
| EmilienM | bnemec: I'll look into it | 20:37 |
| EmilienM | bnemec: thx for the +2, it helps! | 20:39 |
| *** cdearborn has quit IRC | 20:42 | |
| *** rhefner has quit IRC | 20:43 | |
| *** ioggstream has joined #tripleo | 20:43 | |
| *** oneswig has joined #tripleo | 20:44 | |
| *** rhefner has joined #tripleo | 20:48 | |
| *** jcoufal has quit IRC | 21:03 | |
| *** nyechiel has joined #tripleo | 21:05 | |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 21:10 |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| *** julim has quit IRC | 21:12 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 21:13 |
| *** lblanchard has quit IRC | 21:13 | |
| *** bfournie has quit IRC | 21:21 | |
| *** pradk has quit IRC | 21:24 | |
| *** ioggstream has quit IRC | 21:24 | |
| *** dprince has quit IRC | 21:26 | |
| *** nyechiel has quit IRC | 21:28 | |
| *** mburned is now known as mburned_out | 21:35 | |
| *** trozet has quit IRC | 21:36 | |
| *** asbishop has left #tripleo | 21:36 | |
| *** anshul has quit IRC | 21:40 | |
| *** Goneri has quit IRC | 21:43 | |
| *** egafford has quit IRC | 21:53 | |
| *** mburned_out is now known as mburned | 21:54 | |
| *** oneswig has quit IRC | 22:08 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616219 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 22:10 |
| openstack | Launchpad bug 1616219 in tripleo "latest puppetlabs_spec_helper now requires ruby 2.2" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 22:10 |
| *** ayoung has quit IRC | 22:27 | |
| *** ccamacho has quit IRC | 22:42 | |
| *** morazi has quit IRC | 22:44 | |
| *** trozet has joined #tripleo | 22:45 | |
| *** yamahata has quit IRC | 22:47 | |
| *** mburned is now known as mburned_out | 22:54 | |
| *** chlong has quit IRC | 23:03 | |
| *** yamahata has joined #tripleo | 23:09 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 23:10 |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616219 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1616219 in tripleo "latest puppetlabs_spec_helper now requires ruby 2.2" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 23:10 |
| *** rhallisey has quit IRC | 23:11 | |
| *** padkrish has joined #tripleo | 23:14 | |
| *** mpsairam has joined #tripleo | 23:15 | |
| *** jobewan has quit IRC | 23:18 | |
| *** saneax-_-|AFK is now known as saneax | 23:22 | |
| openstackgerrit | Merged openstack/python-tripleoclient: Fix unbracket_ipv6() https://review.openstack.org/359174 | 23:26 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Remve old cache files on the mirror server https://review.openstack.org/348806 | 23:26 |
| *** mlupton has joined #tripleo | 23:26 | |
| openstackgerrit | Merged openstack/tripleo-common: Add Plan Deletion https://review.openstack.org/299937 | 23:26 |
| mlupton | I had a failed undercloud installation and I'm trying to completely remove RabbitMQ so I can start fresh, how can I go about doing this? | 23:29 |
| mlupton | I'm running on RHEL and have tried doing yum remove rabbitmq, yum remove rabbitmq-server, but everything seems like it's still there | 23:30 |
| openstackgerrit | Merged openstack/instack-undercloud: Pin puppetlabs_spec_helper to 1.1.1 https://review.openstack.org/359407 | 23:32 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Pin puppetlabs_spec_helper to 1.1.1 https://review.openstack.org/359408 | 23:39 |
| openstackgerrit | Merged openstack/puppet-tripleo: Pin puppetlabs_spec_helper to 1.1.1 https://review.openstack.org/359406 | 23:40 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Use cached images https://review.openstack.org/359481 | 23:40 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/348695 | 23:42 |
| *** mburned_out is now known as mburned | 23:43 | |
| *** padkrish has quit IRC | 23:46 | |
| *** padkrish has joined #tripleo | 23:47 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Set cinder::api::bind_host within service template https://review.openstack.org/318344 | 23:53 |
| *** padkrish has quit IRC | 23:53 | |
| *** rlandy has quit IRC | 23:55 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!