| *** rlandy is now known as rlandy|afk | 00:02 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add deprecation notes for panko service https://review.openstack.org/460230 | 00:05 |
|---|---|---|
| EmilienM | stevebaker: hey it's me again \o/ | 00:05 |
| EmilienM | stevebaker: when I'm trying to promote again, now I have this error: http://logs.openstack.org/37/457037/7/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/5f1ec27/console.html#_2017-05-02_22_26_46_061856 | 00:08 |
| EmilienM | stevebaker: I didn't catch it yesterday because I wasn't that far | 00:08 |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 00:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 00:10 |
| stevebaker | EmilienM: I'll take a look after lunch | 00:15 |
| EmilienM | stevebaker: I'm staying online, I'm investigating now | 00:15 |
| stevebaker | ok | 00:15 |
| *** limao has joined #tripleo | 00:26 | |
| *** tzumainn has quit IRC | 00:27 | |
| *** artom has quit IRC | 00:43 | |
| *** funzo has joined #tripleo | 00:46 | |
| *** artom has joined #tripleo | 00:46 | |
| EmilienM | stevebaker: I'm not sure but i'm thinking about https://github.com/openstack/os-client-config/commit/e6755872ada4978f585bdf15edf623dbcf72c4ee | 00:50 |
| *** funzo has quit IRC | 00:50 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: DNM - deploy tripleo with all debug https://review.openstack.org/457037 | 00:57 |
| *** bfournie has left #tripleo | 00:58 | |
| *** Goneri has joined #tripleo | 01:03 | |
| openstackgerrit | Harald Jensås proposed openstack/instack-undercloud master: Tripleo routed networks ironic inspector, and Undercloud https://review.openstack.org/437544 | 01:07 |
| *** tiswanso has joined #tripleo | 01:07 | |
| *** bfournie has joined #tripleo | 01:07 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 01:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 01:10 |
| *** thrash is now known as thrash|g0ne | 01:10 | |
| stevebaker | EmilienM: is something writing out a ~/.config/openstack/clouds.yaml somewhere? | 01:20 |
| EmilienM | stevebaker: I don't think so | 01:20 |
| EmilienM | or at least not afik | 01:20 |
| EmilienM | stevebaker: I found https://review.openstack.org/#/c/434372/ suspicious | 01:20 |
| stevebaker | EmilienM: or /etc/openstack/clouds.yaml | 01:20 |
| EmilienM | I can check, gimme a sec | 01:21 |
| EmilienM | stevebaker: nothing | 01:22 |
| stevebaker | hmm | 01:22 |
| EmilienM | I need to be afk 15 Min | 01:25 |
| *** ramishra has joined #tripleo | 01:25 | |
| EmilienM | I think this is the last promotion blocker, other jobs are working | 01:25 |
| openstackgerrit | Merged openstack/tripleo-validations master: Updated from global requirements https://review.openstack.org/461410 | 01:29 |
| stevebaker | EmilienM: so this is on the second run of 'undercloud install' after ~/stackrc has been sourced. I think os-client-config is being configured on those credentials even though its an auth_required=False command | 01:31 |
| openstackgerrit | Merged openstack/tripleo-validations stable/ocata: Add 'become: true' to haproxy validation https://review.openstack.org/461795 | 01:32 |
| *** dmacpher has joined #tripleo | 01:40 | |
| EmilienM | stevebaker: yeah on the second run | 01:40 |
| EmilienM | an dyeah ~/stackrc has been sourced | 01:40 |
| EmilienM | it's failing at the idempotency test | 01:40 |
| EmilienM | what is weird is that I couldn't reproduce it in my env | 01:41 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Set LC_ALL in disk-image-create https://review.openstack.org/461950 | 01:49 |
| *** yamahata has joined #tripleo | 01:57 | |
| *** bkopilov_ has quit IRC | 01:58 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 02:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 02:10 |
| stevebaker | EmilienM: python-openstackclient 3.9.0-0.20170309094639.c0a23b8.el7.centos is ancient compared to git master. Master has a fix where get_one_cloud is only called with validate=True for auth_required=True commands. That should fix this issue | 02:14 |
| *** rlandy|afk is now known as rlandy | 02:20 | |
| EmilienM | stevebaker: ok, let me check | 02:22 |
| EmilienM | we pin on 3.9.0 | 02:23 |
| EmilienM | stevebaker: so there is no CI job out there that test osclient with os-client-config? | 02:23 |
| EmilienM | (except tripleo) | 02:23 |
| EmilienM | and https://github.com/openstack/python-openstackclient/releases is the latest | 02:23 |
| EmilienM | 3.9.0 is latest | 02:23 |
| EmilienM | so we need openstackclient release | 02:24 |
| stevebaker | EmilienM: ...with sourced credentials for a command that requires no auth | 02:24 |
| EmilienM | stevebaker: nice catch | 02:24 |
| EmilienM | thanks again for the help | 02:24 |
| *** japestinho has joined #tripleo | 02:24 | |
| EmilienM | I'm proposing a release of osclient | 02:24 |
| EmilienM | stevebaker: how many beers do I own you again? :) | 02:24 |
| stevebaker | EmilienM: heh, no problem. I think an osc release will do it | 02:26 |
| EmilienM | https://review.openstack.org/#/c/461959/ | 02:33 |
| *** Goneri has quit IRC | 02:36 | |
| *** yamahata has quit IRC | 02:40 | |
| EmilienM | stevebaker: they don't want to release yet | 02:46 |
| EmilienM | stevebaker: so https://review.rdoproject.org/r/#/c/6515/1 | 02:46 |
| *** funzo has joined #tripleo | 02:46 | |
| EmilienM | i'm out | 02:47 |
| *** funzo has quit IRC | 02:51 | |
| *** janki has joined #tripleo | 02:55 | |
| *** eck` is now known as eck`gone | 02:59 | |
| *** fragatin_ has joined #tripleo | 03:03 | |
| *** fragatina has quit IRC | 03:06 | |
| *** fragatin_ has quit IRC | 03:08 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 03:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 03:10 |
| *** bkopilov_ has joined #tripleo | 03:20 | |
| *** cmyster_ has quit IRC | 03:22 | |
| *** EmilienM sets mode: +o stevebaker | 03:26 | |
| *** EmilienM sets mode: +o beagles | 03:27 | |
| *** EmilienM sets mode: +o bandini | 03:27 | |
| *** EmilienM sets mode: +o marios | 03:27 | |
| *** EmilienM sets mode: +o chem | 03:27 | |
| *** psahoo has joined #tripleo | 03:27 | |
| *** tiswanso has quit IRC | 03:42 | |
| *** mich_chap has joined #tripleo | 03:46 | |
| *** links has joined #tripleo | 03:49 | |
| *** jchhatbar has joined #tripleo | 03:49 | |
| *** janki has quit IRC | 03:52 | |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 04:10 |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 04:10 |
| openstackgerrit | Merged openstack/diskimage-builder master: Set LC_ALL in disk-image-create https://review.openstack.org/461950 | 04:16 |
| *** chlong has joined #tripleo | 04:22 | |
| *** fragatina has joined #tripleo | 04:24 | |
| *** fragatina has quit IRC | 04:25 | |
| *** fragatina has joined #tripleo | 04:25 | |
| *** jchhatbar is now known as janki | 04:30 | |
| *** nmathew has joined #tripleo | 04:30 | |
| *** dmacpher is now known as dmacpher-afk | 04:31 | |
| *** ykarel has joined #tripleo | 04:36 | |
| *** yamahata has joined #tripleo | 04:40 | |
| *** mdnadeem has joined #tripleo | 04:52 | |
| *** yamahata has quit IRC | 04:55 | |
| *** dparkes has quit IRC | 04:57 | |
| *** udesale has joined #tripleo | 04:58 | |
| *** udesale has quit IRC | 05:00 | |
| *** udesale has joined #tripleo | 05:01 | |
| *** pgadiya has joined #tripleo | 05:05 | |
| *** enrraja has joined #tripleo | 05:09 | |
| *** ffiore has joined #tripleo | 05:09 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 05:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 05:10 |
| *** ffiore has quit IRC | 05:16 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs https://review.openstack.org/359215 | 05:18 |
| *** lmiccini has joined #tripleo | 05:21 | |
| *** ccamacho has quit IRC | 05:22 | |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder master: Add refactor of tree-like vs graph https://review.openstack.org/461353 | 05:27 |
| *** ffiore has joined #tripleo | 05:47 | |
| *** masco has joined #tripleo | 05:49 | |
| *** tvignaud_ has quit IRC | 05:50 | |
| *** tvignaud_ has joined #tripleo | 05:50 | |
| *** pkovar has joined #tripleo | 05:50 | |
| *** ffiore has quit IRC | 05:51 | |
| *** ffiore has joined #tripleo | 05:51 | |
| *** newmember has joined #tripleo | 05:52 | |
| *** dparkes has joined #tripleo | 05:55 | |
| *** limao has quit IRC | 05:55 | |
| *** limao has joined #tripleo | 05:55 | |
| *** limao has quit IRC | 05:56 | |
| *** limao has joined #tripleo | 06:00 | |
| *** ffiore has quit IRC | 06:00 | |
| *** anshul has joined #tripleo | 06:02 | |
| *** prateek has joined #tripleo | 06:06 | |
| *** limao has quit IRC | 06:06 | |
| *** iranzo has joined #tripleo | 06:07 | |
| *** iranzo has joined #tripleo | 06:07 | |
| *** dsneddon has joined #tripleo | 06:07 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: TLS everywhere: configure mongodb's TLS settings https://review.openstack.org/461780 | 06:09 |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 06:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 06:10 |
| *** milan has joined #tripleo | 06:10 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE: Testing LDAP integration https://review.openstack.org/461865 | 06:11 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: Deploy FreeIPA-backed LDAP domain for keystone in fakeha-caserver job https://review.openstack.org/461864 | 06:11 |
| *** anshul has quit IRC | 06:16 | |
| *** anshul has joined #tripleo | 06:16 | |
| *** ratailor has joined #tripleo | 06:18 | |
| *** jaosorior_away is now known as jaosorior | 06:19 | |
| *** yolanda has quit IRC | 06:20 | |
| *** ebarrera has joined #tripleo | 06:22 | |
| jaosorior | jistr, mandre: Hey guys, could you check this out https://review.openstack.org/#/c/459573/ ? It uses list_concat for the container templates. | 06:22 |
| *** jprovazn has joined #tripleo | 06:23 | |
| *** rwsu has quit IRC | 06:23 | |
| *** limao has joined #tripleo | 06:24 | |
| jaosorior | bandini: Is this ready to merge https://review.openstack.org/#/c/460028/4 ? | 06:24 |
| *** pgadiya has quit IRC | 06:25 | |
| *** newmember has quit IRC | 06:26 | |
| *** pcaruana has joined #tripleo | 06:27 | |
| *** ratailor_ has joined #tripleo | 06:33 | |
| *** ratailor has quit IRC | 06:34 | |
| *** yprokule has joined #tripleo | 06:35 | |
| bandini | jaosorior: yes it is, thanks | 06:36 |
| mandre | jaosorior: wow it looks so much nicer! | 06:36 |
| *** rwsu has joined #tripleo | 06:36 | |
| mandre | who do we need to thank for list_concat? | 06:37 |
| jaosorior | bandini: done | 06:37 |
| bandini | jaosorior: \o/ thanks | 06:37 |
| jaosorior | mandre: the heat devs who merged it. | 06:37 |
| *** dparkes has quit IRC | 06:37 | |
| jaosorior | mandre: it was reviewed pretty damn fast https://review.openstack.org/#/c/458510/ :) those guys are awesome | 06:38 |
| bandini | any takers for a nice and easy review? https://review.openstack.org/#/c/461013/ ;) | 06:39 |
| mandre | jaosorior: indeed, nice work guys | 06:39 |
| *** bkopilov has joined #tripleo | 06:41 | |
| bandini | jaosorior: thanks ;) | 06:42 |
| *** ratailor_ has quit IRC | 06:42 | |
| *** ratailor has joined #tripleo | 06:42 | |
| openstackgerrit | Merged openstack/tripleo-docs master: Add the update-plan-only step for the minor updates workflow https://review.openstack.org/461013 | 06:44 |
| *** florianf has joined #tripleo | 06:47 | |
| *** hjensas has quit IRC | 06:47 | |
| *** funzo has joined #tripleo | 06:48 | |
| *** limao has quit IRC | 06:48 | |
| *** dparkes has joined #tripleo | 06:49 | |
| *** agurenko has joined #tripleo | 06:50 | |
| *** aufi has joined #tripleo | 06:50 | |
| *** yprokule has quit IRC | 06:51 | |
| *** yprokule has joined #tripleo | 06:52 | |
| *** funzo has quit IRC | 06:52 | |
| *** gvrangan has joined #tripleo | 06:54 | |
| jaosorior | marios, mandre thanks for the review | 06:56 |
| *** shardy has joined #tripleo | 07:00 | |
| *** jlinkes has joined #tripleo | 07:02 | |
| *** ccamacho has joined #tripleo | 07:04 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 07:10 |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 07:10 |
| *** ffiore has joined #tripleo | 07:11 | |
| *** cmyster has joined #tripleo | 07:20 | |
| *** agurenko has quit IRC | 07:22 | |
| *** agurenko has joined #tripleo | 07:27 | |
| *** jpena|off is now known as jpena | 07:33 | |
| *** dsariel has joined #tripleo | 07:37 | |
| gvrangan | All, any tips ti use tripoleo quickstart in a proxy env | 07:37 |
| gvrangan | should we add http_proxy, https_proxy and proxy to all ansible yaml files? | 07:38 |
| openstackgerrit | Merged openstack/puppet-tripleo master: firewall: generally accept "jump" param and use tripleo:firewall for log rule https://review.openstack.org/441128 | 07:39 |
| *** gfidente|afk is now known as gfidente|doc | 07:39 | |
| *** dtantsur|afk is now known as dtantsur | 07:40 | |
| *** cmyster has quit IRC | 07:45 | |
| *** amoralej|off is now known as amoralej | 07:48 | |
| *** limao has joined #tripleo | 07:50 | |
| *** oanson has quit IRC | 07:52 | |
| *** oanson has joined #tripleo | 07:53 | |
| *** dbecker has quit IRC | 07:56 | |
| *** dbecker has joined #tripleo | 07:57 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** zzzeek has joined #tripleo | 08:00 | |
| *** ratailor has quit IRC | 08:01 | |
| *** nyechiel has joined #tripleo | 08:09 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 08:10 |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 08:10 |
| *** paramite has joined #tripleo | 08:11 | |
| *** ratailor has joined #tripleo | 08:11 | |
| *** gvrangan has quit IRC | 08:14 | |
| *** fragatina has quit IRC | 08:20 | |
| *** hjensas has joined #tripleo | 08:23 | |
| *** lucas-afk is now known as lucasagomes | 08:26 | |
| *** gvrangan has joined #tripleo | 08:30 | |
| *** ckyriakidou has joined #tripleo | 08:30 | |
| *** zoli|gone is now known as zoli | 08:32 | |
| openstackgerrit | Jan Provaznik proposed openstack/tripleo-docs master: Add documentation how to access manila shares https://review.openstack.org/459244 | 08:34 |
| *** cmyster has joined #tripleo | 08:35 | |
| *** sshnaidm|pto is now known as sshnaidm | 08:36 | |
| *** derekh has joined #tripleo | 08:39 | |
| *** milan has quit IRC | 08:41 | |
| *** bvandenh has quit IRC | 08:42 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Disable ComputeNeutron* for cisco-nexus-ucsm https://review.openstack.org/462035 | 08:45 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: TLS everywhere: configure mongodb's TLS settings https://review.openstack.org/461780 | 08:46 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates stable/ocata: Fix for the resource ControllerPostPuppetMaintenanceModeDeployment https://review.openstack.org/462037 | 08:49 |
| *** salmankhan has joined #tripleo | 08:53 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-quickstart-extras master: Enable deploy of TLS-everywhere containers https://review.openstack.org/456226 | 08:53 |
| *** stendulker has joined #tripleo | 08:54 | |
| *** mcornea has joined #tripleo | 08:56 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: Run introspection 20 nodes at a time https://review.openstack.org/439568 | 09:00 |
| *** dparkes has quit IRC | 09:04 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: NO MERGEY - testing low concurrency https://review.openstack.org/449215 | 09:06 |
| d0ugal | I wish I knew what I was thinking a month or two ago | 09:08 |
| apetrich | gchamoul, thanks for the change +1ed it | 09:08 |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 09:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 09:10 |
| *** dparkes has joined #tripleo | 09:13 | |
| openstackgerrit | Marios Andreou proposed openstack/puppet-tripleo master: WIP - allow libvirt tcp migration for upgrade. https://review.openstack.org/462047 | 09:14 |
| *** assassin has quit IRC | 09:15 | |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-common master: move mistral base action dependency to mistral_lib https://review.openstack.org/454632 | 09:19 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates stable/ocata: Fix for the resource ControllerPostPuppetMaintenanceModeDeployment https://review.openstack.org/462037 | 09:20 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates stable/ocata: Merge pre|post puppet resources into pre|post config. https://review.openstack.org/459572 | 09:20 |
| *** limao has quit IRC | 09:20 | |
| *** tosky has joined #tripleo | 09:21 | |
| *** yprokule has quit IRC | 09:23 | |
| *** artom has quit IRC | 09:23 | |
| *** artom has joined #tripleo | 09:23 | |
| *** yprokule has joined #tripleo | 09:23 | |
| *** artom has quit IRC | 09:23 | |
| *** artom has joined #tripleo | 09:24 | |
| *** yprokule has quit IRC | 09:24 | |
| *** yprokule has joined #tripleo | 09:24 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates stable/ocata: Fix for the resource ControllerPostPuppetMaintenanceModeDeployment https://review.openstack.org/462037 | 09:25 |
| *** vpickard_ has joined #tripleo | 09:25 | |
| *** vpickard has quit IRC | 09:25 | |
| *** ubijtsa has joined #tripleo | 09:27 | |
| *** ubijtsa has joined #tripleo | 09:27 | |
| *** zoli is now known as zoli|lunch | 09:28 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates master: WIP: Add AllowLibvirtMigrationTCP - libvirt tcp migration for upgrade. https://review.openstack.org/461367 | 09:28 |
| *** ubijtsa is now known as assassin | 09:30 | |
| *** athomas has joined #tripleo | 09:32 | |
| *** milan has joined #tripleo | 09:38 | |
| *** athomas has quit IRC | 09:44 | |
| *** udesale has quit IRC | 09:45 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Internal TLS: use common CA file parameter for libvirt CA cert https://review.openstack.org/460025 | 09:45 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Internal TLS: Use specific CA file for haproxy https://review.openstack.org/460024 | 09:45 |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-common master: move mistral base action dependency to mistral_lib https://review.openstack.org/454632 | 09:46 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Internal TLS: use common CA file parameter for libvirt CA cert https://review.openstack.org/460025 | 09:46 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Internal TLS: Use specific CA file for haproxy https://review.openstack.org/460024 | 09:46 |
| jokke_ | anyone around who could confirm something for me about upgrades? | 09:47 |
| jokke_ | Do we overwrite the policy.json during upgrade if it has changed or do we keep the changed version? | 09:48 |
| shardy | jokke_: we don't manage the policy by default, although there are puppet interfaces that can do it | 09:49 |
| shardy | so provided the packaging leaves the modifications we shouldn't overwrite them AFAIK | 09:50 |
| jokke_ | shardy: ok, cool. Thanks for that! | 09:50 |
| arxcruz | adarazs: sshnaidm can I have it reviewed https://review.openstack.org/#/c/459981/ ? | 09:52 |
| sshnaidm | arxcruz, sure | 09:53 |
| arxcruz | sshnaidm: thanks :) | 09:53 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo master: MySQL client: Make CA file configurable https://review.openstack.org/462062 | 09:55 |
| *** abishop has joined #tripleo | 09:55 | |
| jprovazn | dsneddon: hi, are you around? | 09:55 |
| openstackgerrit | Alfredo Moralejo proposed openstack/tripleo-quickstart master: Add subpackages from openstack-tripleo-common from delorean-current https://review.openstack.org/462063 | 09:55 |
| *** tesseract has joined #tripleo | 09:56 | |
| sshnaidm | arxcruz, you know, I'm not sure it's good idea not to build in periodic job | 09:57 |
| *** suuuper has joined #tripleo | 09:57 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Internal TLS: Use specific CA file for mysql-client https://review.openstack.org/462066 | 09:58 |
| sshnaidm | arxcruz, it means you will download last promoted image while other periodic jobs will build it and have newer code there. It will confuse | 09:59 |
| openstackgerrit | Alfredo Moralejo proposed openstack-infra/tripleo-ci master: Add subpackages from openstack-tripleo-common from delorean-current https://review.openstack.org/462067 | 09:59 |
| sshnaidm | arxcruz, if we don't have time - let's increase the timeout for job | 09:59 |
| arxcruz | sshnaidm: actually, weshay wanted it to not build and use the last known good image on these jobs | 10:00 |
| arxcruz | I'm okay to keep building even though last failures was due build image issues | 10:00 |
| arxcruz | today was timeout | 10:00 |
| sshnaidm | arxcruz, ok, it was the issue in all periodic jobs I suppose, so it's ok to fail | 10:03 |
| sshnaidm | arxcruz, let's wait for weshay then, can we set it on hold atm? | 10:04 |
| *** athomas has joined #tripleo | 10:07 | |
| dtantsur | jprovazn, he's in US west | 10:09 |
| dtantsur | 4-5pm our time is more realistic | 10:09 |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 10:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 10:10 |
| jprovazn | dtantsur: ack, thanks | 10:10 |
| arxcruz | sshnaidm: sure :) | 10:12 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: TLS everywhere: configure mongodb's TLS settings https://review.openstack.org/461780 | 10:17 |
| *** oanson has quit IRC | 10:19 | |
| *** oanson has joined #tripleo | 10:21 | |
| *** tvignaud_ has quit IRC | 10:24 | |
| *** jkilpatr has quit IRC | 10:31 | |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-pacemaker master: Fix a typo in ipv6 addrlabel https://review.openstack.org/462073 | 10:31 |
| *** ratailor_ has joined #tripleo | 10:32 | |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo master: IPv6 VIP addresses need to be /128 https://review.openstack.org/460028 | 10:32 |
| *** ratailor has quit IRC | 10:35 | |
| *** tvignaud_ has joined #tripleo | 10:36 | |
| sshnaidm | EmilienM, mandre, jistr I think we need deep-dive about containers in tripleo :) | 10:38 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Move containers common volumes from yaql to list_concat https://review.openstack.org/459573 | 10:43 |
| *** funzo has joined #tripleo | 10:46 | |
| *** jkilpatr has joined #tripleo | 10:49 | |
| *** adarazs is now known as adarazs_lunch | 10:51 | |
| *** pkovar has quit IRC | 10:51 | |
| *** funzo has quit IRC | 10:51 | |
| *** rbowen has joined #tripleo | 10:54 | |
| *** nyechiel has quit IRC | 10:55 | |
| *** thrash|g0ne is now known as thrash | 10:59 | |
| *** rbowen has quit IRC | 11:00 | |
| *** zoli|lunch is now known as zoli | 11:01 | |
| *** ratailor_ has quit IRC | 11:02 | |
| *** bkopilov has quit IRC | 11:03 | |
| *** bkopilov_ has quit IRC | 11:04 | |
| EmilienM | sshnaidm: for sure | 11:05 |
| dr_gogeta86 | anybody can help me with this ssl problem | 11:08 |
| dr_gogeta86 | ERROR (SSLError): SSL exception connecting to https://10.20.3.250:13774/v2.1: hostname '10.20.3.250' doesn't match 'mydomain.it' | 11:08 |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 11:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| *** stendulker has quit IRC | 11:10 | |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 11:10 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: NO MERGEY - testing low concurrency https://review.openstack.org/449215 | 11:13 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: Send a zaqar message on each introspection completion https://review.openstack.org/462078 | 11:13 |
| *** rhallisey has joined #tripleo | 11:14 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: NO MERGEY - testing low concurrency https://review.openstack.org/449215 | 11:17 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: Send a zaqar message on each introspection completion https://review.openstack.org/462078 | 11:17 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: Remove update abort https://review.openstack.org/449242 | 11:21 |
| *** gfidente|doc is now known as gfidente | 11:23 | |
| *** lucasagomes is now known as lucas-hungry | 11:28 | |
| *** athomas has quit IRC | 11:30 | |
| *** rbowen has joined #tripleo | 11:30 | |
| *** adarazs_lunch is now known as adarazs | 11:34 | |
| *** tosky has quit IRC | 11:34 | |
| *** nmathew has quit IRC | 11:39 | |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Use httpd in Heat docker services https://review.openstack.org/447504 | 11:40 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: Run introspection 20 nodes at a time https://review.openstack.org/439568 | 11:42 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: NO MERGEY - testing low concurrency https://review.openstack.org/449215 | 11:42 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: Send a zaqar message on each introspection completion https://review.openstack.org/462078 | 11:42 |
| *** rbowen has quit IRC | 11:42 | |
| *** pradk has joined #tripleo | 11:43 | |
| *** tosky has joined #tripleo | 11:45 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient master: Allow users to specify the Introspection batch size https://review.openstack.org/451228 | 11:45 |
| *** ckyriakidou has quit IRC | 11:46 | |
| *** athomas has joined #tripleo | 11:49 | |
| *** ffiore has quit IRC | 11:49 | |
| *** morazi has joined #tripleo | 11:50 | |
| sshnaidm | EmilienM, is gate-tripleo-ci-centos-7-undercloud-containers voting job now? | 11:50 |
| EmilienM | sshnaidm: seems like | 11:51 |
| *** bfournie has quit IRC | 11:51 | |
| sshnaidm | EmilienM, do you know who can troubleshoot it? Or explain what is going there | 11:52 |
| EmilienM | sshnaidm: mandre or dprince I guess at least | 11:52 |
| sshnaidm | mandre, ping | 11:52 |
| sshnaidm | EmilienM, it's also gating job now | 11:53 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo master: MySQL client: Make CA file configurable https://review.openstack.org/462062 | 11:53 |
| *** pcaruana has quit IRC | 11:53 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Internal TLS: Use specific CA file for mysql-client https://review.openstack.org/462066 | 11:53 |
| EmilienM | sshnaidm: I'm in a mtg right now and for the next hours, I can't help | 11:55 |
| *** jayg|g0n3 is now known as jayg | 11:55 | |
| *** ckyriakidou has joined #tripleo | 11:56 | |
| mandre | sshnaidm: hey | 11:57 |
| sshnaidm | mandre, do you know id undercloud-containers supposed to be gating job? | 11:57 |
| *** pcaruana has joined #tripleo | 11:57 | |
| sshnaidm | s/id/if/ | 11:57 |
| jkilpatr | d0ugal, cloud working ok for you? | 11:58 |
| mandre | sshnaidm: hmm, I don't think so, this is news to me | 11:59 |
| mandre | sshnaidm: is it failing? | 11:59 |
| sshnaidm | mandre, yeah | 11:59 |
| *** fzdarsky has joined #tripleo | 11:59 | |
| sshnaidm | mandre, https://github.com/openstack-infra/project-config/commit/22b3e830d2f488a26d75d85d13ec2e6644f35fdf | 12:00 |
| *** fzdarsky has quit IRC | 12:00 | |
| *** links has quit IRC | 12:00 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: NO MERGEY - testing low concurrency https://review.openstack.org/449215 | 12:01 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: Send a zaqar message on each introspection completion https://review.openstack.org/462078 | 12:01 |
| mandre | sshnaidm: oh man | 12:01 |
| d0ugal | jkilpatr: yup, seems fine - I have not tested it much yet, just about to - I realised i was really overcomplicating things and put up a much simpler patch. | 12:01 |
| sshnaidm | mandre, seems like you added it to gate | 12:01 |
| mandre | sshnaidm: I must have blindly applied review comments or something | 12:01 |
| sshnaidm | mandre, it should be in check pipeline, right? | 12:02 |
| d0ugal | jkilpatr: so I am much more confident that it will work now | 12:02 |
| sshnaidm | mandre, should it vote at all? | 12:02 |
| mandre | sshnaidm: let's try to fix the gate first | 12:02 |
| mandre | sshnaidm: yeah, we want to eventually make it voting | 12:03 |
| sshnaidm | mandre, is it stable enough for voting? because right now it fails on gates.. | 12:04 |
| shardy | I thought we discussed making it voting last week, since it'd been working quite well and we keep breaking container deployment | 12:04 |
| shardy | sshnaidm: do you have an example of a failing job? | 12:05 |
| shardy | cistatus looks mostly green | 12:05 |
| sshnaidm | https://review.openstack.org/#/c/460558/ | 12:06 |
| sshnaidm | mandre, http://logs.openstack.org/58/460558/2/gate/gate-tripleo-ci-centos-7-undercloud-containers/dad0e38/console.html | 12:07 |
| mandre | it failed to get a token from keystone, http://logs.openstack.org/58/460558/2/gate/gate-tripleo-ci-centos-7-undercloud-containers/dad0e38/logs/var/log/undercloud_install.txt.gz#_2017-05-03_11_44_22_000 | 12:07 |
| jistr | hm failed pingtest | 12:07 |
| jistr | http://logs.openstack.org/58/460558/2/gate/gate-tripleo-ci-centos-7-undercloud-containers/dad0e38/logs/var/log/undercloud_install.txt.gz#_2017-05-03_11_44_20_000 | 12:08 |
| *** rlandy has joined #tripleo | 12:08 | |
| jaosorior | adarazs: could you check this out https://review.openstack.org/#/c/461804/1 ? | 12:08 |
| adarazs | jaosorior: looking | 12:08 |
| jistr | shardy, mandre, sshnaidm: maybe we could keep it a voting check job, but remove it from gate? | 12:09 |
| mandre | sshnaidm: did we stop collecting the containers logs for undercloud-containers job? | 12:09 |
| *** dprince has joined #tripleo | 12:09 | |
| jistr | mandre: it failed the AllNodesValidation deployment, so maybe it didn't start deploying any containers yet | 12:10 |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 12:10 |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 12:10 |
| adarazs | jaosorior: submitted. | 12:11 |
| shardy | jistr: yeah maybe the check queue is enough, but I really hope we can avoid going back to non voting or experimental | 12:11 |
| jaosorior | adarazs: thanks dude | 12:11 |
| mandre | jistr, shardy: FYI, this is what I did first in https://review.openstack.org/#/c/446940/ | 12:12 |
| sshnaidm | mandre, jistr shardy I think the consensus was to make job firstly non-voting (after it was experimental) and only then after a stable period to make it voting | 12:12 |
| mandre | I didn't realize it was voting | 12:12 |
| EmilienM | mandre: FYI all jobs in "gate" are voting | 12:13 |
| sshnaidm | we skipped the non-voting step here | 12:13 |
| EmilienM | https://review.openstack.org/#/c/446940/2 | 12:14 |
| EmilienM | mandre: do we move back to non voting now? | 12:14 |
| shardy | Ok lets do that | 12:15 |
| * shardy sighs | 12:15 | |
| EmilienM | shardy: do we know why it fails? | 12:15 |
| EmilienM | has someone looked at it? | 12:15 |
| shardy | EmilienM: jistr just linked the reason, AllNodesValidationDeployment is failing because it can't ping an IP | 12:16 |
| *** liverpooler has joined #tripleo | 12:16 | |
| shardy | I don't think we yet know the underlying cause | 12:16 |
| *** enrraja has quit IRC | 12:16 | |
| mandre | EmilienM, sshnaidm: so voting means it has to run in check & gate? or can we have it voting and check only? | 12:17 |
| EmilienM | shardy: we could also exit 0 in tripleo-ci for a very short workaround? | 12:17 |
| EmilienM | mandre: voting means check + gate. If you want it non voting, you need to have it in check only with -nv suffix. | 12:18 |
| sshnaidm | mandre, EmilienM shardy I think it should be like this: https://review.openstack.org/#/c/462112/ | 12:18 |
| EmilienM | mandre: next time, just add as reviewer or ask me here to look before we merge it | 12:18 |
| sshnaidm | mandre, yeah, it's better to add EmilienM and somebody from tripleo-ci cores to such reviews in project-config | 12:18 |
| shardy | http://logs.openstack.org/58/460558/2/gate/gate-tripleo-ci-centos-7-undercloud-containers/dad0e38/console.html#_2017-05-03_10_35_24_582201 | 12:18 |
| shardy | That shows the failed ping IP is reachable | 12:19 |
| mandre | EmilienM: you were added as reviewer :-/ | 12:19 |
| EmilienM | mandre: I missed it in my huge backlog, probably. | 12:19 |
| EmilienM | shardy: we should run this job in THT gate | 12:19 |
| *** abishop has quit IRC | 12:20 | |
| EmilienM | shardy: i'm pretty sure it will help to not break it next time | 12:20 |
| mandre | EmilienM: yeah, of course, I understand, I'm not trying to put it on you :) | 12:20 |
| EmilienM | shardy: even puppet-tripleo | 12:20 |
| *** dsariel_ has joined #tripleo | 12:21 | |
| *** cylopez has joined #tripleo | 12:21 | |
| EmilienM | shardy, mandre, sshnaidm: do we want to make it non voting until we fix it ? or just exit 0 in tripleo-ci ? | 12:21 |
| shardy | EmilienM: yeah, we need it in t-h-t and puppet-tripleo check queue I think | 12:21 |
| *** dsariel has quit IRC | 12:21 | |
| shardy | EmilienM: I'd prefer non voting as we can then see more clearly when this is failing | 12:22 |
| sshnaidm | EmilienM, I'd like to have it non-voting | 12:22 |
| shardy | e.g if this is a spurious or permanent problem | 12:22 |
| *** rhallisey has quit IRC | 12:22 | |
| EmilienM | ok | 12:22 |
| mandre | agreed | 12:22 |
| *** rhallisey has joined #tripleo | 12:22 | |
| EmilienM | ok so 1. let's make it non voting and 2. run it in puppet-tripleo and THT | 12:22 |
| EmilienM | and 3. make it voting again | 12:22 |
| sshnaidm | so please vote https://review.openstack.org/#/c/462112/ | 12:22 |
| *** bfournie has joined #tripleo | 12:25 | |
| *** nyechiel has joined #tripleo | 12:26 | |
| weshay | arxcruz, sshnaidm hey | 12:28 |
| sshnaidm | weshay, hi | 12:28 |
| *** rbowen has joined #tripleo | 12:29 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates master: Use httpd in Zaqar docker service https://review.openstack.org/462133 | 12:31 |
| *** psahoo has quit IRC | 12:31 | |
| adarazs | weshay: hi there. | 12:32 |
| weshay | hola | 12:32 |
| adarazs | weshay: I checked on the image building/uploading issue, updated and rebooted the image server, but I think it was a transient network issue. | 12:32 |
| weshay | adarazs, it was full | 12:32 |
| adarazs | weshay: looking at the master promote. | 12:32 |
| weshay | I cleaned it out yesterday | 12:32 |
| hjensas | How can I run a check-requirements test locally? | 12:33 |
| adarazs | oh okay. I didn't have time to check it out yesterday before leaving. | 12:33 |
| *** ansmith has joined #tripleo | 12:33 | |
| weshay | adarazs, k.. what ever you can do bring us out of the red here https://dashboards.rdoproject.org/rdo-dev w/ master being the priority | 12:33 |
| weshay | sshnaidm, arxcruz there was some question re: tempest? | 12:37 |
| sshnaidm | weshay, yeah, about non building images in periodic tempest job - I'm not sure it's good idea to skip building there | 12:37 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Parameterize OVB stack flavors https://review.openstack.org/445163 | 12:37 |
| arxcruz | weshay: hey :) | 12:38 |
| weshay | sshnaidm, ok.. give me more detail | 12:38 |
| sshnaidm | weshay, we always will test for previous image, instead of new one. And if we promote image this night, it means that we tested something old and our results are not for the current state | 12:39 |
| weshay | note: the tempest periodic job should be working w/ known good builds | 12:39 |
| adarazs | weshay: so about the master promote, I see a bunch of timeouts, I did not analyze it in details, but it seems that everything is just taking longer *before* the overcloud deployment, and so the overcloud deployment times out, but it's below the time it took in our last promote. | 12:39 |
| sshnaidm | weshay, it's about packages, I'm talking about image only | 12:39 |
| weshay | sshnaidm, correct | 12:39 |
| weshay | sshnaidm, the tempest job should be testing one promotion behind | 12:40 |
| weshay | the other periodic jobs | 12:40 |
| sshnaidm | weshay, and it will confuse people I think, because all other periodic do different.. | 12:40 |
| *** jpena is now known as jpena|lunch | 12:40 | |
| *** jprovazn has quit IRC | 12:40 | |
| *** links has joined #tripleo | 12:41 | |
| weshay | sshnaidm, the tempest job will be testing the same hash as the check gates | 12:41 |
| sshnaidm | weshay, why not to test the current image? | 12:41 |
| weshay | when you say current image.. let's make sure we're talking about the same thing | 12:41 |
| *** lucas-hungry is now known as lucasagomes | 12:42 | |
| weshay | imho I would like the tempest job to use the current-tripleo pin and the same image used in the check jobs | 12:42 |
| sshnaidm | weshay, I meant image that we built right now in this job with packages from this current hash | 12:42 |
| weshay | sshnaidm, is there any need to rebuild the promoted image in that job that I'm missing? | 12:42 |
| *** dprince has quit IRC | 12:43 | |
| pabelanger | EmilienM: did we never make modulesync work with propose-bot? I thought I wrote up a patch for that | 12:43 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: WIP: Add OVB support to devmode.sh https://review.openstack.org/452249 | 12:43 |
| jaosorior | jistr, marios: Could you check this out if you have some time? https://review.openstack.org/#/c/460024/ | 12:44 |
| sshnaidm | weshay, no, if we need to build it with same image of check jobs, than it's right, we need to skip. | 12:44 |
| *** dmacpher-afk has quit IRC | 12:44 | |
| weshay | sshnaidm, ya.. the purpose of the tempest job is to take a known good build and run tempest | 12:44 |
| weshay | it's not take a hash w/ unknown quality and run tempest | 12:45 |
| weshay | it's not to take | 12:45 |
| sshnaidm | weshay, ok, now it's clear | 12:45 |
| pabelanger | EmilienM: https://review.openstack.org/#/c/211744/ | 12:45 |
| weshay | sshnaidm, arxcruz thanks guys | 12:45 |
| pabelanger | EmilienM: guess we could bring that back online again | 12:45 |
| *** ffiore has joined #tripleo | 12:46 | |
| arxcruz | sshnaidm: adarazs so, can you guys review now? :D | 12:46 |
| marios | ack jaosorior | 12:49 |
| mandre | EmilienM, sshnaidm, shardy: the undercloud-container job was already in puppet-tripleo, I'm adding it to t-h-t, see https://review.openstack.org/#/c/462141/ | 12:50 |
| sshnaidm | mandre, do we have anywhere to read what this job is doing? just curious | 12:51 |
| openstackgerrit | Christian Schwede proposed openstack/tripleo-heat-templates master: Add missing Swift services for containerized deployments https://review.openstack.org/445891 | 12:52 |
| openstackgerrit | Harald Jensås proposed openstack/instack-undercloud master: Tripleo routed networks ironic inspector, and Undercloud https://review.openstack.org/437544 | 12:53 |
| *** jaganathan has joined #tripleo | 12:54 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Rename misleading undercloud/overcloud hieradata overrides https://review.openstack.org/458501 | 12:55 |
| *** karthiks has joined #tripleo | 12:55 | |
| *** jmelvin has joined #tripleo | 12:56 | |
| *** nmathew has joined #tripleo | 12:56 | |
| *** skramaja has joined #tripleo | 12:56 | |
| mandre | sshnaidm: it's mostly happening in the undercloud_containers function, https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/tripleo.sh#L535 | 12:58 |
| adarazs | weshay: so for the master promote, let's try to increase the timeout from 240min to 300min. | 12:58 |
| adarazs | weshay: and see if that helps. I think it will. | 12:58 |
| *** bkopilov_ has joined #tripleo | 13:00 | |
| *** bkopilov has joined #tripleo | 13:00 | |
| adarazs | weshay: I don't see anything else going wrong. | 13:01 |
| adarazs | weshay, trown|outtypewww: https://review.rdoproject.org/r/6527 | 13:03 |
| EmilienM | mandre: ok | 13:03 |
| *** zoli is now known as zoli|afk | 13:03 | |
| EmilienM | slagle: really cool for the Deep Dive, thanks!!! | 13:06 |
| *** nmathew has quit IRC | 13:06 | |
| d0ugal | jkilpatr: how is your testing going? did you manage to get past that error? | 13:07 |
| jkilpatr | d0ugal, I got a deployment with the existing tripleo tools with 10 nodes, was going to try 15 until jenkins rebooted this morning | 13:08 |
| jkilpatr | d0ugal, 20+ doesn't get past introspection so that's where you come in | 13:08 |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 13:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 13:10 |
| *** tosky has quit IRC | 13:11 | |
| *** vkhanna has joined #tripleo | 13:13 | |
| *** vkhanna has left #tripleo | 13:13 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE: Testing LDAP integration https://review.openstack.org/461865 | 13:19 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: Deploy FreeIPA-backed LDAP domain for keystone in fakeha-caserver job https://review.openstack.org/461864 | 13:19 |
| *** tiswanso has joined #tripleo | 13:20 | |
| *** tzumainn has joined #tripleo | 13:20 | |
| *** jprovazn has joined #tripleo | 13:20 | |
| *** trown|outtypewww is now known as trown | 13:21 | |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates master: Add role specific information to the service template https://review.openstack.org/437956 | 13:22 |
| *** sshnaidm is now known as sshnaidm|afk | 13:22 | |
| *** rbowen has quit IRC | 13:23 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE: Testing LDAP integration https://review.openstack.org/461865 | 13:23 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: Deploy FreeIPA-backed LDAP domain for keystone in fakeha-caserver job https://review.openstack.org/461864 | 13:23 |
| *** rbowen has joined #tripleo | 13:25 | |
| *** rbowen has quit IRC | 13:25 | |
| *** rbowen has joined #tripleo | 13:25 | |
| *** chlong has quit IRC | 13:28 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE: Testing LDAP integration https://review.openstack.org/461865 | 13:28 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: Deploy FreeIPA-backed LDAP domain for keystone in fakeha-caserver job https://review.openstack.org/461864 | 13:28 |
| gfidente | dtantsur did you see https://bugs.launchpad.net/ironic/+bug/1679726 ? | 13:30 |
| openstack | Launchpad bug 1679726 in Ironic Inspector "discovery does not include /dev/disk/by-path device name" [Wishlist,Triaged] | 13:30 |
| gfidente | have opinions? | 13:30 |
| dtantsur | gfidente, this is an RFE, it can be implemented :) wrong component though, should be ironic-python-agent | 13:31 |
| gfidente | dtantsur so I think Ben is mainly looking for someone to adopt his patch and go through the formal gerrit process | 13:31 |
| *** arxcruz has quit IRC | 13:33 | |
| dtantsur | gfidente, moving it to our team is a great way to delay it a few years.. I'd recommend him pushing it himself. | 13:33 |
| dtantsur | I can certainly help with reviews | 13:34 |
| gfidente | ahahaha | 13:34 |
| *** eck`gone is now known as eck` | 13:34 | |
| *** arxcruz has joined #tripleo | 13:34 | |
| *** dtantsur is now known as dtantsur|brb | 13:35 | |
| *** hjensas has quit IRC | 13:37 | |
| *** jaosorior is now known as jaosorior_away | 13:41 | |
| *** abishop has joined #tripleo | 13:41 | |
| *** jpena|lunch is now known as jpena | 13:41 | |
| *** tosky has joined #tripleo | 13:42 | |
| *** fpan has quit IRC | 13:42 | |
| d0ugal | jkilpatr: I'm hoping that it should pass with more than 20 nodes and my latest patch set | 13:43 |
| jkilpatr | d0ugal, some reasonable margin for failure tolerence + batching should work on arbitrary numbers of nodes | 13:43 |
| d0ugal | jkilpatr: yup | 13:45 |
| jkilpatr | d0ugal, then we get into doing the same thing for the overcloud, but that's a much more complicated problem, one that we need to pull in a lot of stakeholders to get done. | 13:45 |
| jkilpatr | baby steps. | 13:45 |
| gfidente | jkilpatr for the overcloud deployment there is heat doing the batching already | 13:47 |
| jkilpatr | gfidente, is that a patchset or is it merged? | 13:48 |
| *** karimb has joined #tripleo | 13:48 | |
| *** nyechiel has quit IRC | 13:49 | |
| gfidente | jkilpatr it's merged, doing it from the workflow would be wrong | 13:49 |
| *** dprince has joined #tripleo | 13:49 | |
| jkilpatr | gfidente, cool do you know where I can find docs on how it works? | 13:50 |
| gfidente | jkilpatr https://review.openstack.org/#/c/446927/ | 13:50 |
| gfidente | jkilpatr it's a generic feature heat has for resourcegroups | 13:50 |
| jkilpatr | gfidente, cool so I'll find this in Ocata, I look forward to testing it. Scale (300ish nodes) Ocata testing should happen eventually. | 13:51 |
| gfidente | jkilpatr I think this hasn't been ported to ocata | 13:52 |
| jkilpatr | oh this is pike/master? | 13:52 |
| gfidente | it's pike yes | 13:52 |
| jkilpatr | no wonder we haven't seen it yet. | 13:52 |
| *** Goneri has joined #tripleo | 13:52 | |
| *** janki has quit IRC | 13:53 | |
| gfidente | jkilpatr sorry I was wrong, it is in the stable/ocata branch too | 13:53 |
| jkilpatr | nice, ^ sai | 13:54 |
| gfidente | aaaaah no it isn't , I was in the wrong branch | 13:54 |
| gfidente | :) | 13:54 |
| jkilpatr | either way we're going to run into it. | 13:54 |
| jkilpatr | gfidente, what are your thoughts on failure tolerence? in a 100 node cloud failing a deploy because of one comptue doesn't make much sense. | 13:54 |
| gfidente | jkilpatr we were having a similar conversation with ceph folks about one of the disks possibly failing | 13:55 |
| gfidente | our conclusion was that we should fail in such a scenario | 13:56 |
| gfidente | and let the operator know why, eventually blacklisting/removing the disk from the list | 13:56 |
| jkilpatr | an entire ceph cluster of arbitrary size because of one disk? | 13:57 |
| gfidente | but not assume that running with one less disk is sane just because the cloud 'overall' can work with it | 13:57 |
| gfidente | yes the distinguo was indeed what to do on the initial deployment | 13:57 |
| *** atoth has joined #tripleo | 13:58 | |
| *** links has quit IRC | 13:58 | |
| *** Goneri has quit IRC | 13:58 | |
| gfidente | the idea was that if a disk fails during operations ceph can deal with it, but deploying with one less disk than expected because one disk failed is against the intentions | 13:59 |
| jkilpatr | The real issue is that these deploys can take a very long time and restarting each time can make debugging painful. From an operator perspective they would much rather a "x has failed, fix y then resume where you left off" | 13:59 |
| jkilpatr | rather than a multi-hour deploy failing for some trivial reason 2 hours in and you have to repeat until there are no more failures. | 14:00 |
| gfidente | yeah we discussed this same argument | 14:00 |
| gfidente | but then again, I am not sure how long it takes the overcloud deployment with 300nodes | 14:00 |
| jkilpatr | like a day | 14:00 |
| jkilpatr | the workflow is -> tiny initial deployment -> scale by 30, debug problems with those 30 nodes -> repeat until done | 14:01 |
| gfidente | yeah but if a node is broken, I think you should take it out from the ironic node list and update | 14:01 |
| jkilpatr | if you do the batching in heat itself that's great, but there's no chance to debug | 14:01 |
| jkilpatr | gfidente, put it in manageable? yes but you usually go and try and fix those nodes and put them back into the pool to use. | 14:02 |
| gfidente | yeah that sounds like valid to me | 14:03 |
| gfidente | on the initial deployment | 14:03 |
| gfidente | should it fail during operations, that is a different problem | 14:03 |
| gfidente | as it is with ceph, nova will deal with it | 14:04 |
| jkilpatr | In general I agree with 'success is defined as what people ask to deploy' but I think we should provide more chances to fix failures rather than just failing out | 14:04 |
| jkilpatr | it's not costly to stop the workflow and wait for the user to go and fix stuff before trying the next batch | 14:04 |
| gfidente | jkilpatr so you would finish the deployment and give back a warning saying one or multiple nodes failed? | 14:06 |
| *** eck` is now known as eck`gone | 14:06 | |
| gfidente | pausing the deployment is what currently happens | 14:06 |
| gfidente | batched or not | 14:07 |
| jkilpatr | gfidente, so by pausing the deployment you mean putting the stack in a create/update failed state but leaving nodes already deployed on and running? | 14:08 |
| sai | jkilpatr: so this batching is for creating the resources if i understand correctly, however we will only see a problem if max_concurrent_builds is set to greater>30 due to the tftp limitation u mentioned isnt it? | 14:08 |
| *** cylopez has quit IRC | 14:09 | |
| jkilpatr | sai, that's fixed now. At this point it's just a best practices thing. I know you've done a 100 node deploy in one shot. | 14:09 |
| gfidente | jkilpatr yes | 14:09 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 14:10 |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 14:10 |
| jkilpatr | gfidente, I've found that to be hit or miss depending on when and what the failure is. But I could see that doing what we need with more helpful error messages. | 14:10 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Parameterize OVB stack flavors https://review.openstack.org/445163 | 14:10 |
| sai | gfidente: jkilpatr so with this patch stack will be created by scaling up in multiples fo 30? | 14:11 |
| gfidente | sai yes but it's not included in ocata | 14:11 |
| *** pradk has joined #tripleo | 14:12 | |
| sai | cool, ok this has nothing to max_concurrent_builds (which is the tftp limitation mentioned in the commit msg) | 14:12 |
| *** rbowen has quit IRC | 14:13 | |
| *** hjensas has joined #tripleo | 14:13 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Move OVB settings to tripleo-quickstart-extras https://review.openstack.org/448805 | 14:14 |
| *** zoli|afk is now known as zoli|wfh | 14:14 | |
| *** atoth has quit IRC | 14:14 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Move OVB settings to tripleo-quickstart-extras https://review.openstack.org/448805 | 14:16 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Move OVB settings to tripleo-quickstart-extras https://review.openstack.org/448805 | 14:17 |
| *** eck`gone is now known as eck` | 14:17 | |
| *** funzo has joined #tripleo | 14:18 | |
| *** openstackgerrit has quit IRC | 14:18 | |
| gfidente | sai right, it batches the heat resources but does not change anything in the undercloud tftp | 14:21 |
| *** funzo has quit IRC | 14:23 | |
| *** openstackgerrit has joined #tripleo | 14:23 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Parameterize OVB stack flavors https://review.openstack.org/445163 | 14:23 |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates master: Modify vhost user socket directory's default value in environment file https://review.openstack.org/462171 | 14:24 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: WIP: Add OVB support to devmode.sh https://review.openstack.org/452249 | 14:25 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates master: Use httpd in Zaqar docker service https://review.openstack.org/462133 | 14:25 |
| *** ffiore has quit IRC | 14:27 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-quickstart-extras master: Don't overwrite tht_dir for upgrades to master https://review.openstack.org/462172 | 14:28 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-quickstart master: TripleO CI mixed release master UC / ocata OC https://review.openstack.org/459789 | 14:28 |
| openstackgerrit | Jiri Stransky proposed openstack-infra/tripleo-ci master: Use mixed release for container upgrade https://review.openstack.org/460061 | 14:30 |
| *** rbowen has joined #tripleo | 14:30 | |
| *** adarazs is now known as adarazs_brb | 14:31 | |
| openstackgerrit | Jiri Stransky proposed openstack-infra/tripleo-ci master: Use mixed release for container upgrade https://review.openstack.org/460061 | 14:31 |
| EmilienM | we just got a TripleO CI promotion | 14:32 |
| jistr | whohoo \o/ | 14:33 |
| *** mdnadeem has quit IRC | 14:33 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Parameterize OVB stack flavors https://review.openstack.org/445163 | 14:35 |
| *** ffiore has joined #tripleo | 14:38 | |
| *** jlinkes has quit IRC | 14:38 | |
| *** prateek has quit IRC | 14:39 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: WIP: Add OVB support to devmode.sh https://review.openstack.org/452249 | 14:40 |
| *** gfidente has quit IRC | 14:41 | |
| shardy | Does anyone have any experience of sr-iov combined with cisco nexus ucsm? | 14:41 |
| shardy | I've posted a couple of patches trying to fix the ucsm integration, but now have questions related to if/how it's supposed to play with sr-iov | 14:42 |
| *** gfidente has joined #tripleo | 14:42 | |
| *** udesale has joined #tripleo | 14:43 | |
| *** jlinkes has joined #tripleo | 14:45 | |
| ccamacho | rnoriega you might share your sr-iov kungfu here | 14:46 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: Run introspection 20 nodes at a time https://review.openstack.org/439568 | 14:46 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: NO MERGEY - testing low concurrency https://review.openstack.org/449215 | 14:46 |
| ccamacho | shardy rnoriega, related to these patches? https://review.openstack.org/#/q/topic:bug/1687597 | 14:47 |
| *** jmelvin has quit IRC | 14:47 | |
| rnoriega | shardy, ccamacho, hi! only sriov kungfu...no cisco | 14:47 |
| shardy | ccamacho: yeah, specifically https://review.openstack.org/#/c/462035/ | 14:48 |
| shardy | rnoriega: so the question is around the ml2 configuration on the compute nodes | 14:48 |
| shardy | rnoriega: historically that was only done on the controller for cisco ucsm | 14:48 |
| *** jlinkes_ has joined #tripleo | 14:48 | |
| shardy | but it seems we require compute configuration for sr-iov | 14:48 |
| *** jlinkes has quit IRC | 14:49 | |
| pliu | beagles, ccamacho, could you please help to review the patch https://review.openstack.org/#/c/451175/ | 14:49 |
| shardy | I suspect nobody has tried this combination given the current bugfixes needed for ucsm | 14:49 |
| *** jlinkes_ has quit IRC | 14:49 | |
| beagles | pliu: ack | 14:49 |
| *** enrraja has joined #tripleo | 14:49 | |
| shardy | I've got no way to test it, so really just trying to figure out the best path forward | 14:49 |
| *** fragatina has joined #tripleo | 14:49 | |
| shardy | my current patches reinstate the pre-newton integration, but that may no longer be enough with sr-iov things in the mix | 14:50 |
| beagles | pliu: sorry about that I meant to chat with ccamacho about this last week, but kept getting sidetracked | 14:50 |
| pliu | beagles, it's ok. I 've updated the patch as suggested. | 14:51 |
| beagles | pliu: k | 14:51 |
| ccamacho | hey pliu beagles Ill check the changes right now I think there are some missing, let me check it | 14:51 |
| rnoriega | shardy, sriov uses an agent in the compute nodes and the sriovnicswitch mechanism driver | 14:51 |
| rnoriega | shardy, but I've never worked with the cisco nexus stuff. | 14:52 |
| shardy | rnoriega: Ok, and the ucsm ml2 plugin requires a different mechanism driver, which fails on compute because we don't write the ucsm hieradata on that role currently | 14:52 |
| pliu | ccamacho, basically, i just merged the 2 env files. please let me know if there is anything wrong. | 14:52 |
| rnoriega | shardy, alright. | 14:52 |
| *** jlinkes has joined #tripleo | 14:53 | |
| shardy | rnoriega: Ok, sounds like we probably need guidance from cisco folks on this, or someone with more direct ucsm access | 14:53 |
| *** gvrangan has quit IRC | 14:53 | |
| shardy | rnoriega: thanks for the info re sriov, the error reported makes more sense now | 14:53 |
| rnoriega | shardy, yep. let me see if someone in my team has experience with ucsm | 14:53 |
| jrist | florianf: if you have a chance can you look at and +A this? https://review.openstack.org/#/c/451431/5 | 14:53 |
| rnoriega | shardy, ping me for any sriov related config | 14:53 |
| rnoriega | shardy, glad to help | 14:54 |
| *** gvrangan has joined #tripleo | 14:54 | |
| shardy | rnoriega: excellent thanks - I may put those trying to make this work in contact if that's OK | 14:54 |
| beagles | rnoriega: shardy: bug ref? | 14:54 |
| florianf | jrist: let me just test it... | 14:54 |
| beagles | rnoriega: may be of no help whatsoever, but I'm curious now :) | 14:54 |
| shardy | beagles: https://bugs.launchpad.net/tripleo/+bug/1687597 and https://bugzilla.redhat.com/show_bug.cgi?id=1445030 | 14:55 |
| openstack | Launchpad bug 1687597 in tripleo "AllNodesExtraConfig should run before AllNodesDeploySteps" [High,Fix released] - Assigned to Emilien Macchi (emilienm) | 14:55 |
| openstack | bugzilla.redhat.com bug 1445030 in openstack-tripleo "RH OSP10: With Cisco ML2 plugin for ucsm, Overcloud install failing due to config not being generated in time." [High,Assigned] - Assigned to shardy | 14:55 |
| shardy | beagles: but that only covers the ucsm integration breakage | 14:55 |
| jrist | d0ugal: I like the middle one https://review.openstack.org/#/c/439568/15/workbooks/baremetal.yaml | 14:55 |
| jrist | simpler | 14:55 |
| shardy | beagles: I've just discovered those involved are also trying to get ucsm and sr-iov to co-exist | 14:55 |
| shardy | so there may be another bug incoming I guess | 14:55 |
| beagles | shardy: ah | 14:56 |
| *** dtantsur|brb is now known as dtantsur | 14:56 | |
| d0ugal | jrist: middle one? | 14:56 |
| ccamacho | supaahh | 14:57 |
| *** cylopez has joined #tripleo | 14:57 | |
| *** sid1 has joined #tripleo | 14:58 | |
| sid1 | hi | 14:58 |
| *** yprokule has quit IRC | 14:58 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: WIP: Add OVB support to devmode.sh https://review.openstack.org/452249 | 14:58 |
| *** cylopez has quit IRC | 14:59 | |
| shardy | sid1: Hi, I've been chatting with rnoriega and beagles about your requirement to combine ucsm and sr-iov | 15:00 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-quickstart master: TripleO CI mixed release master UC / ocata OC https://review.openstack.org/459789 | 15:00 |
| sid1 | Thanks, any feedback on this is welcome | 15:00 |
| shardy | sid1: it sounds like it's a combination that isn't tested yet, and you are right to say disabling the neutron configuration on the compute node may be a problem | 15:00 |
| *** garyk has joined #tripleo | 15:01 | |
| *** Goneri has joined #tripleo | 15:01 | |
| shardy | sid1: I'd suggest we go ahead and prove the fixes for ucsm integration, and raise a second bug to explore the issues of combining it with sr-iov | 15:01 |
| sid1 | @shardy yes, that is what i was wondering | 15:01 |
| shardy | it sounds like there's work required to make those features play nicely together, and I don't think there's a quick fix unless anyone else sees one | 15:02 |
| beagles | sid1: the ucsm support is implemented as an ml2 plugin? | 15:02 |
| sid1 | @shardy I agree with you | 15:02 |
| shardy | e.g we could deploy the ucsm hiera on Compute roles, but I'm not sure if that really solves the problem | 15:02 |
| shardy | beagles: yup, it adds a driver to mechanism_drivers and puppet configures things on the Controller | 15:02 |
| sid1 | @beagles, we have cisco_usm as a NeutronMechanismDriver | 15:02 |
| shardy | beagles: it's all wired up in a pre-composable-roles fashion because it's been around for a while | 15:03 |
| shardy | and because it does some weird stuff like collecting all the mac addresses via AllNodesExtraConfig | 15:03 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/extraconfig/all_nodes/neutron-ml2-cisco-nexus-ucsm.yaml | 15:04 |
| shardy | beagles: ^^ | 15:04 |
| rnoriega | shardy, not sure if this is related but I tested SRIOV and OpenDaylight environment | 15:05 |
| rnoriega | shardy, if you see the env file for opendayglight: https://github.com/openstack/tripleo-heat-templates/blob/master/environments/neutron-opendaylight.yaml | 15:05 |
| rnoriega | shardy, it disables the neutron core plugin on the compute... | 15:06 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/newton: Ensure we configure ssl.conf https://review.openstack.org/460560 | 15:06 |
| rnoriega | shardy, but with SRIOV that doesn't work... so you have to "mix" it with OS::TripleO::Services::ComputeNeutronCorePlugin: ../puppet/services/neutron-plugin-ml2.yaml | 15:06 |
| sid1 | Did we get help/support from Cisco side on this ? I don't think so.. | 15:06 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/newton: Touch /etc/httpd/conf.d/ssl.conf https://review.openstack.org/460555 | 15:06 |
| *** agurenko has quit IRC | 15:07 | |
| openstackgerrit | Merged openstack/tripleo-common master: Fix the reporting from the pre-deployment checks workflows https://review.openstack.org/461026 | 15:08 |
| openstackgerrit | Merged openstack/tripleo-common master: Move the gathering of flavors into the CollectFlavors action https://review.openstack.org/459704 | 15:08 |
| openstackgerrit | Merged openstack/tripleo-common master: Ignore '_comment' as a invalid field in instackenv.json https://review.openstack.org/458887 | 15:09 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Add documentation for hieradata overrides for the undercloud https://review.openstack.org/461804 | 15:09 |
| *** athomas has quit IRC | 15:09 | |
| *** Goneri has quit IRC | 15:10 | |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1687706 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 15:10 |
| openstack | Launchpad bug 1687706 in tripleo "__init__() got an unexpected keyword argument 'url'" [Critical,In progress] - Assigned to Emilien Macchi (emilienm) | 15:10 |
| shardy | rnoriega: yeah I think that's what sid1 is seeing, but that won't work unless we modify the ucsm all-nodes template to write hieradata to the Compute | 15:11 |
| shardy | we could do that | 15:11 |
| *** Goneri has joined #tripleo | 15:11 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: Deploy FreeIPA-backed LDAP domain for keystone in fakeha-caserver job https://review.openstack.org/461864 | 15:12 |
| *** jmelvin has joined #tripleo | 15:13 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE: Testing LDAP integration https://review.openstack.org/461865 | 15:13 |
| *** jlinkes has quit IRC | 15:14 | |
| *** chlong has joined #tripleo | 15:15 | |
| *** rbowen has quit IRC | 15:21 | |
| *** udesale has quit IRC | 15:22 | |
| *** dsariel_ has quit IRC | 15:22 | |
| *** enrraja has quit IRC | 15:24 | |
| *** rbowen has joined #tripleo | 15:25 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-quickstart master: TripleO CI mixed release master UC / ocata OC https://review.openstack.org/459789 | 15:25 |
| *** dciabrin has quit IRC | 15:26 | |
| *** dciabrin has joined #tripleo | 15:26 | |
| openstackgerrit | Merged openstack/python-tripleoclient master: Explicitly set 'builders' option https://review.openstack.org/457966 | 15:27 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Use Sphinx 1.5 warning-is-error https://review.openstack.org/457981 | 15:27 |
| *** gvrangan has quit IRC | 15:27 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Internal TLS: Use specific CA file for haproxy https://review.openstack.org/460024 | 15:28 |
| *** sid1 has quit IRC | 15:28 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Expose metric delay processing metric https://review.openstack.org/458959 | 15:29 |
| *** trown is now known as trown|brb | 15:31 | |
| *** sou|manos has joined #tripleo | 15:31 | |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Consolidate writing json configs with HostPrepTasks https://review.openstack.org/454816 | 15:34 |
| *** gvrangan has joined #tripleo | 15:34 | |
| mandre | dprince: the kolla zaqar change landed, can you take care of building new container images since you did it last time or do you want me to do it? | 15:34 |
| *** trown|brb is now known as trown | 15:35 | |
| *** rbowen has quit IRC | 15:35 | |
| *** dsariel_ has joined #tripleo | 15:38 | |
| *** Goneri has quit IRC | 15:40 | |
| *** ykarel has quit IRC | 15:41 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Parameterize OVB stack flavors https://review.openstack.org/445163 | 15:41 |
| *** Goneri has joined #tripleo | 15:41 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Move OVB settings to tripleo-quickstart-extras https://review.openstack.org/448805 | 15:42 |
| mandre | shardy: did you just squash all of your commits on purpose in https://review.openstack.org/#/c/454816/15? | 15:43 |
| *** ebarrera has quit IRC | 15:44 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: WIP: Add OVB support to devmode.sh https://review.openstack.org/452249 | 15:44 |
| dprince | mandre: I've got it built already | 15:44 |
| dprince | mandre: I will post a new zaqar container shortly | 15:44 |
| EmilienM | owalsh: promotion just happened in tirpleo ci, we'll need to wait for a new image for ovb and recheck https://review.openstack.org/#/c/461115/ | 15:44 |
| mandre | thanks dprince, that was quick | 15:44 |
| *** gbarros has joined #tripleo | 15:46 | |
| *** adarazs_brb is now known as adarazs | 15:50 | |
| *** gbarros has quit IRC | 15:51 | |
| shardy | mandre: yes, wasn't getting many reviews on that series so I'm going to abandon some and focus on a smaller number of slightly larger patches | 15:52 |
| *** milan has quit IRC | 15:53 | |
| shardy | mandre: I'll abandon some of the squashed ones shortly | 15:53 |
| *** dougbtv has quit IRC | 15:54 | |
| owalsh | EmilienM: sweet, thanks! | 15:55 |
| *** aufi has quit IRC | 15:57 | |
| d0ugal | thrash: you around? | 15:58 |
| *** gvrangan_ has joined #tripleo | 16:00 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK https://review.openstack.org/430277 | 16:00 |
| *** cylopez has joined #tripleo | 16:01 | |
| openstackgerrit | Ben Kero proposed openstack/tripleo-quickstart-extras master: Remove repo-setup role, being added to tripleo-quickstart https://review.openstack.org/456288 | 16:02 |
| openstackgerrit | Ben Kero proposed openstack/tripleo-quickstart master: Add repo-setup role from -extras, use it to inject repos into images https://review.openstack.org/456286 | 16:02 |
| *** mcornea has quit IRC | 16:07 | |
| *** ccamacho has quit IRC | 16:07 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Set reasonable TTL defaults for Ceilometer DB https://review.openstack.org/462203 | 16:07 |
| *** dparkes has quit IRC | 16:07 | |
| *** gvrangan has quit IRC | 16:07 | |
| *** lmiccini has quit IRC | 16:09 | |
| *** cylopez has quit IRC | 16:09 | |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 16:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| *** gvrangan_ has quit IRC | 16:11 | |
| *** gvrangan has joined #tripleo | 16:14 | |
| *** Lokesh_Jain_ has joined #tripleo | 16:15 | |
| *** gvrangan_ has joined #tripleo | 16:15 | |
| *** rlandy is now known as rlandy|brb | 16:16 | |
| *** rbowen has joined #tripleo | 16:16 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: Run introspection 20 nodes at a time https://review.openstack.org/439568 | 16:17 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: NO MERGEY - testing low concurrency https://review.openstack.org/449215 | 16:17 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Source ci mirrors file if it exists only https://review.openstack.org/460022 | 16:18 |
| *** nyechiel has joined #tripleo | 16:20 | |
| *** lucasagomes is now known as lucas-afk | 16:21 | |
| *** pcaruana has quit IRC | 16:24 | |
| *** bfournie has quit IRC | 16:27 | |
| *** shardy has quit IRC | 16:28 | |
| d0ugal | jkilpatr: so things seem to be working well for me now | 16:29 |
| d0ugal | jkilpatr: introspecting 3 nodes at a time on your bm cloud | 16:29 |
| *** karimb has quit IRC | 16:30 | |
| jkilpatr | d0ugal, great! You should run into a spurious failure every 10ish inspections on that cloud so you can test that too | 16:30 |
| jkilpatr | well the exact rate varies, but you get spurious failures often enough that you should run into one. | 16:30 |
| d0ugal | jkilpatr: cool, the error handling isn't great yet - I want to start improving that tomorrow. | 16:31 |
| d0ugal | jkilpatr: I have been mimicing failures by aborting introspections, they appear to fail with "Canceled by operator" | 16:31 |
| jkilpatr | you can abort an in progress inspection? TOL | 16:31 |
| jkilpatr | TIL* | 16:31 |
| d0ugal | jkilpatr: openstack baremetal introspection abort UUID | 16:32 |
| bandini | EmilienM: https://review.openstack.org/#/c/462073/ if you have a minute (and sorry again there) | 16:32 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Convert puppet and docker steps to ansible https://review.openstack.org/462211 | 16:32 |
| *** Goneri has quit IRC | 16:36 | |
| *** bfournie has joined #tripleo | 16:37 | |
| *** rlandy|brb is now known as rlandy | 16:37 | |
| *** trown is now known as trown|lunch | 16:38 | |
| *** tesseract has quit IRC | 16:38 | |
| EmilienM | bandini: always for you | 16:38 |
| bandini | :D | 16:38 |
| *** dtantsur is now known as dtantsur|afk | 16:40 | |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-ui master: Add validations toolbar https://review.openstack.org/461128 | 16:43 |
| *** athomas has joined #tripleo | 16:44 | |
| *** gvrangan_ has quit IRC | 16:45 | |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-ui master: Add validations toolbar https://review.openstack.org/461128 | 16:46 |
| *** gvrangan_ has joined #tripleo | 16:47 | |
| *** suuuper has quit IRC | 16:48 | |
| *** dprince has quit IRC | 16:48 | |
| *** ffiore has quit IRC | 16:49 | |
| openstackgerrit | Merged openstack/puppet-pacemaker master: Fix a typo in ipv6 addrlabel https://review.openstack.org/462073 | 16:50 |
| *** dsneddon has quit IRC | 16:51 | |
| *** cl0secall has left #tripleo | 16:56 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-docs master: Add documentation for split-stack https://review.openstack.org/461871 | 17:02 |
| *** derekh has quit IRC | 17:02 | |
| *** dsneddon has joined #tripleo | 17:03 | |
| *** dprince has joined #tripleo | 17:04 | |
| *** athomas has quit IRC | 17:05 | |
| *** jpena is now known as jpena|off | 17:06 | |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 17:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| *** atoth has joined #tripleo | 17:10 | |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-ui master: Replace plan list table with a card view https://review.openstack.org/462230 | 17:12 |
| *** atoth has quit IRC | 17:15 | |
| *** atoth has joined #tripleo | 17:15 | |
| *** gvrangan_ has quit IRC | 17:15 | |
| *** gvrangan_ has joined #tripleo | 17:16 | |
| *** ckyriakidou has quit IRC | 17:18 | |
| *** gvrangan has quit IRC | 17:21 | |
| *** tosky has quit IRC | 17:21 | |
| *** trown|lunch is now known as trown | 17:22 | |
| Lokesh_Jain_ | Hi.. I am trying to include a new mechanism driver in ML2 code in Newton. But the code is not getting executed. Is openstack-puppet/modules/tripleo/manifests/profile/base/neutron/plugins/ml2.pp the right place to do this? | 17:25 |
| *** ffiore has joined #tripleo | 17:26 | |
| *** masco has quit IRC | 17:26 | |
| *** cylopez has joined #tripleo | 17:28 | |
| *** amoralej is now known as amoralej|off | 17:29 | |
| *** gvrangan has joined #tripleo | 17:32 | |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart master: Add supplemental node provisioner https://review.openstack.org/451523 | 17:34 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart master: Update quickstart-extras to deploy FreeIPA https://review.openstack.org/453223 | 17:35 |
| *** ffiore has quit IRC | 17:38 | |
| beagles | Lokesh_Jain_: yes - this is where we conditionally include puppet code depending on the values in neutron::plugins::ml2::mechanism_drivers | 17:42 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Cnfigure snmpd auth params in ceilometer profile https://review.openstack.org/462237 | 17:42 |
| *** atoth has quit IRC | 17:42 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Configure snmpd auth params in ceilometer profile https://review.openstack.org/462237 | 17:42 |
| beagles | Lokesh_Jain_: have you done this and it isn't working? | 17:43 |
| *** ffiore has joined #tripleo | 17:43 | |
| *** dbecker has quit IRC | 17:43 | |
| *** salmankhan has quit IRC | 17:44 | |
| *** gfidente is now known as gfidente|afk | 17:45 | |
| *** ffiore has quit IRC | 17:48 | |
| *** fragatina has quit IRC | 17:49 | |
| Lokesh_Jain_ | @beagles yes.. i did this and the code isn't working.. are there any dependencies that need to be taken care of? | 17:49 |
| beagles | Lokesh_Jain_: dumb question - did you update your overcloud images to include the changed code? | 17:49 |
| Lokesh_Jain_ | yes :) | 17:49 |
| *** cylopez has quit IRC | 17:50 | |
| beagles | :) | 17:50 |
| Lokesh_Jain_ | if I include my code in ml2 puppet code in the image.. then everything works fine.. its not picking up from tripleo.../ml2.pp | 17:50 |
| beagles | Lokesh_Jain_: that would cause me to suspect that hiera isn't being set properly. Can you verify that neutron::plugins::ml2::mechanism_drivers hiera has your mechanism driver included on the relevant node? | 17:52 |
| Lokesh_Jain_ | beagles: yes.. it is there.. i suspected that so i also tried including my code directly in profile/base/neutron/plugins/ml2.pp without the mechanism driver condition.. just include it always.. still the same result | 17:54 |
| Lokesh_Jain_ | beagles: is there any puppet log that I can take a look at and figure out whats going on.. can add notice/notify and check that | 17:55 |
| *** rbowen has quit IRC | 17:57 | |
| beagles | Lokesh_Jain_: /var/log/messages might help you out there | 17:58 |
| *** gvrangan has quit IRC | 18:06 | |
| Lokesh_Jain_ | beagles: /var/log/messages indicate that my expected code did not run and shows errors pertaining to that.. nothing about why the code didnt execute | 18:08 |
| *** jaganathan has quit IRC | 18:08 | |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 18:10 |
| *** gvrangan has joined #tripleo | 18:11 | |
| *** anshul has quit IRC | 18:12 | |
| *** rbowen has joined #tripleo | 18:13 | |
| *** zoli|wfh is now known as zoli|gone | 18:15 | |
| ansiwen | mwhahaha: could you restore your +2 here?: https://review.openstack.org/#/c/456699/ | 18:18 |
| Lokesh_Jain_ | beagles: are there any debug logs that can be enabled to debug the issue? | 18:19 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart master: Add supplemental node provisioner https://review.openstack.org/451523 | 18:20 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart master: Update quickstart-extras to deploy FreeIPA https://review.openstack.org/453223 | 18:21 |
| *** fragatina has joined #tripleo | 18:21 | |
| *** rbowen has quit IRC | 18:24 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates master: Add back Heat conditions in upgrade workflow https://review.openstack.org/453238 | 18:25 |
| *** jaganathan has joined #tripleo | 18:25 | |
| *** dsariel_ has quit IRC | 18:29 | |
| *** nyechiel has quit IRC | 18:32 | |
| openstackgerrit | Merged openstack/tripleo-ui master: Upgrade to keystone v3 https://review.openstack.org/451431 | 18:33 |
| *** atoth has joined #tripleo | 18:36 | |
| *** atoth has quit IRC | 18:36 | |
| *** atoth has joined #tripleo | 18:37 | |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart-extras master: Parameterize OVB stack flavors https://review.openstack.org/445163 | 18:39 |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart-extras master: Move OVB settings to tripleo-quickstart-extras https://review.openstack.org/448805 | 18:39 |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart-extras master: Use openstackclient from the venv when deleting ovb stacks https://review.openstack.org/462261 | 18:39 |
| *** dbecker has joined #tripleo | 18:39 | |
| *** atoth has quit IRC | 18:40 | |
| *** paramite has quit IRC | 18:40 | |
| *** trown is now known as trown|brb | 18:40 | |
| *** chem is now known as chem_gone | 18:41 | |
| *** trown|brb is now known as trown | 18:46 | |
| *** gvrangan has quit IRC | 18:46 | |
| thrash | d0ugal: wassup? | 18:49 |
| beagles | Lokesh_Jain_: it's curious - is there yaml included in that deployment that includes that puppet. It would take the form of the step_config in neutron-plugin-ml2.yaml | 18:50 |
| thrash | EmilienM: trying to reproduce locally. I did. The reason the introspect is failing does not seem to be because of your patch. But something else is wonky. | 18:51 |
| thrash | EmilienM: It's trying to connect to the ironic-inspector via the wrong IP | 18:51 |
| thrash | EmilienM: I'm not sure where that's controlled TBH | 18:52 |
| thrash | EmilienM: let me check the service catalog | 18:52 |
| EmilienM | thrash: interesting | 18:52 |
| *** slagle has quit IRC | 18:53 | |
| thrash | EmilienM: yeah... it's trying to connect to ironic-inspector on the main interface, and not one of the undercloud vips | 18:54 |
| thrash | which it's not listening on. | 18:54 |
| -openstackstatus- NOTICE: Gerrit on review.openstack.org is being restarted to accomodate a memory leak in Gerrit. Service should return shortly. | 18:54 | |
| EmilienM | trown: hey, can you look https://review.openstack.org/#/c/462063/ please? | 18:57 |
| *** masterjcool has quit IRC | 18:58 | |
| *** gvrangan has joined #tripleo | 19:00 | |
| *** florianf has quit IRC | 19:00 | |
| *** paramite has joined #tripleo | 19:02 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 19:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| *** masterjcool has joined #tripleo | 19:10 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Parameterize OVB stack flavors https://review.openstack.org/445163 | 19:14 |
| *** paramite has quit IRC | 19:16 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Parameterize OVB stack flavors https://review.openstack.org/445163 | 19:16 |
| jprovazn | dsneddon, ping | 19:17 |
| dsneddon | jprovazn, Hello | 19:17 |
| jprovazn | dsneddon, hi, when you have a minute, could you please take a look at this thread http://lists.openstack.org/pipermail/openstack-dev/2017-April/115169.html (longer)? | 19:20 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-docs master: First draft of TripleO Release Management document https://review.openstack.org/459764 | 19:20 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: WIP: Add OVB support to devmode.sh https://review.openstack.org/452249 | 19:20 |
| jprovazn | dsneddon, and because it is a big task, for Pike release we thought that for tech-preview we might use something like this https://blueprints.launchpad.net/tripleo/+spec/nfs-ganesha | 19:20 |
| dsneddon | jprovazn, Thanks, this is the first I've looked at this. | 19:21 |
| jprovazn | dsneddon, I wonder what you think about possibilities how to solve networking setup in long term and for blueprint if it's reasonable to use the solution in the BP | 19:22 |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-ui master: [POC] Use prettier for linting https://review.openstack.org/461377 | 19:27 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: CI test - never merge https://review.openstack.org/461000 | 19:31 |
| EmilienM | sshnaidm|afk: why gate-tripleo-ci-centos-7-ovb-ha-oooq-nv is always red? | 19:31 |
| sshnaidm|afk | EmilienM, that what I try to understand the whole day.. | 19:32 |
| EmilienM | sshnaidm|afk: oops sorry | 19:32 |
| pradk | flaper87, Hi when you have a minute, can you please review https://review.openstack.org/#/c/454314/ | 19:32 |
| sshnaidm|afk | EmilienM, compute node is not deployed, I don't know why yet.. | 19:32 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Parameterize OVB stack flavors https://review.openstack.org/445163 | 19:35 |
| *** mich_chap has quit IRC | 19:37 | |
| *** vkhanna has joined #tripleo | 19:38 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: Add OVB support to devmode.sh https://review.openstack.org/452249 | 19:40 |
| Lokesh_Jain_ | beagles: the default deployment with CorePlugin as ml2 would include puppet/services/neutron-plugin-ml2.yaml, right? | 19:41 |
| *** iranzo has quit IRC | 19:41 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: IPv6 VIP addresses need to be /128 https://review.openstack.org/460028 | 19:43 |
| beagles | Lokesh_Jain_: yeah, unless something is mucking with the resource registry you should be good - it's pretty weird that isn't running. | 19:44 |
| beagles | Lokesh_Jain_: so with respect to the heat templates, all your re doing is adding it NeutronMechanismDrivers in an environment file or something? | 19:45 |
| Lokesh_Jain_ | beagle: but ml2.pp is getting included.. so some code is still including that class | 19:45 |
| *** leanderthal|afk has quit IRC | 19:45 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart master: [WIP] Fix network isolation args for ovb oooq job https://review.openstack.org/462278 | 19:47 |
| Lokesh_Jain_ | beagles: yes mostly.. there is NeutronMechanismDrivers, along with other ml2 config and mechanism driver config parameters | 19:48 |
| * beagles thinks he must be forgetting something obvious | 19:48 | |
| Lokesh_Jain_ | "3:46 PM — @beagles thinks he must be forgetting something obvious" <---- i feel the same way | 19:49 |
| beagles | Lokesh_Jain_: I have to go AFK for awhile to do the supper thing and take son to judo - I'll check when I get back | 19:54 |
| Lokesh_Jain_ | beagles: Thanks for your help! | 19:54 |
| *** artom has quit IRC | 19:57 | |
| *** artom has joined #tripleo | 19:57 | |
| *** leanderthal|afk has joined #tripleo | 19:59 | |
| *** liverpooler has quit IRC | 20:04 | |
| *** rbowen has joined #tripleo | 20:04 | |
| *** gvrangan has quit IRC | 20:07 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add parameter Ec2ApiExternalNetwork for VPCs https://review.openstack.org/456699 | 20:08 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Add subpackages from openstack-tripleo-common from delorean-current https://review.openstack.org/462067 | 20:10 |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 20:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| *** cylopez has joined #tripleo | 20:10 | |
| *** dparkes has joined #tripleo | 20:11 | |
| *** gvrangan has joined #tripleo | 20:11 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: Add OVB support to devmode.sh https://review.openstack.org/452249 | 20:12 |
| *** rbowen has quit IRC | 20:15 | |
| *** toure is now known as toure|gone | 20:18 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: Add OVB support to devmode.sh https://review.openstack.org/452249 | 20:19 |
| *** dprince has quit IRC | 20:19 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo master: Restrict nova migration ssh tunnel https://review.openstack.org/458077 | 20:20 |
| *** skramaja has quit IRC | 20:21 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Restrict nova migration ssh tunnel https://review.openstack.org/458082 | 20:21 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: [N->O] Add openstack-nova-migration to compute nodes. https://review.openstack.org/461115 | 20:21 |
| tdasilva | hello, i'm trying to deploy a containerized overcloud and am running into issues: https://paste.fedoraproject.org/paste/Nhi4coWjz05n1VyWOASK6F5M1UNdIGYhyRLivL9gydE= | 20:23 |
| tdasilva | can anybody provide a pointer? | 20:24 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: Add OVB support to devmode.sh https://review.openstack.org/452249 | 20:25 |
| tdasilva | I tried to configure dns server according to this: http://tripleo.org/basic_deployment/basic_deployment_cli.html#configure-a-nameserver-for-the-overcloud but didn't seem to help | 20:25 |
| *** jaganathan has quit IRC | 20:28 | |
| *** karthiks has quit IRC | 20:28 | |
| dsneddon | jprovazn, My suggestion is to mirror what we do with Swift/Swift Proxy. So the front-end access to Ganesha would be on the Storage network (between the compute/controllers and ganesha role servers), and Ganesha would use the Storage Management network on the back end. | 20:29 |
| *** eck` is now known as eck`gone | 20:29 | |
| dsneddon | jprovazn, We are in progress of implementing custom networks, so you could create one or more dedicated back-end networks for Ganesha. | 20:30 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Support heterogeneous OVB environments https://review.openstack.org/458651 | 20:30 |
| dsneddon | jprovazn, But this means that Ganesha would need its own VIP. | 20:30 |
| dsneddon | jprovazn, Unless it were colocated on the Controllers. | 20:31 |
| *** tiswanso has quit IRC | 20:32 | |
| *** jaganathan has joined #tripleo | 20:32 | |
| jprovazn | dsneddon, re VIP - I think ganesha needs its own VIP no matter if it's on controller nodes or not - it won't be loadbalanced, so VIP should failover together with the ganesha-service itself | 20:33 |
| *** ramishra has quit IRC | 20:34 | |
| *** trown is now known as trown|outtypewww | 20:34 | |
| jprovazn | dsneddon, re networks - if ganesha will be available (front-end) on storage network it means I have to expose this network to tenants, right? | 20:35 |
| *** tiswanso has joined #tripleo | 20:36 | |
| jprovazn | dsneddon, the thing is that according to ceph folks access to the ceph public network (which is mapped to storage network too) is a big privilege is should not be allowed if possible | 20:37 |
| *** gbarros has joined #tripleo | 20:37 | |
| jprovazn | dsneddon, so the goal is to make nfs-ganesha accessible from user instances and keep ceph public network sealed for them | 20:39 |
| *** jaganathan has quit IRC | 20:39 | |
| *** tiswanso has quit IRC | 20:40 | |
| *** ckyriakidou has joined #tripleo | 20:41 | |
| dsneddon | jprovazn, Wait, maybe I misunderstand. | 20:41 |
| dsneddon | jprovazn, Ganesha is the service which presents the storage, but Manila is the client, right? | 20:42 |
| dsneddon | jprovazn, I'm just now realizing that I don't really know how we handle Manila networking in TripleO. | 20:43 |
| jprovazn | dsneddon, manila is openstack share service (so you create, delete shares through it), it uses a backend for shares itself - ceph in this case, ganesha is a proxy service users will interact with when accessing these shares | 20:44 |
| *** gvrangan has quit IRC | 20:44 | |
| jprovazn | dsneddon, manila is extra tricky as it introduces requirement to access shares from user instances (in compare to other services like cinder where these are access from compute nodes only) | 20:45 |
| *** Goneri has joined #tripleo | 20:45 | |
| dsneddon | jprovazn, OK, so that is what is mentioned in the email " ceph is used through ganesha server (what we don't support yet but | 20:45 |
| dsneddon | we would like)" | 20:45 |
| dsneddon | jprovazn, Rule of thumb is to put external services on the External network. I don't see the issue if the Ganesha servers aren't the same boxes as the Controller boxes. They each have their own share of bandwidth on that network. | 20:46 |
| dsneddon | jprovazn, And the tenants are external, too, when you get right down to it. | 20:47 |
| openstackgerrit | Lokesh Jain proposed openstack/tripleo-heat-templates stable/newton: Addition of firewall rules for Nuage https://review.openstack.org/462286 | 20:48 |
| jprovazn | dsneddon, great - that makes the short term BP proposal promising | 20:48 |
| dsneddon | jprovazn, So External (which is really for all External APIs) is a good candidate. The issue, though, is if the tenants are accessing Ganesha's public VIP, are the compute nodes also accessing that VIP? | 20:49 |
| dsneddon | jprovazn, Because the Compute nodes themselves don't have good access to the External network. | 20:49 |
| *** gvrangan_ has quit IRC | 20:49 | |
| dsneddon | jprovazn, In that case, we'd be better off having two Ganesha VIPs, one on Storage and one on External. | 20:50 |
| jprovazn | dsneddon, so user instances would access to ganesha servers through the Storage VIP? | 20:53 |
| *** Erming__ has joined #tripleo | 20:54 | |
| dsneddon | jprovazn, No, tenant instances (VM or bare metal) should access Ganesha through the External network. | 20:54 |
| dsneddon | jprovazn, My point is that if any services running on the Compute virt host itself (not the user instances) needs to access Ganesha, the route to the External network will be through the Control Plane (Provisioning). | 20:55 |
| *** gvrangan has joined #tripleo | 20:56 | |
| dsneddon | jprovazn, So if there are reasons why the Compute nodes would have services that needed to mount Ganesha shares, then we might want a VIP on Storage for that connection. | 20:56 |
| *** Erming_ has quit IRC | 20:56 | |
| jprovazn | dsneddon, ah, well, good news is that ganesha servers need to be accessed from user instances and controller nodes (from nodes where manila-share is running which is now controller node because it requires pacemaker) | 20:56 |
| dsneddon | jprovazn, Cool, then you were right to use the External network. We will enable a port on that network for the Ganesha role. | 20:57 |
| jprovazn | dsneddon, awesome, I did some basic experiment wit this today, AIUI I should: | 20:58 |
| jprovazn | dsneddon, add a new port similar to https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud.j2.yaml#L563 | 20:58 |
| *** tiswanso has joined #tripleo | 20:59 | |
| dsneddon | jprovazn, Yeah, take a look at the VIP we create for Redis. | 21:00 |
| dsneddon | jprovazn, But of course you want that on the External network, like the Public VIP. | 21:00 |
| *** abishop has quit IRC | 21:01 | |
| dsneddon | jprovazn, So you also have to create an entry in the service_net_map.j2.yaml | 21:01 |
| openstackgerrit | Lokesh Jain proposed openstack/tripleo-heat-templates stable/ocata: Addition of firewall rules for Nuage https://review.openstack.org/462289 | 21:01 |
| jprovazn | dsneddon, aha | 21:02 |
| jprovazn | dsneddon, great, thanks, so I'll try copycat Redis* setting tomorrow | 21:03 |
| dsneddon | jprovazn, OK, let me know if you need a hand. | 21:04 |
| jprovazn | (apparently I missed service_net_map.j2.yaml before) | 21:04 |
| *** gbarros has quit IRC | 21:04 | |
| jprovazn | dsneddon, will do, thanks! | 21:04 |
| *** tiswanso has quit IRC | 21:04 | |
| *** jprovazn has quit IRC | 21:04 | |
| dsneddon | jprovazn, Also, you might want to default the VIP to noop.yaml in the overcloud-resource-registry.yaml, then you can override that if Ganesha is enabled. | 21:05 |
| *** Goneri has quit IRC | 21:06 | |
| *** cylopez has quit IRC | 21:09 | |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 21:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Add subpackages from openstack-tripleo-common from delorean-current https://review.openstack.org/462063 | 21:14 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Expose metric delay processing metric https://review.openstack.org/458959 | 21:14 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Set reasonable TTL defaults for Ceilometer DB https://review.openstack.org/462203 | 21:14 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: snmp: add SnmpdBindHost parameter https://review.openstack.org/461839 | 21:14 |
| openstackgerrit | Merged openstack/puppet-tripleo master: snmp: remove useless parameter for binding https://review.openstack.org/461791 | 21:14 |
| *** dtrainor has quit IRC | 21:15 | |
| *** chlong has quit IRC | 21:17 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add back Heat conditions in upgrade workflow https://review.openstack.org/453238 | 21:23 |
| *** bfournie has quit IRC | 21:23 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: Add OVB support to devmode.sh https://review.openstack.org/452249 | 21:24 |
| *** jayg is now known as jayg|g0n3 | 21:27 | |
| *** gvrangan has quit IRC | 21:29 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: Add OVB support to devmode.sh https://review.openstack.org/452249 | 21:32 |
| *** rlandy is now known as rlandy|afk | 21:35 | |
| *** dtrainor has joined #tripleo | 21:38 | |
| EmilienM | soudns like upgrade jobs are broken when upgrading Nova rpm | 21:40 |
| EmilienM | http://logs.openstack.org/00/461000/6/check/gate-tripleo-ci-centos-7-scenario004-multinode-upgrades-nv/042ac17/console.html#_2017-05-03_21_17_46_172971 | 21:40 |
| EmilienM | but I think it was fixed by https://github.com/rdo-packages/nova-distgit/commit/9d4494f925b637440158004762f3dd66f4135e35 | 21:40 |
| *** gfidente|afk has quit IRC | 21:47 | |
| *** garyk has quit IRC | 21:48 | |
| *** vkhanna has left #tripleo | 21:51 | |
| *** gvrangan has joined #tripleo | 21:56 | |
| *** jmelvin has quit IRC | 22:02 | |
| *** jkilpatr has quit IRC | 22:05 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs https://review.openstack.org/359215 | 22:06 |
| EmilienM | sshnaidm|afk: I just kicked a promotion and it passed | 22:07 |
| EmilienM | sshnaidm|afk: do we have progress on adding multinode jobs in the pipeline yet? | 22:07 |
| sshnaidm|afk | EmilienM, oh.. I saw updates failed there | 22:07 |
| EmilienM | sshnaidm|afk: yeah but it's timeout | 22:08 |
| EmilienM | sshnaidm|afk: the pingtest passed | 22:08 |
| EmilienM | so I promoted manually | 22:08 |
| *** dparkes has quit IRC | 22:08 | |
| sshnaidm|afk | EmilienM, will start this tomorrow or next week | 22:09 |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 22:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| *** dciabrin has quit IRC | 22:12 | |
| openstackgerrit | Ben Kero proposed openstack/tripleo-quickstart master: Clean up incorrect comment relating to unused variable test_tempest https://review.openstack.org/462306 | 22:16 |
| openstackgerrit | Ben Kero proposed openstack/tripleo-quickstart-extras master: Clean up unused variables in defaults and comments https://review.openstack.org/462307 | 22:20 |
| *** fragatin_ has joined #tripleo | 22:21 | |
| *** mmethot_ has joined #tripleo | 22:22 | |
| dtrainor | hrm, i have a deployment i did with the ui that shows 'nova list' being empty but 'heat stack-list' shows a stack deployment. | 22:22 |
| *** mgkwill_ has joined #tripleo | 22:23 | |
| dtrainor | "ComputeCount": 0, | 22:24 |
| dtrainor | hrmmm... | 22:24 |
| *** dmacpher has joined #tripleo | 22:26 | |
| *** apetrich_ has joined #tripleo | 22:27 | |
| *** ubijtsa has joined #tripleo | 22:27 | |
| *** ubijtsa has joined #tripleo | 22:27 | |
| *** rodrigod` has joined #tripleo | 22:27 | |
| *** slagle has joined #tripleo | 22:27 | |
| *** tdasilva- has joined #tripleo | 22:28 | |
| *** hexo- has joined #tripleo | 22:28 | |
| *** mgkwill has quit IRC | 22:28 | |
| *** fragatina has quit IRC | 22:28 | |
| *** rodrigods has quit IRC | 22:28 | |
| *** apetrich has quit IRC | 22:28 | |
| *** assassin has quit IRC | 22:28 | |
| *** tdasilva has quit IRC | 22:28 | |
| *** hexo has quit IRC | 22:28 | |
| *** mmethot has quit IRC | 22:28 | |
| *** d0ugal has quit IRC | 22:28 | |
| *** hexo- is now known as hexo | 22:28 | |
| *** mgkwill_ is now known as mgkwill | 22:28 | |
| *** artom has quit IRC | 22:30 | |
| *** artom has joined #tripleo | 22:30 | |
| *** gvrangan has quit IRC | 22:30 | |
| *** artom has quit IRC | 22:32 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart master: [WIP] Fix network isolation args for ovb oooq job https://review.openstack.org/462278 | 22:32 |
| *** artom has joined #tripleo | 22:32 | |
| *** artom has quit IRC | 22:33 | |
| *** artom has joined #tripleo | 22:33 | |
| *** gvrangan has joined #tripleo | 22:37 | |
| *** jkilpatr has joined #tripleo | 22:39 | |
| openstackgerrit | Ben Kero proposed openstack/tripleo-quickstart-extras master: WIP: Give build-test-packages role the option to inject https://review.openstack.org/456321 | 22:41 |
| *** limao_ has joined #tripleo | 22:43 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: CI test - never merge https://review.openstack.org/461000 | 22:44 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: CI test - never merge https://review.openstack.org/461001 | 22:44 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Deploy Mistral with Keystone v3 options (authtoken) https://review.openstack.org/461040 | 22:44 |
| *** morazi has quit IRC | 22:46 | |
| EmilienM | bandini: sounds like CI is failing | 22:46 |
| EmilienM | http://logs.openstack.org/77/458077/19/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/2e8b060/logs/postci.txt.gz#_2017-05-03_21_50_43_000 | 22:46 |
| EmilienM | bandini: on a patch on puppet-pacemaker | 22:46 |
| EmilienM | sounds like it's fixed in https://review.openstack.org/#/c/462073/ | 22:47 |
| *** d0ugal has joined #tripleo | 22:53 | |
| *** ckyriakidou has quit IRC | 22:58 | |
| *** bfournie has joined #tripleo | 22:58 | |
| *** pradk has quit IRC | 23:00 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud deploy/update time" [Critical,In progress] | 23:10 |
| *** gvrangan has quit IRC | 23:10 | |
| *** bugzy has quit IRC | 23:13 | |
| *** bugzy has joined #tripleo | 23:14 | |
| *** ramishra has joined #tripleo | 23:15 | |
| *** funzo has joined #tripleo | 23:21 | |
| *** funzo has quit IRC | 23:26 | |
| *** gvrangan has joined #tripleo | 23:33 | |
| *** ramishra has quit IRC | 23:46 | |
| *** gvrangan has quit IRC | 23:55 | |
| *** garyk has joined #tripleo | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!