| *** fragatina has quit IRC | 00:01 | |
| *** fragatin_ has quit IRC | 00:02 | |
| *** mbound has quit IRC | 00:04 | |
| *** fragatina has joined #tripleo | 00:05 | |
| openstackgerrit | Merged openstack/tripleo-quickstart: add scripts required to test ooo-usbkey workflow https://review.openstack.org/326001 | 00:06 |
|---|---|---|
| *** fragatina has quit IRC | 00:07 | |
| *** fragatin_ has joined #tripleo | 00:07 | |
| *** fragatin_ has quit IRC | 00:08 | |
| *** dmacpher has quit IRC | 00:10 | |
| *** morazi has quit IRC | 00:15 | |
| *** mburned_out is now known as mburned | 00:22 | |
| *** MaxPC has quit IRC | 00:23 | |
| *** MaxPC has joined #tripleo | 00:25 | |
| *** MaxPC has quit IRC | 00:28 | |
| *** dmacpher has joined #tripleo | 00:52 | |
| *** mburned is now known as mburned_out | 00:53 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Release notes for 1.17.0 https://review.openstack.org/326191 | 00:57 |
| *** penick has joined #tripleo | 00:58 | |
| openstackgerrit | Merged openstack/diskimage-builder: Cleanup source-repositories output https://review.openstack.org/321961 | 00:59 |
| openstackgerrit | Merged openstack/diskimage-builder: Refactor: rename temporary directories https://review.openstack.org/325053 | 00:59 |
| openstackgerrit | Merged openstack/diskimage-builder: Export die() function https://review.openstack.org/321892 | 01:00 |
| openstackgerrit | Merged openstack/diskimage-builder: Handle file magic type varying order matching https://review.openstack.org/321126 | 01:00 |
| openstackgerrit | Merged openstack/diskimage-builder: Export FS_TYPE and remove hardcoded ext4 values https://review.openstack.org/318400 | 01:01 |
| openstackgerrit | Merged openstack/diskimage-builder: Add release to pkg-map https://review.openstack.org/319049 | 01:01 |
| openstackgerrit | Merged openstack/diskimage-builder: Refactor: remove unused functions https://review.openstack.org/324792 | 01:01 |
| *** saneax is now known as saneax_AFK | 01:02 | |
| openstackgerrit | Merged openstack/diskimage-builder: Add PS4 to show file/function/line in debug output https://review.openstack.org/317260 | 01:14 |
| openstackgerrit | Merged openstack/diskimage-builder: Add cloud-initramfs-growroot for Precise https://review.openstack.org/318461 | 01:14 |
| *** Goneri has joined #tripleo | 01:14 | |
| *** Goneri has quit IRC | 01:20 | |
| *** mburned_out is now known as mburned | 01:27 | |
| *** cwolferh has quit IRC | 01:36 | |
| *** cwolferh has joined #tripleo | 01:39 | |
| *** r-mibu has quit IRC | 01:42 | |
| *** yamahata has quit IRC | 01:51 | |
| *** cwolferh has quit IRC | 01:56 | |
| *** r-mibu has joined #tripleo | 01:57 | |
| *** lblanchard has quit IRC | 02:08 | |
| *** penick has quit IRC | 02:10 | |
| *** cwolferh has joined #tripleo | 02:15 | |
| openstackgerrit | Merged openstack/diskimage-builder: Release notes for 1.17.0 https://review.openstack.org/326191 | 02:36 |
| *** mbound has joined #tripleo | 02:47 | |
| *** mbound has quit IRC | 02:51 | |
| *** cmyster has quit IRC | 03:03 | |
| *** fragatina has joined #tripleo | 03:10 | |
| *** fragatin_ has joined #tripleo | 03:11 | |
| *** xinwu has quit IRC | 03:11 | |
| *** cmyster has joined #tripleo | 03:12 | |
| *** fragatina has quit IRC | 03:15 | |
| *** panda|Zz has quit IRC | 03:46 | |
| *** panda|Zz has joined #tripleo | 03:46 | |
| *** links has joined #tripleo | 03:56 | |
| *** sshnaidm|afk has quit IRC | 03:59 | |
| *** xinwu has joined #tripleo | 04:00 | |
| *** yamahata has joined #tripleo | 04:04 | |
| *** rbrady has quit IRC | 04:07 | |
| *** rasca has quit IRC | 04:09 | |
| *** rasca has joined #tripleo | 04:10 | |
| *** pcaruana has joined #tripleo | 04:11 | |
| openstackgerrit | Jason E. Rist proposed openstack/tripleo-ui: [WIP] Status Page UI - work in progress https://review.openstack.org/310670 | 04:13 |
| *** rbrady has joined #tripleo | 04:20 | |
| *** masco has joined #tripleo | 04:22 | |
| *** pcaruana has quit IRC | 04:24 | |
| openstackgerrit | Jason E. Rist proposed openstack/tripleo-ui: [WIP] Status Page UI - work in progress https://review.openstack.org/310670 | 04:34 |
| *** pcaruana has joined #tripleo | 04:34 | |
| *** oshvartz has quit IRC | 04:35 | |
| *** numans has joined #tripleo | 04:37 | |
| *** pcaruana has quit IRC | 04:40 | |
| *** diazjf has joined #tripleo | 04:42 | |
| *** dmacpher has quit IRC | 04:43 | |
| *** diazjf has quit IRC | 04:46 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Move pkg-map to dib-python https://review.openstack.org/326228 | 05:02 |
| *** jaosorior has joined #tripleo | 05:05 | |
| *** dmacpher has joined #tripleo | 05:19 | |
| *** jprovazn has joined #tripleo | 05:25 | |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder: Introduce `exit` phase https://review.openstack.org/324811 | 05:31 |
| *** rcernin has joined #tripleo | 05:36 | |
| *** yamahata has quit IRC | 05:36 | |
| *** yamahata has joined #tripleo | 05:37 | |
| *** jaosorior has quit IRC | 05:52 | |
| *** jaosorior has joined #tripleo | 05:53 | |
| *** oshvartz has joined #tripleo | 06:03 | |
| jaosorior | coolsvap: Well, seems that the CI is broken :/ | 06:04 |
| jaosorior | coolsvap: Relevant commit https://review.openstack.org/#/c/326182/ | 06:05 |
| *** dmacpher has quit IRC | 06:06 | |
| *** jaosorior has quit IRC | 06:06 | |
| *** jaosorior has joined #tripleo | 06:07 | |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder: Add python logger configuration https://review.openstack.org/325409 | 06:10 |
| *** jaosorior has quit IRC | 06:11 | |
| *** jaosorior has joined #tripleo | 06:12 | |
| coolsvap | jaosorior, somehow i was able to get my local ci working yesterday night with an exception of pingtest failure | 06:14 |
| coolsvap | i also tried overcloud-update | 06:14 |
| coolsvap | and my ci is again lost | 06:15 |
| coolsvap | :/ | 06:15 |
| *** gumbyblake has joined #tripleo | 06:16 | |
| coolsvap | jaosorior, same issue the primary interface ip configuration is lost | 06:17 |
| *** jaosorior has quit IRC | 06:21 | |
| *** milan has joined #tripleo | 06:23 | |
| *** jaosorior has joined #tripleo | 06:24 | |
| *** ramishra has quit IRC | 06:27 | |
| *** ramishra has joined #tripleo | 06:28 | |
| jaosorior | coolsvap: What do you mean by local CI? Do you have a local zuul/nodepool provisioning hosts? Or do you mean local deployment? | 06:29 |
| *** dsariel has joined #tripleo | 06:32 | |
| coolsvap | jaosorior, local deployment using tripleo.sh | 06:32 |
| *** TSCHAK has quit IRC | 06:33 | |
| jaosorior | coolsvap: right, so the issue is not the tripleo deployment (at the moment) The issue is the nodepool configuration | 06:33 |
| jaosorior | recently there was a switch for using centos nodes from nodepool. And it seems that didn't go too well | 06:33 |
| coolsvap | ah ok | 06:33 |
| *** fragatin_ has quit IRC | 06:34 | |
| *** tremble has joined #tripleo | 06:34 | |
| *** tremble has joined #tripleo | 06:34 | |
| *** ramishra has quit IRC | 06:35 | |
| *** ramishra has joined #tripleo | 06:36 | |
| *** ccamacho|afk has quit IRC | 06:42 | |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-quickstart: Enable gating ansible tripleo roles https://review.openstack.org/326037 | 06:47 |
| *** jtomasek has joined #tripleo | 06:48 | |
| *** ccamacho has joined #tripleo | 06:51 | |
| *** fzdarsky|afk has joined #tripleo | 06:53 | |
| *** fzdarsky|afk is now known as fzdarsky | 06:54 | |
| *** jaosorior has quit IRC | 07:00 | |
| *** jaosorior has joined #tripleo | 07:01 | |
| *** jaosorior has quit IRC | 07:01 | |
| *** gumbyblake has quit IRC | 07:02 | |
| *** florianf has joined #tripleo | 07:04 | |
| *** anshul has joined #tripleo | 07:10 | |
| *** anshul is now known as Guest66820 | 07:10 | |
| *** leanderthal|afk is now known as leanderthal | 07:12 | |
| *** zoli_gone-proxy is now known as zoliXXL | 07:12 | |
| *** mcornea has joined #tripleo | 07:14 | |
| *** xinwu has quit IRC | 07:15 | |
| zoliXXL | good morning | 07:16 |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-quickstart: Enable gating ansible tripleo roles https://review.openstack.org/326037 | 07:20 |
| *** shardy has joined #tripleo | 07:21 | |
| *** jpena|off is now known as jpena | 07:24 | |
| *** jaosorior has joined #tripleo | 07:24 | |
| ccamacho | morning! | 07:25 |
| *** coolsvap is now known as coolsvap|lunch | 07:25 | |
| *** coolsvap|lunch is now known as coolsvap | 07:26 | |
| jaosorior | ccamacho: Gate is broken | 07:26 |
| *** athomas has joined #tripleo | 07:26 | |
| ccamacho | yeahp, for the centos7 slaves? | 07:26 |
| jaosorior | so, no rechecking will work at the moment :/ | 07:26 |
| jaosorior | maybe | 07:26 |
| jaosorior | that's the best guess | 07:26 |
| jaosorior | ccamacho: and the attempt by dprince to fix it https://review.openstack.org/#/c/326182/ is blocked by the project-config-dib gate being broken | 07:27 |
| ccamacho | hey, since a few days ago there are a lot of jobs getting timeouts, (more than usual) | 07:27 |
| jaosorior | yep | 07:27 |
| jaosorior | no idea why | 07:27 |
| *** dtantsur|afk is now known as dtantsur | 07:28 | |
| *** liverpooler has joined #tripleo | 07:28 | |
| ccamacho | the only thing I was able to get from logs, was, all times mysql didnt started.. (on my local env and reviews logs) | 07:28 |
| jaosorior | I see | 07:29 |
| jaosorior | was there anything going on on the mysql logs? | 07:29 |
| *** mikelk has joined #tripleo | 07:29 | |
| jaosorior | ccamacho: You got a patchset that has this behavior? Maybe I could take a look at those logs | 07:29 |
| ccamacho | not saying that is 100% sure, but at least is a clue | 07:29 |
| ccamacho | sure let me check, is it possible to get logs from previous rechecks? | 07:30 |
| dtantsur | yeah, DIB has broken the CI, so stay tuned | 07:30 |
| jaosorior | dtantsur: Was it dib that broke the CI then? | 07:31 |
| jaosorior | dtantsur: cause this was proposed https://review.openstack.org/#/c/326182/ | 07:31 |
| dtantsur | jaosorior, I don't think it broke anything but project-config | 07:32 |
| jaosorior | ok | 07:32 |
| *** liverpooler has quit IRC | 07:34 | |
| *** ebarrera has joined #tripleo | 07:34 | |
| *** ramishra has quit IRC | 07:36 | |
| jaosorior | ccamacho: Anyway, if you see a commit that had this behavior, pass it to me, I have time to take a look | 07:38 |
| *** ramishra has joined #tripleo | 07:38 | |
| ccamacho | sure, the next commit with that error ill let you know ^ | 07:39 |
| *** saneax_AFK is now known as saneax | 07:39 | |
| *** ramishra has quit IRC | 07:47 | |
| *** ifarkas has joined #tripleo | 07:47 | |
| *** ohamada has joined #tripleo | 07:47 | |
| *** ramishra has joined #tripleo | 07:48 | |
| *** dbecker has joined #tripleo | 08:01 | |
| *** rwsu has quit IRC | 08:02 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-tripleo: WIP: integration of the new puppet pacemaker. https://review.openstack.org/309069 | 08:08 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-tripleo: WIP: integration of the new puppet pacemaker. https://review.openstack.org/309069 | 08:21 |
| *** jaosorior has quit IRC | 08:23 | |
| *** jaosorior has joined #tripleo | 08:24 | |
| *** mcornea has quit IRC | 08:28 | |
| *** mcornea has joined #tripleo | 08:29 | |
| *** numans has quit IRC | 08:29 | |
| *** tosky has joined #tripleo | 08:34 | |
| shardy | Morning all - couple of simple docs patches here ready to land if anyone has a moment: | 08:37 |
| shardy | https://review.openstack.org/#/q/topic:nodes_yaml | 08:37 |
| jaosorior | shardy: done | 08:38 |
| shardy | thanks! | 08:38 |
| jaosorior | no worriews | 08:38 |
| jaosorior | *worries | 08:38 |
| openstackgerrit | Merged openstack/tripleo-docs: Update baremetal import to not use --json option https://review.openstack.org/291147 | 08:42 |
| openstackgerrit | Merged openstack/tripleo-docs: Add link from node import to where file format is documented https://review.openstack.org/321035 | 08:42 |
| d0ugal | hrm, so I want to make this change: https://review.openstack.org/#/c/326036/ | 08:44 |
| d0ugal | but it breaks the CLI | 08:44 |
| d0ugal | but if I update the CLI first it still breaks | 08:44 |
| d0ugal | so how should I do it? | 08:44 |
| *** derekh has joined #tripleo | 08:46 | |
| d0ugal | I guess I need to update it to make it optional, remove it from the CLI then update tripleo-common again | 08:46 |
| * d0ugal sighs | 08:46 | |
| jaosorior | derekh: CI seemed pretty broken. Seems like it was the move to centos, so this just merged https://review.openstack.org/#/c/326182/ | 08:47 |
| derekh | jaosorior: ack, thanks | 08:48 |
| *** panda|Zz is now known as panda | 08:50 | |
| *** tesseract has joined #tripleo | 08:50 | |
| *** panda is now known as Guest3007 | 08:51 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Enable Manila integration - as a composable controller service https://review.openstack.org/188137 | 08:51 |
| *** Guest3007 is now known as panda | 08:51 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Revert "Update cireport for centos-7 nodes" https://review.openstack.org/326301 | 08:55 |
| derekh | jaosorior: we should probably revert this also ^^ to switch the report back to fedora jobs | 08:55 |
| shardy | derekh: approved | 08:55 |
| jaosorior | daaamn, that was fast | 08:56 |
| shardy | I just fast-approved it seeing as it only affects the website report | 08:56 |
| jaosorior | yeah dude, no biggie | 08:56 |
| derekh | shardy: makes sense | 08:56 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-docs: Composable roles within services Tutorial https://review.openstack.org/311512 | 08:56 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Revert "Update cireport for centos-7 nodes" https://review.openstack.org/326301 | 08:58 |
| *** milan has quit IRC | 08:59 | |
| *** ramishra has quit IRC | 09:00 | |
| *** ramishra has joined #tripleo | 09:01 | |
| *** electrofelix has joined #tripleo | 09:03 | |
| *** dmk0202 has joined #tripleo | 09:05 | |
| *** Guest66820 is now known as anshul_behl | 09:05 | |
| ccamacho | Folks, I have created a youtube channel for our project (https://www.youtube.com/channel/UCNGDxZGwUELpgaBoLvABsTA), im creating play-lists to collect our work and have a common place for video content. Any interested on help with it as an administrator? | 09:10 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Add a compatability shim to transition tripleoclient https://review.openstack.org/326311 | 09:10 |
| *** dtantsur is now known as dtantsur|bbl | 09:10 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Switch to register_all_nodes_compat to allow an API update https://review.openstack.org/326312 | 09:11 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Remove the unused service_host arg from node registration https://review.openstack.org/326036 | 09:14 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Remove the compatability shim for the CLI https://review.openstack.org/326320 | 09:19 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Remove the unused service_host flag https://review.openstack.org/326321 | 09:19 |
| *** sambetts|afk is now known as sambetts | 09:23 | |
| *** akrivoka has joined #tripleo | 09:27 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Add a compatability shim to transition tripleoclient https://review.openstack.org/326311 | 09:30 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Remove the compatability shim for the CLI https://review.openstack.org/326320 | 09:30 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Remove the unused service_host arg from node registration https://review.openstack.org/326036 | 09:30 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Remove the unused service_host flag https://review.openstack.org/326321 | 09:30 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Switch to register_all_nodes_compat to allow an API update https://review.openstack.org/326312 | 09:30 |
| *** yamahata has quit IRC | 09:30 | |
| *** numans has joined #tripleo | 09:31 | |
| *** paramite has joined #tripleo | 09:37 | |
| *** skramaja has joined #tripleo | 09:38 | |
| *** mbound has joined #tripleo | 09:39 | |
| ccamacho | jistr, marios, quick question, is there any docs related to tripleo current upgrade process? Ill like to review your submissions for composable upgrades, but will like to do some research first. | 09:43 |
| marios | ccamacho: there is a wip doc for upstream https://review.openstack.org/#/c/308985/ | 09:44 |
| ccamacho | marios thanks! | 09:45 |
| marios | ccamacho: and also this https://access.redhat.com/documentation/en/red-hat-openstack-platform/8/director-installation-and-usage/chapter-10-upgrading-the-environment | 09:45 |
| jistr | ccamacho: not sure if by current you mean the old or the new :) re new, the proposed tripleo-spec might be worth reading https://review.openstack.org/#/c/319264/ -- for some of the questions there we already picked answers and started moving forward, so the spec would deserve an update, but in general there's quite a lot of good context imo | 09:45 |
| jistr | re old upgrades what marios linked | 09:46 |
| jistr | and there's a bunch of submissions for new upgrades in code https://review.openstack.org/#/q/status:open+project:openstack/tripleo-heat-templates+branch:master+topic:bp/overcloud-upgrades-per-service | 09:46 |
| ccamacho | jistr, marios thanks a lot :) Ill start reading about it | 09:47 |
| marios | jistr: hey man thanks for the +2 @ https://review.openstack.org/#/c/321027/ ... i think there will be a v10 though so i'll be asking for a revote :/ | 09:47 |
| marios | jistr: am still looking for it but... | 09:47 |
| marios | jistr: 40962:Jun 06 14:36:23 overcloud-controller-2 os-collect-config[2382]: [2016-06-06 14:36:23,627] (heat-config) [INFO] {"deploy_stdout": "overcloud-controller-2: Starting Cluster...\novercloud-controller-1: Starting Cluster...\novercloud-controller-0: Starting Cluster...\nOFFLINE: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]\nOFFLINE: [ overcloud-controller-0 overcloud-cont | 09:48 |
| marios | roller-1 overcloud-controller-2 ]\nOFFLINE: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]\novercloud-controller-2 isn't bootstrap, skipping check for ip-172.16.2.5 here\n", "deploy_stderr": "", "deploy_status_code": 0} | 09:48 |
| jistr | marios: sure thing | 09:48 |
| marios | oops sry... anyway jistr i mean it looks like the cluster start is happening on all nodes? | 09:48 |
| marios | jistr: k | 09:48 |
| jistr | marios: unless such output is from all 3 SoftwareDeployments, i wouldn't think it's necessarily a problem. I mean, it might be just a way that pcs prints progress of cluster start --all, even if it's executed from just one node. | 09:50 |
| marios | jistr: yes this is what i'm rying to work out. but there is a nit somewhere... may not be this | 09:52 |
| marios | jistr: thanks | 09:52 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Don't overwrite ceph osd_journal_size https://review.openstack.org/323245 | 09:53 |
| openstackgerrit | Robin Cernin proposed openstack/tripleo-heat-templates: [tripleo-heat-templates] Fix multi-line description https://review.openstack.org/324890 | 10:01 |
| jokke_ | shardy: around? | 10:08 |
| shardy | jokke_: hi! | 10:08 |
| jokke_ | shardy: ref https://review.openstack.org/#/c/323245/ do we want to backport that? If so, I can open a bug for it (even the master patch was not linked to it) | 10:09 |
| jokke_ | shardy: Originally I did not think it being worth of backporting, but I see the point of it now | 10:10 |
| shardy | jokke_: If the default isn't production appropriate and it won't impact upgrades then it seems like a worthwhile backport candidate to me | 10:12 |
| shardy | if you raise a bug please just link the master patch, and reference the bug in the commit message of any backport | 10:13 |
| jokke_ | I have another of these one liners coming as well so thus interested ... on the line 5 we define the defaulth path to /srv/data which is not perhaps the greatest as OSD uses /var/lib/ceph/osd and if another path is defined it creates symlink to that (this though would impact upgrade so maybe even more worth while to discuss in the bug first?) | 10:13 |
| jokke_ | shardy: makes sense. I'll open the bug and propose the backport | 10:14 |
| shardy | Yeah, anything which impacts upgrades will require some thought, so discussion in a bug is a good place for that | 10:14 |
| *** rwsu has joined #tripleo | 10:15 | |
| *** milan has joined #tripleo | 10:16 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: Allow to manually disable post-puppet restarts https://review.openstack.org/326364 | 10:19 |
| *** zoliXXL is now known as zoli|lunch | 10:19 | |
| *** zoli|lunch is now known as zoli_gone-proxy | 10:20 | |
| *** rwsu has quit IRC | 10:20 | |
| *** rwsu has joined #tripleo | 10:21 | |
| jokke_ | shardy: https://bugs.launchpad.net/tripleo/+bug/1589909 if you don't mind scope it for backporting | 10:23 |
| openstack | Launchpad bug 1589909 in tripleo "Overwitten default OSD journal size is not sufficient for production" [Undecided,New] - Assigned to Erno Kuvaja (jokke) | 10:23 |
| *** athomas has quit IRC | 10:24 | |
| *** ccamacho is now known as ccamacho|lunch | 10:28 | |
| *** rwsu has quit IRC | 10:31 | |
| *** rwsu has joined #tripleo | 10:31 | |
| *** dtantsur|bbl is now known as dtantsur | 10:32 | |
| *** paramite is now known as paramite|afk | 10:35 | |
| openstackgerrit | Erno Kuvaja proposed openstack/tripleo-heat-templates: Don't overwrite ceph osd_journal_size https://review.openstack.org/326372 | 10:36 |
| *** rhallisey has joined #tripleo | 10:37 | |
| *** rhallisey has quit IRC | 10:38 | |
| *** rhallisey has joined #tripleo | 10:38 | |
| *** julim has joined #tripleo | 10:40 | |
| *** itamarl has joined #tripleo | 10:46 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-docs: Rework nodes registration and configuration https://review.openstack.org/322776 | 10:49 |
| *** MaxPC has joined #tripleo | 10:50 | |
| openstackgerrit | Erno Kuvaja proposed openstack/tripleo-heat-templates: Use /var/lib/ceph/osd as Ceph expects OSDs to be found there https://review.openstack.org/326376 | 10:51 |
| *** rwsu has quit IRC | 10:56 | |
| *** rhallisey has quit IRC | 10:57 | |
| *** weshay has joined #tripleo | 10:57 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: update the readme https://review.openstack.org/319239 | 11:00 |
| *** paramite|afk is now known as paramite | 11:02 | |
| *** cmyster has quit IRC | 11:12 | |
| *** MaxPC has quit IRC | 11:13 | |
| *** cmyster has joined #tripleo | 11:17 | |
| *** pkovar has joined #tripleo | 11:21 | |
| openstackgerrit | Merged openstack/tripleo-quickstart: update the readme https://review.openstack.org/319239 | 11:22 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Add a compatability shim to transition tripleoclient https://review.openstack.org/326311 | 11:28 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Remove the compatibility shim for the CLI https://review.openstack.org/326320 | 11:28 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Remove the unused service_host arg from node registration https://review.openstack.org/326036 | 11:28 |
| *** ccamacho|lunch is now known as ccamacho | 11:31 | |
| openstackgerrit | Robin Cernin proposed openstack/tripleo-heat-templates: [tripleo-heat-templates] Fix multi-line description https://review.openstack.org/324890 | 11:32 |
| *** jpena is now known as jpena|lunch | 11:34 | |
| *** rhallisey has joined #tripleo | 11:37 | |
| *** lucasagomes is now known as lucas-hungry | 11:40 | |
| *** coolsvap_ has joined #tripleo | 11:44 | |
| *** coolsvap has quit IRC | 11:45 | |
| *** panda has quit IRC | 11:45 | |
| *** coolsvap_ has quit IRC | 11:46 | |
| *** coolsvap has joined #tripleo | 11:46 | |
| *** panda has joined #tripleo | 11:47 | |
| *** anshul_behl has quit IRC | 11:50 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Gate ansible-role-tripleo-* roles https://review.openstack.org/326037 | 11:54 |
| *** thrash|g0ne is now known as thrash | 11:56 | |
| *** akuznetsov has joined #tripleo | 11:57 | |
| *** dprince has joined #tripleo | 11:59 | |
| *** paramite is now known as paramite|afk | 12:02 | |
| *** pkovar has quit IRC | 12:02 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Add a compatability shim to transition tripleoclient https://review.openstack.org/326311 | 12:04 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Remove the compatibility shim for the CLI https://review.openstack.org/326320 | 12:04 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Remove the unused service_host arg from node registration https://review.openstack.org/326036 | 12:04 |
| *** anshul_behl has joined #tripleo | 12:05 | |
| *** coolsvap has quit IRC | 12:07 | |
| *** rodrigods has quit IRC | 12:09 | |
| *** rodrigods has joined #tripleo | 12:09 | |
| *** jayg|g0n3 is now known as jayg | 12:11 | |
| *** TSCHAK has joined #tripleo | 12:11 | |
| *** dmacpher has joined #tripleo | 12:11 | |
| openstackgerrit | Erno Kuvaja proposed openstack/diskimage-builder: Add cinder-backup mappings https://review.openstack.org/326411 | 12:15 |
| openstackgerrit | Erno Kuvaja proposed openstack/tripleo-image-elements: Install and enable cinder-backup https://review.openstack.org/326412 | 12:16 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-tripleo: WIP: integration of the new puppet pacemaker. https://review.openstack.org/309069 | 12:16 |
| *** mgould|afk is now known as mgould | 12:17 | |
| *** pkovar has joined #tripleo | 12:20 | |
| *** trown|outtypewww is now known as trown | 12:22 | |
| EmilienM | hello | 12:23 |
| *** saneax is now known as saneax_AFK | 12:24 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: Additional note for usbkey users regarding the latest images https://review.openstack.org/326419 | 12:24 |
| *** rlandy has quit IRC | 12:28 | |
| *** rlandy has joined #tripleo | 12:28 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Remove nova-cert from TripleO https://review.openstack.org/326076 | 12:29 |
| *** paramite|afk is now known as paramite | 12:29 | |
| *** jcoufal has joined #tripleo | 12:30 | |
| openstackgerrit | Ryan Brady proposed openstack/tripleo-common: Add Heat Capabilities Actions https://review.openstack.org/284460 | 12:34 |
| *** saneax_AFK is now known as saneax | 12:35 | |
| *** MaxPC has joined #tripleo | 12:35 | |
| social__ | we use nova network on undercloud? DEFAULT/linuxnet_interface_driver nova.network.linux_net.LinuxOVSInterfaceDriver | 12:36 |
| *** jdob has quit IRC | 12:36 | |
| *** jdob has joined #tripleo | 12:37 | |
| *** zoli_gone-proxy is now known as zoliXXL | 12:37 | |
| *** paramite is now known as paramite|afk | 12:39 | |
| *** lucas-hungry is now known as lucasagomes | 12:40 | |
| *** MaxPC has joined #tripleo | 12:40 | |
| *** paramite|afk is now known as paramite | 12:41 | |
| *** jpena|lunch is now known as jpena | 12:44 | |
| jaosorior | pabelanger: awake yet? | 12:44 |
| *** numans has quit IRC | 12:47 | |
| jaosorior | shardy: Do you know anyone from infra-root | 12:47 |
| shardy | jaosorior: No, but you could ask in #openstack-infra | 12:47 |
| jaosorior | shardy: I did | 12:47 |
| *** paramite is now known as paramite|afk | 12:49 | |
| openstackgerrit | Lukas Bezdicka proposed openstack/instack-undercloud: [WIP] Move class options to hiera https://review.openstack.org/326436 | 12:49 |
| *** links has quit IRC | 12:51 | |
| openstackgerrit | Lukas Bezdicka proposed openstack/instack-undercloud: [WIP] Use oslo::messaging::default for setting rpc_response_timeout https://review.openstack.org/326438 | 12:51 |
| *** links has joined #tripleo | 12:52 | |
| jaosorior | shardy: found someone already. Our CI issues are being looked at now | 12:55 |
| *** paramite|afk is now known as paramite | 12:55 | |
| *** morazi has joined #tripleo | 12:55 | |
| *** paramite is now known as paramite|afk | 12:56 | |
| shardy | jaosorior: thanks! | 12:57 |
| jaosorior | EmilienM: Just saw you did a recheck. By the way, CI is broken. there was an issue with the migration to centos nodes. And this morning a revert landed; But the revert hasn't reflected in openstack infra | 12:57 |
| EmilienM | ah ok | 12:57 |
| EmilienM | thx | 12:57 |
| jaosorior | no worries | 12:57 |
| jaosorior | there is someone taking a look at it, apparently there hasn't been a refresh of configuration in more than 24 hours | 12:57 |
| *** akuznetsov has quit IRC | 13:00 | |
| social__ | EmilienM: I'm trying to cleanup undercloud manifests :( | 13:00 |
| social__ | EmilienM: looking into if we really need the options | 13:00 |
| *** paramite|afk has quit IRC | 13:00 | |
| EmilienM | social__: cool, I figured.. | 13:00 |
| EmilienM | I did the same a few weeks ago | 13:00 |
| EmilienM | but didn't continue for other services | 13:01 |
| *** pkovar has quit IRC | 13:03 | |
| *** rwsu has joined #tripleo | 13:03 | |
| *** anshul_behl has quit IRC | 13:05 | |
| *** fultonj has quit IRC | 13:06 | |
| *** fultonj has joined #tripleo | 13:07 | |
| *** Goneri has joined #tripleo | 13:08 | |
| *** zoliXXL is now known as zoli|afk | 13:15 | |
| *** jaosorior has quit IRC | 13:15 | |
| *** jaosorior has joined #tripleo | 13:16 | |
| *** pkovar has joined #tripleo | 13:17 | |
| *** akshai has joined #tripleo | 13:17 | |
| *** anshul_behl has joined #tripleo | 13:18 | |
| openstackgerrit | Brad P. Crochet proposed openstack/puppet-tripleo: Add Mistral profiles https://review.openstack.org/323431 | 13:22 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Remove the unused service_host flag https://review.openstack.org/326321 | 13:23 |
| *** dsariel has quit IRC | 13:25 | |
| *** dkehn has quit IRC | 13:32 | |
| *** _milan_ has joined #tripleo | 13:32 | |
| *** lblanchard has joined #tripleo | 13:33 | |
| *** milan has quit IRC | 13:33 | |
| *** dkehn has joined #tripleo | 13:34 | |
| *** oshvartz has quit IRC | 13:34 | |
| *** dsariel has joined #tripleo | 13:38 | |
| *** tzumainn has joined #tripleo | 13:47 | |
| *** egafford has joined #tripleo | 13:48 | |
| EmilienM | weekly meeting in 10 min | 13:51 |
| shardy | https://wiki.openstack.org/wiki/Meetings/TripleO | 13:52 |
| shardy | we don't currently have any one-off items, feel free to add them if you wish | 13:52 |
| jokke_ | what's the channel for that? I seem to miss these | 13:53 |
| shardy | #openstack-meeting-alt | 13:53 |
| jokke_ | thanks ... I think I do not have that on my reg channel list | 13:53 |
| jaosorior | I don't see anything in the agenda though | 13:53 |
| jaosorior | oh, I just saw shardy mentioned it :P | 13:54 |
| shardy | jaosorior: well we have all the recurring items and anything folks want to raise in open discussion | 13:54 |
| shardy | maybe it'll be a quick meeting today ;) | 13:54 |
| *** tzumainn has quit IRC | 13:55 | |
| EmilienM | do we have an ETA on CI to come back? | 13:55 |
| jaosorior | EmilienM: No. But at least the infra-root guys said that things were looking normal again. So maybe it's just a matter of the infra's configuration refreshing sometime soon | 13:56 |
| jaosorior | not sure how long that refresh takes | 13:56 |
| *** zoli|afk is now known as zoli | 13:56 | |
| EmilienM | so many variables | 13:57 |
| EmilienM | let's try now | 13:57 |
| *** rohitpagedar__ has joined #tripleo | 13:58 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Remove loadbalancer profile https://review.openstack.org/325893 | 13:58 |
| trozet | dprince: per our discussion about kernel args yesterday, was able to set them doing this: https://github.com/trozet/ironic-python-agent/commit/49f4e7599384031f2d8f6bab462612b938c458a5 | 14:00 |
| pabelanger | morning | 14:00 |
| trozet | dprince: its a hack that allows us to do what we need in opnfv, but I'll submit a proper fix upstream when I have time | 14:00 |
| dprince | trozet: cool, yeah it would be nice to somehow integrate the management of the args alongside the glance image perhaps | 14:01 |
| *** Goneri has quit IRC | 14:01 | |
| *** yamahata has joined #tripleo | 14:02 | |
| trozet | dprince: also, in opnfv we have ran into a bunch of race conditions/resource contention with tripleO HA deployments in step 2 | 14:02 |
| trozet | dprince: with stable/mitaka | 14:02 |
| pabelanger | I see the migration to centos-7 was reverted. Looking at the logs, I don't see any obvious issues with centos-7. So the issues from yesterday have likely exposed problems (race conditions) in the scripts for centos. Specifically, openstack overcloud deploy seems to take 2+ hours ( with no logs generated ) and gearman server seems to be problematic. Who has time today to look into the failed jobs to get more | 14:02 |
| pabelanger | information? | 14:02 |
| trozet | dprince: the problem results in mysql/galera failures and ceph client failures during deployment with timeouts while trying to use the services | 14:03 |
| dprince | pabelanger: perhaps we can bring this up under CI in #openstack-meeting-alt now? | 14:03 |
| trozet | dprince: to fix it we moved ceph setup to step 1, and galera to step 1, along with serializing the DB schema upgrades and putting sleeps between them | 14:03 |
| pabelanger | dprince: sure | 14:03 |
| dprince | trozet: interesting, step 1 was meant to be loadbalancer only | 14:04 |
| trozet | dprince: testing this we now see 15/16 deployments successful, not sure how to get those fixes into stable/mitaka now | 14:04 |
| dprince | trozet: we can revisit the steps I think if we need to for these things | 14:04 |
| trozet | dprince: yeah we saw this on baremetal deployments as well, so we knew it wasn't a VM resource issue anymore in virtual deployments | 14:06 |
| trozet | dprince: the first fix that helped is this one: https://github.com/trozet/opnfv-tht/commit/2a074ad92c88c2e6e9d336ffbd256c56a8e01437 | 14:06 |
| *** olap has joined #tripleo | 14:06 | |
| trozet | dprince: the follow up commit that has enhanced the stability: https://github.com/trozet/opnfv-tht/pull/21/files | 14:07 |
| social__ | EmilienM: doesn't that mean that if I have ironic and nova and I set rpc_service_timeout I'll get duplicate definition error from puppet? | 14:08 |
| EmilienM | social__: currently in OOO meeting... | 14:08 |
| EmilienM | social__: please read puppet-nova code | 14:09 |
| EmilienM | and how we use puppet-oslo | 14:09 |
| trozet | dprince: i'm sure it there is probably a better solution than just sleeps, but either way, I think something needs to be directly committed to stable/mitaka to fix this stuff | 14:09 |
| social__ | ack | 14:09 |
| openstackgerrit | Dan Prince proposed openstack-infra/tripleo-ci: Add TripleO blogs from Marios to planet TripleO https://review.openstack.org/326493 | 14:10 |
| dprince | marios: ^^ | 14:10 |
| *** fungi has joined #tripleo | 14:10 | |
| dprince | marios: tested locally and it works fine | 14:11 |
| marios | heh tx dprince 'blogs' there is only one tripleo post there so far :) | 14:11 |
| marios | thanks | 14:11 |
| EmilienM | social__: https://review.openstack.org/#/c/321818/ | 14:12 |
| EmilienM | social__: this is the way to go ^ | 14:12 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Aodh upgrade heat template and manifests https://review.openstack.org/320150 | 14:14 |
| *** chlong has quit IRC | 14:16 | |
| *** Goneri has joined #tripleo | 14:17 | |
| *** ramishra has quit IRC | 14:18 | |
| *** ramishra has joined #tripleo | 14:19 | |
| *** anshul_behl has quit IRC | 14:23 | |
| *** sshnaidm|afk has joined #tripleo | 14:23 | |
| *** Goneri has quit IRC | 14:23 | |
| openstackgerrit | François Charlier proposed openstack/tripleo-image-elements: Fix is_local_ip matching https://review.openstack.org/326511 | 14:29 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: Use the default quickstart dir for hosts and ssh.config https://review.openstack.org/326514 | 14:31 |
| *** chlong has joined #tripleo | 14:32 | |
| *** _milan_ has quit IRC | 14:32 | |
| *** Goneri has joined #tripleo | 14:37 | |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-ui: [WIP]Adds thunk component, actions, records, reducer for a service status page. https://review.openstack.org/312494 | 14:38 |
| *** noslzzp has quit IRC | 14:42 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Gate ansible-role-tripleo-* roles https://review.openstack.org/326037 | 14:44 |
| openstackgerrit | François Charlier proposed openstack/tripleo-image-elements: Fix is_local_ip matching https://review.openstack.org/326511 | 14:50 |
| jmiu | can anyone tell me what the dashes in this file stand for? | 14:51 |
| jmiu | https://paste.fedoraproject.org/375884/10994146/ | 14:51 |
| jmiu | specifically, the one under network_config and members. i've been trying to learn how to edit heat templates to tweak my overcloud deployment | 14:52 |
| derekh | dprince: shardy pabelanger I believe the geard problem should now be resolved, geard seems to be leaking open files so I restarted it, I'll do a lsof later and see if I can figure out whats being left open | 14:54 |
| dprince | derekh: could you add my ssh key to that server and or document how to ninja in. | 14:55 |
| dprince | derekh: also, I'd like to get pabelanger access as well so he can jump in to troubleshoot as well | 14:55 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Rework the pacemaker_common_functions for M..N upgrades https://review.openstack.org/321027 | 14:55 |
| jrist | https://i.imgflip.com/15fumt.jpg | 14:55 |
| *** weshay has quit IRC | 14:56 | |
| derekh | dprince: your key is on all the servers, I believe, its definitely on this one | 14:56 |
| derekh | dprince: you can ssh -A to the mirror server (it has a floating IP), from there ssh to the geard server or proxy etc... | 14:57 |
| dprince | derekh: sorry, I forget these things. I said that blindly just now :) | 14:57 |
| dprince | derekh: cool | 14:57 |
| *** masco has quit IRC | 14:57 | |
| pabelanger | derekh: dprince: thanks, do you mind helping to see why logs were copied on openstack overcloud deploy failures? | 14:58 |
| derekh | dprince: no prob, before we did the HW upgrade I made sure keys for you, slagle, bnemec and zoli were on all the servers they needed to be on as some people we one some servers on not others | 14:59 |
| *** leanderthal is now known as leanderthal|afk | 14:59 | |
| *** tzumainn has joined #tripleo | 15:00 | |
| derekh | pabelanger: sure, I guess the timout means the log copy just didn't get run, can you point me at an example | 15:00 |
| dprince | pabelanger: could we try an experimental job again to see if that hits the same issue? | 15:00 |
| *** mcornea has quit IRC | 15:01 | |
| derekh | shardy: bnemec I'm suggesting we move the tripleo-cd-admins to tripleo-ci as part of the move to OVB https://review.openstack.org/#/c/295243/6/scripts/tripleo-cd-admins | 15:01 |
| *** absubram has quit IRC | 15:02 | |
| bnemec | derekh: Makes sense to me. | 15:03 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-incubator: Mark this repo as deprecated https://review.openstack.org/326561 | 15:05 |
| pabelanger | dprince: yes, check experimental on a patch should work | 15:05 |
| * bnemec solves the bad SEO problem with tripleo-docs | 15:05 | |
| *** mcornea has joined #tripleo | 15:05 | |
| dtantsur | folks, quick question: is the tripleo CI broken right now? | 15:06 |
| pabelanger | derekh: http://logs.openstack.org/13/318413/20/check-tripleo/gate-tripleo-ci-centos-7-nonha/7597d08/console.html is an example where tripleo.sh --overcloud-deploy took more then 2+ hours and was eventually likked | 15:06 |
| pabelanger | killed* | 15:07 |
| *** tzumainn has quit IRC | 15:07 | |
| trown | dtantsur: yes | 15:07 |
| dtantsur | got it, thnx | 15:08 |
| *** oshvartz has joined #tripleo | 15:09 | |
| *** saneax is now known as saneax_AFK | 15:10 | |
| dprince | shardy: I think you pushed a patch to drop a neutron param at one point. Here is a bunch of them https://review.openstack.org/#/c/325625/ | 15:14 |
| *** paramite has joined #tripleo | 15:15 | |
| dprince | jaosorior: hey, so I'll be +2 on your https://review.openstack.org/#/c/325755/ once it matches t-h-t again. Sorry for the confusion here but I don't think that is related to any of the functional failures I've seen yet because the base manifests still include ::neutron at this point anyways. | 15:15 |
| marios | bandini: jistr i reworked the logic a bit for v10 esp https://review.openstack.org/#/c/321027/9..10/extraconfig/tasks/pacemaker_common_functions.sh (see the is_systemd_managed and is_pacemaker_managed especially) added relevant comment on review thanks appreciate (whenever tomorrow etc) | 15:15 |
| *** coolsvap has joined #tripleo | 15:15 | |
| dprince | jaosorior: anyways, we can land your patch too and then I'll update t-h-t to drop the remaining ::neutron ::neutron::config bits too | 15:16 |
| *** rcernin has quit IRC | 15:17 | |
| *** tremble has quit IRC | 15:21 | |
| shardy | dprince: nice, lgtm | 15:23 |
| *** coolsvap has quit IRC | 15:26 | |
| *** links has quit IRC | 15:26 | |
| *** weshay has joined #tripleo | 15:27 | |
| derekh | pabelanger: was on a call, looking now | 15:28 |
| *** itamarl has quit IRC | 15:28 | |
| *** olap has quit IRC | 15:28 | |
| pabelanger | derekh: thanks | 15:29 |
| pabelanger | dprince: 325330 is running in experimental-tripleo | 15:29 |
| *** coolsvap has joined #tripleo | 15:30 | |
| derekh | pabelanger: ya, so it looks like what I thought, we have a function "postci" that is called with an trap on EXIT http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_instack.sh#n100 | 15:31 |
| derekh | pabelanger: but because the overcloud deploy command never returned the script didn't exit , it just go killed | 15:31 |
| derekh | pabelanger: so postci never got run | 15:31 |
| *** xinwu has joined #tripleo | 15:31 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Add TripleO blogs from Marios to planet TripleO https://review.openstack.org/326493 | 15:32 |
| derekh | pabelanger: postci is what gets logs from the undercloud / overcloud nodes | 15:32 |
| *** mikelk has quit IRC | 15:33 | |
| derekh | pabelanger: a quick a dirty way to debug this problem would be to just put a timout around "tripleo.sh --overcloud-deploy" so that it happens before devstack-gate timeout | 15:33 |
| derekh | pabelanger: or a proper solution would be to possibly move our log file retrieval into its own publisher in the jjb or put a timeout around the call to toci_instack.sh that is triggered before the devstack-gate timeout | 15:35 |
| *** mgarciam has joined #tripleo | 15:37 | |
| *** lifeless has quit IRC | 15:37 | |
| openstackgerrit | Matt Young proposed openstack/tripleo-quickstart: Add clean virtualenv option to quickstart https://review.openstack.org/326586 | 15:38 |
| jmiu | so, back to what we were saying | 15:42 |
| ayoung | jmiu,you were using the extra config approach? | 15:42 |
| jmiu | yes | 15:42 |
| ayoung | jaosorior, did you have an example that I missed? | 15:43 |
| jmiu | i see another typo. let me make that go away | 15:43 |
| jaosorior | not that I know of. Only documentation about it that I know of in the one in the docs, the one we talked about yesterday | 15:44 |
| jmiu | wait hang on this is exciting it hasn't failed | 15:44 |
| jaosorior | yay! | 15:45 |
| openstackgerrit | Matt Young proposed openstack/tripleo-quickstart: Add clean virtualenv option to quickstart https://review.openstack.org/326586 | 15:45 |
| jmiu | i'm going to be very angry at myself if it was because of that typo | 15:45 |
| jaosorior | lol | 15:45 |
| * jmiu flaps arms in excitement | 15:45 | |
| jaosorior | jmiu: You have no idea how many issues I've had because of typos I've made | 15:45 |
| jmiu | jaosorior: haha, to be human | 15:46 |
| *** lifeless has joined #tripleo | 15:49 | |
| *** rbrady_ has joined #tripleo | 15:52 | |
| jaosorior | jmiu: Let us know how it goes | 15:52 |
| jmiu | jaosorior: will do! still deploying | 15:53 |
| *** rbrady has quit IRC | 15:53 | |
| *** rbrady_ has quit IRC | 15:53 | |
| *** absubram has joined #tripleo | 15:55 | |
| openstackgerrit | Matt Young proposed openstack/tripleo-quickstart: Add clean virtualenv option to quickstart https://review.openstack.org/326586 | 15:56 |
| *** smalleni has joined #tripleo | 15:56 | |
| EmilienM | shardy: https://bugs.launchpad.net/tripleo/+bug/1590057 | 15:58 |
| openstack | Launchpad bug 1590057 in tripleo "keystoneauth1 500 error on stable/mitaka during pingtest" [Critical,Confirmed] | 15:58 |
| *** rbrady has joined #tripleo | 15:58 | |
| EmilienM | shardy: are you sure it used to work > 6 days ago? | 15:58 |
| rook | trown so, wrt TripleO and how neutron is configured, we see there is a dnsmasq entry that is really messing things up -- and that we are setting the MTU to 1400 (which in turn neutron defaults to 1350, when instead we should leave 1500 (the default) so neutron would -50 and we would get 1450)... | 15:59 |
| rook | trown so, we want to see where this fix needs to be put into place. | 15:59 |
| rook | looking through the tht i don't see anything other than the dnsmasq stuff that should just be removed for Mitaka. | 15:59 |
| trown | rook: is it a puppet-neutron default we just inherit? | 15:59 |
| rook | i don't see anything there either, but I might be missing something. | 15:59 |
| *** dsariel has quit IRC | 16:03 | |
| shardy | EmilienM: No, I'm only sure it's been broken for at least 6 days | 16:05 |
| rook | trown this might be our issue : https://github.com/openstack/tripleo-heat-templates/blob/a6438a2082792e7528bfed682c64e352367f8f6f/puppet/services/neutron-dhcp.yaml#L23 | 16:06 |
| shardy | all of the cistatus jobs were failing and the oldest was 5 days yesterday | 16:06 |
| EmilienM | shardy: ah ok | 16:06 |
| rook | also, i think we can stop creating this : https://github.com/openstack/tripleo-heat-templates/blob/a6438a2082792e7528bfed682c64e352367f8f6f/puppet/services/neutron-dhcp.yaml#L16 | 16:06 |
| EmilienM | shardy: how can I know since when it's broken? | 16:06 |
| rook | dprince ^^ seems like you have touched this file :) | 16:06 |
| EmilienM | shardy: I'll lookup in gerrit and manually find latest patch that worked :) | 16:07 |
| trown | rook: ah yep, though that is overridable via environment file | 16:07 |
| shardy | EmilienM: I started hacking on the script we use in CI to find that out, give me a few mins and I'll see if I can finish it | 16:07 |
| * trown does not know why the default is 1400, but suspects there is some reason | 16:07 | |
| rook | trown: this is causing huge issues -- mtu mismatches which causes ssh to fail to the guest. | 16:07 |
| dprince | rook: if I recall EmilienM tried to remove the dnsmas options as well but it didn't functionally work for me | 16:07 |
| dprince | rook: have you tried it without the file? | 16:08 |
| rook | trown all the tap devices will have 1350 as the MTU, but dnsmasq gives out 1400 | 16:08 |
| rook | smalleni ^^ | 16:08 |
| rook | yes | 16:08 |
| dprince | rook: I think this may be a case of Neutron says they support something but it didn't work for us | 16:08 |
| rook | dprince: so, i think this is new to Mitaka | 16:08 |
| EmilienM | shardy: it worked on 18th may | 16:09 |
| dprince | rook: sure, what I'm saying is this came up with we landed the puppet-tripleo neutron profile (in Mitaka) | 16:09 |
| smalleni | dprince, we tried commenting out the /etc/neutron/dnsmaq-neutron.conf file and the instance where honoring the mtu on the network | 16:09 |
| dprince | rook: and when we tried not configuring the dhcp-option settings it failed CI | 16:09 |
| EmilienM | shardy: it broke after | 16:09 |
| rook | dprince: interesting, what failed? | 16:09 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Aodh upgrade heat template and manifests https://review.openstack.org/320150 | 16:09 |
| EmilienM | jayg: can you recall when you updated OPM for mitaka? | 16:09 |
| *** xinwu has quit IRC | 16:09 | |
| dprince | rook: not sure exactly. We can try it again I suppose | 16:09 |
| EmilienM | jayg: we're investigating why stable/mitaka CI on tripleo is broken | 16:10 |
| EmilienM | it's only on nonha job though, so I have a doubt about OPM but let's make sure | 16:10 |
| jayg | EmilienM: let me check, I think it was over a week ago | 16:10 |
| dprince | rook: if you comment out this it would test it I think: http://git.openstack.org/cgit/openstack/puppet-tripleo/tree/manifests/profile/base/neutron/dhcp.pp#n38 | 16:11 |
| jayg | EmilienM: newest opm build I tagged is -https://github.com/redhat-openstack/openstack-puppet-modules/releases/tag/8.1.0 from 11 days ago | 16:11 |
| rook | so two things i think need to happen dprince smalleni | 16:11 |
| EmilienM | jayg: and before? I think something broke around May 18th | 16:11 |
| *** yamahata has quit IRC | 16:11 | |
| rook | comment out what dprice mentions | 16:11 |
| rook | but also; update this : https://github.com/openstack/tripleo-heat-templates/blob/a6438a2082792e7528bfed682c64e352367f8f6f/puppet/services/neutron-dhcp.yaml#L30 to 1500 | 16:12 |
| rook | that way, neutron will default to 1450 | 16:12 |
| rook | smalleni: mind giving that a try? | 16:12 |
| smalleni | rodrigods, yes i think updating that is a good idea | 16:12 |
| smalleni | not at all | 16:12 |
| smalleni | sorry that was supposed to rook | 16:12 |
| dprince | rook: perhaps, but we also need to be mindful of existing defaults for these things. Like swapping the tenant_mtu on an upgrade, making sure that file gets removed, etc. | 16:13 |
| jayg | EmilienM: there was a 8.0.7 at 18 days, and a 8.0.6 at 22- https://github.com/redhat-openstack/openstack-puppet-modules/releases | 16:13 |
| *** tesseract has quit IRC | 16:13 | |
| dprince | rook: like I'm happy Neutron supports this now, but is there really a rush for us to go and use it as a default ATM | 16:13 |
| smalleni | rook, so redeploy after removing that file on undercloud? | 16:14 |
| dprince | smalleni: you'd need to comment out the file in your overcloud image | 16:14 |
| rook | dprince well right now things are pretty fubar as i mentioned... with the mismatched MTUs | 16:14 |
| rook | dprince you will end up with a guest with at 1400 MTU, but all the veth/tap devices nova creates for the guest have a mtu of 1350 | 16:15 |
| *** zoli is now known as zoli|gone | 16:15 | |
| smalleni | dprince, rook even work the external is also set to 1350 | 16:15 |
| smalleni | worse* | 16:15 |
| rook | smalleni yup. | 16:15 |
| rook | this is just ugly all around :/ | 16:15 |
| *** zoli|gone is now known as zoli_gone-proxy | 16:16 | |
| * rook eats. | 16:16 | |
| dprince | rook: even though some of the code in t-h-t is moving around we haven't changed our defaults for some time | 16:16 |
| *** dprince has quit IRC | 16:17 | |
| smalleni | dprince mind telling me how i would go about changing the overcloud image, i have them built | 16:19 |
| EmilienM | jayg: it might be something in 8.0.7 I'm investigating | 16:19 |
| jayg | EmilienM: ok, let me know if you need anything from me - I wonder why 8.1.0 is not being used though? | 16:19 |
| EmilienM | I'm pretty sure something broke betweek 18 & 31th may | 16:20 |
| EmilienM | I'm investigating all our merges | 16:20 |
| jayg | 8.0.7 is the only one in that range | 16:20 |
| EmilienM | ok I saw the occurence of https://bugs.launchpad.net/tripleo/+bug/1590057 on May 24th | 16:22 |
| openstack | Launchpad bug 1590057 in tripleo "keystoneauth1 500 error on stable/mitaka during pingtest" [Critical,Confirmed] | 16:22 |
| EmilienM | and latest success was on 18th | 16:23 |
| pabelanger | derekh: is there somebody that can do that leg work? | 16:23 |
| *** jaosorior has quit IRC | 16:24 | |
| *** jaosorior has joined #tripleo | 16:25 | |
| *** ifarkas has quit IRC | 16:25 | |
| rook | smalleni: what do you need a new oc image for> | 16:26 |
| *** jaosorior has quit IRC | 16:26 | |
| EmilienM | jayg: I found it | 16:27 |
| EmilienM | jayg: now I need to find the root cause of the issue I just found | 16:28 |
| EmilienM | shardy: fyi ^ | 16:28 |
| EmilienM | auth_url in nova.conf on nonha jobs is wrong | 16:28 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: Update .sh references from openstack-keystone to openstack-core https://review.openstack.org/326631 | 16:28 |
| EmilienM | I'm investigating now | 16:28 |
| jayg | EmilienM: so not opm, phew! | 16:28 |
| openstackgerrit | Steven Hardy proposed openstack-infra/tripleo-ci: Add text option to tripleo-jobs-gerrit.py https://review.openstack.org/326632 | 16:29 |
| EmilienM | jayg: maybe :P | 16:29 |
| *** xinwu has joined #tripleo | 16:29 | |
| shardy | EmilienM: good news, thanks! | 16:29 |
| shardy | FYI I pushed the tripleo-jobs-gerrit hack I started, it confirms the last successful run was may 18th | 16:30 |
| smalleni | rook, dprince mentioned i needed to comment out that file in the overcloud "image" | 16:30 |
| *** dmk0202 has quit IRC | 16:30 | |
| jayg | well, opm just tracks upstream, so if anything, upstream would be broken as wel | 16:30 |
| derekh | pabelanger: no sure, not me at the moment sorry, if you want to try and find out what happening in this specific case you could just try running a patch with "timeout -s 9 90m tripleo-ci/scripts/tripleo.sh --overcloud-deploy" | 16:30 |
| shardy | https://review.openstack.org/#/c/306350/1 was the last passing job I think | 16:30 |
| EmilienM | shardy: I had same results :) | 16:30 |
| EmilienM | shardy: I'm investigating the root cause now | 16:30 |
| shardy | EmilienM: cool, I could have just used the HTML ones I suppose :) | 16:30 |
| derekh | pabelanger: that should at least kill it befor the timeout so you get some logs | 16:30 |
| EmilienM | shardy: can you show me how you did though? :P | 16:31 |
| shardy | ./tripleo-jobs-gerrit.py -b stable/mitaka -j gate-tripleo-ci-f22-nonha -t | 16:31 |
| *** oshvartz has quit IRC | 16:31 | |
| shardy | the -t option is added by my patch | 16:31 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: use the to store playbooks https://review.openstack.org/326633 | 16:31 |
| shardy | tripleo-jobs.txt is generated, looks like: | 16:31 |
| *** oshvartz has joined #tripleo | 16:32 | |
| shardy | http://paste.fedoraproject.org/375909/53171161 | 16:32 |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Add timeout to deploy.sh https://review.openstack.org/326634 | 16:32 |
| pabelanger | derekh: first attempt ^ | 16:33 |
| *** coolsvap has quit IRC | 16:33 | |
| rook | smalleni: i must of missed that | 16:33 |
| *** dtantsur is now known as dtantsur|afk | 16:33 | |
| shardy | EmilienM: I had to add http://paste.fedoraproject.org/375910/65317224/ to my ssh config to make the gerrit query work | 16:34 |
| *** fragatina has joined #tripleo | 16:34 | |
| *** coolsvap has joined #tripleo | 16:34 | |
| *** yamahata has joined #tripleo | 16:34 | |
| *** fragatina has quit IRC | 16:35 | |
| *** fragatina has joined #tripleo | 16:35 | |
| derekh | pabelanger: ack, if you run you centos job on that you should at least be protected from the 2+ hour timeout | 16:35 |
| EmilienM | shardy: which query? | 16:36 |
| *** oshvartz has quit IRC | 16:36 | |
| pabelanger | derekh: yup, testing now | 16:36 |
| * EmilienM brb lunch | 16:37 | |
| *** ebarrera has quit IRC | 16:37 | |
| derekh | pabelanger: gotta go in a minute, but just a FYI, we have a timeout in place fr the overcloud deploy http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_gate_test.sh#n53 | 16:39 |
| derekh | pabelanger: looks like it mightn't be working if we're hitting 2 hours | 16:39 |
| pabelanger | derekh: okay, I will look into what that does | 16:40 |
| *** derekh has quit IRC | 16:40 | |
| *** cwolferh has quit IRC | 16:41 | |
| *** florianf has quit IRC | 16:42 | |
| *** tzumainn has joined #tripleo | 16:46 | |
| *** lucasagomes is now known as lucas-dinner | 16:52 | |
| *** dprince has joined #tripleo | 16:53 | |
| *** paramite is now known as paramite|afk | 16:57 | |
| openstackgerrit | Miles Gould proposed openstack/instack-undercloud: Fix inspection_enable_uefi description https://review.openstack.org/320071 | 16:58 |
| openstackgerrit | Miles Gould proposed openstack/instack-undercloud: Fix inspection_enable_uefi description https://review.openstack.org/324553 | 17:00 |
| openstackgerrit | Miles Gould proposed openstack/instack-undercloud: Add option to enable introspection of UEFI nodes https://review.openstack.org/321118 | 17:00 |
| *** cwolferh has joined #tripleo | 17:02 | |
| openstackgerrit | Merged openstack/tripleo-quickstart: Use the default quickstart dir for hosts and ssh.config https://review.openstack.org/326514 | 17:02 |
| openstackgerrit | Miles Gould proposed openstack/instack-undercloud: Fix inspection_enable_uefi description https://review.openstack.org/324555 | 17:02 |
| openstackgerrit | Miles Gould proposed openstack/instack-undercloud: Add option to enable introspection of UEFI nodes https://review.openstack.org/321119 | 17:02 |
| *** trown is now known as trown|lunch | 17:02 | |
| *** fzdarsky is now known as fzdarsky|afk | 17:04 | |
| openstackgerrit | Miles Gould proposed openstack/python-tripleoclient: Document --initial-state param on baremetal import https://review.openstack.org/321117 | 17:05 |
| openstackgerrit | Miles Gould proposed openstack/python-tripleoclient: Document --initial-state param on baremetal import https://review.openstack.org/321116 | 17:06 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Aodh upgrade heat template and manifests https://review.openstack.org/320150 | 17:07 |
| *** ohamada has quit IRC | 17:13 | |
| rook | dprince: i don't think it is anything tripleo changed, but neutron changed. | 17:14 |
| dprince | rook: okay, it may be we need to adapt a bit, just want to consider the upgrades path too | 17:18 |
| *** anshul_behl has joined #tripleo | 17:18 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Add coordination url to gnocchi https://review.openstack.org/326136 | 17:24 |
| rook | dprince sure, and i personally do not know enough about the upgrade path to say this wont fubar it. | 17:24 |
| rook | dprince this would be my suggestion then... In this template, https://github.com/openstack/tripleo-heat-templates/blob/a6438a2082792e7528bfed682c64e352367f8f6f/puppet/services/neutron-dhcp.yaml#L17 set %MTU% to be %MTU-50% | 17:25 |
| rook | that will fix the mismatched MTU | 17:25 |
| rook | dprince i think smalleni is going to open a bz for us to track this | 17:25 |
| dprince | rook: cool | 17:27 |
| *** tzumainn has quit IRC | 17:30 | |
| *** jpena is now known as jpena|off | 17:30 | |
| *** anshul_behl has quit IRC | 17:33 | |
| *** cwolferh has quit IRC | 17:39 | |
| *** mcornea has quit IRC | 17:39 | |
| *** shardy has quit IRC | 17:41 | |
| *** sambetts is now known as sambetts|afk | 17:44 | |
| *** cwolferh has joined #tripleo | 17:46 | |
| *** fragatina has quit IRC | 17:56 | |
| *** pkovar has quit IRC | 17:58 | |
| *** cwolferh has quit IRC | 17:58 | |
| *** rwsu has quit IRC | 17:58 | |
| *** electrofelix has quit IRC | 18:02 | |
| *** ebalduf_ has joined #tripleo | 18:06 | |
| openstackgerrit | Dan Prince proposed openstack/puppet-tripleo: Include neutron in step 3 https://review.openstack.org/325755 | 18:07 |
| *** mgould is now known as mgould|afk | 18:07 | |
| openstackgerrit | Dan Prince proposed openstack/puppet-tripleo: Include neutron in step 3 https://review.openstack.org/325755 | 18:09 |
| *** trown|lunch is now known as trown | 18:10 | |
| *** cwolferh has joined #tripleo | 18:13 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Add timeout to deploy.sh https://review.openstack.org/326634 | 18:13 |
| smalleni | dprince, rook https://bugs.launchpad.net/tripleo/+bug/1590101 https://bugs.launchpad.net/tripleo/+bug/1590100 | 18:16 |
| openstack | Launchpad bug 1590101 in tripleo "global_physnet_mtu is set to 1400 instead of ethernet standard 1500" [Undecided,New] | 18:16 |
| openstack | Launchpad bug 1590100 in tripleo "Instances do not honor MTU of network in Mitaka as /etc/neutron/dnsmasq.conf enforces MTU" [Undecided,New] | 18:16 |
| pabelanger | how is 'openstack overcloud deploy' install in tripleo-ci? | 18:20 |
| pabelanger | a better question, which packages adds that plugin? | 18:20 |
| *** fultonj has quit IRC | 18:21 | |
| trown | pabelanger: python-tripleoclient | 18:21 |
| dprince | pabelanger: ^^ that along with the magic of python-openstackclient | 18:22 |
| dprince | smalleni: cool, thanks | 18:22 |
| pabelanger | dprince: trown: Thanks, so I think the issue with logs not getting generate on centos-7 failures is the -t flag for openstack overcloud deploy is not working correctly. And trying to see why there would be a difference on fedora and centos | 18:23 |
| dprince | pabelanger: not sure, sorry | 18:32 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Revert "Update management_from_pool template version to 2015-10-15" https://review.openstack.org/326691 | 18:37 |
| *** saneax_AFK is now known as saneax | 18:39 | |
| *** coolsvap has quit IRC | 18:41 | |
| EmilienM | thrash: I might need an heat expert | 18:42 |
| *** smalleni has quit IRC | 18:42 | |
| thrash | EmilienM: what's up? | 18:42 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: use the to store playbooks https://review.openstack.org/326633 | 18:43 |
| EmilienM | thrash: I'm investigating why mitaka CI fails on non-ha job and I figured that some hiera parameters were wrongly computed | 18:43 |
| EmilienM | it has something to do with KeystoneV3Admin | 18:43 |
| EmilienM | I'm trying to revert a patch, see https://review.openstack.org/326691 | 18:43 |
| EmilienM | oh I might have found why it fails | 18:44 |
| thrash | :) | 18:44 |
| EmilienM | thrash: sorry for noise, I just found something that I missed before, I'll send a patch when I'm sure | 18:45 |
| *** xinwu has quit IRC | 18:45 | |
| thrash | EmilienM: ack. Let me know if I can be of further assistance. :D | 18:45 |
| EmilienM | bingo | 18:47 |
| EmilienM | https://review.openstack.org/#/c/284265/ | 18:47 |
| EmilienM | we need this backport | 18:47 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Don't have separate protocols/ports for Keystone v3 https://review.openstack.org/326695 | 18:47 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Aodh upgrade heat template and manifests https://review.openstack.org/320150 | 18:48 |
| *** TSCHAK has quit IRC | 18:48 | |
| EmilienM | jayg: it's not OPM | 18:50 |
| *** xinwu has joined #tripleo | 18:51 | |
| *** xinwu has quit IRC | 18:51 | |
| *** mbound has quit IRC | 18:51 | |
| *** TSCHAK has joined #tripleo | 18:52 | |
| *** fragatina has joined #tripleo | 18:58 | |
| radez | I'm working on adding congress to triple-o, I get overcloud create_complete, but then deploy bails | 18:58 |
| radez | https://paste.fedoraproject.org/375964/65325892/ | 18:58 |
| radez | what is triple-o doing after the create is complete? | 18:59 |
| radez | this is my code: https://github.com/trozet/opnfv-tht/pull/16/commits/eecba6bcb88febc7f96662a6250b9bda30265a7d | 18:59 |
| radez | and I have a couple lines added to tripleo client that generates the passwords | 18:59 |
| jayg | EmilienM: glad to hear it! | 19:01 |
| EmilienM | radez: what puppet module do you use? | 19:02 |
| radez | https://github.com/opnfv/puppet-congress | 19:02 |
| EmilienM | please don't tell me you wrote your own | 19:02 |
| radez | wrote that one | 19:02 |
| EmilienM | .... | 19:02 |
| radez | is there one already:) | 19:02 |
| radez | ? | 19:02 |
| EmilienM | sigh | 19:02 |
| EmilienM | why not reaching Puppet OpenStack community to move the module? | 19:03 |
| EmilienM | http://docs.openstack.org/developer/puppet-openstack-guide/new-module.html | 19:03 |
| radez | it was a pretty cheap copy and paste job trying to get things working | 19:03 |
| EmilienM | well, it would be more than great to work with OpenStack community | 19:03 |
| radez | EmilienM: I think I used those docs | 19:03 |
| radez | oh maybe that's a new doc | 19:04 |
| EmilienM | this is not the way to go radez | 19:04 |
| radez | hm, yea I've not seen that doc before | 19:04 |
| EmilienM | we have a wiki page for 2 years | 19:04 |
| radez | I used a doc that helped generate new code | 19:04 |
| EmilienM | https://wiki.openstack.org/wiki/Puppet/New_module | 19:04 |
| radez | or the skeleton code | 19:05 |
| EmilienM | radez: please move the module to OpenStack | 19:05 |
| EmilienM | and please next time work with us | 19:05 |
| radez | I'll work on getting it moved over | 19:05 |
| EmilienM | radez: I'm a bit sad to be Puppet OpenStack PTL and see Red Hat writting Puppet OpenStack modules on other git repos than OpenStack | 19:05 |
| EmilienM | so please, move it and drop old repo | 19:06 |
| EmilienM | I'm going to send a patch to move it | 19:06 |
| radez | okey doke | 19:06 |
| *** tzumainn has joined #tripleo | 19:16 | |
| thrash | EmilienM: have you looked at https://review.openstack.org/#/c/323431/ ? I separated out the api and engine for mistral pacemaker profiles. Does that look sane? | 19:18 |
| EmilienM | thrash: yes it looks better | 19:19 |
| thrash | EmilienM: cool. | 19:19 |
| *** ramishra has quit IRC | 19:21 | |
| openstackgerrit | Stig Telfer proposed openstack/tripleo-docs: Added documentation for iSER configuration https://review.openstack.org/325129 | 19:28 |
| *** tzumainn has quit IRC | 19:28 | |
| *** diazjf has joined #tripleo | 19:29 | |
| jmiu | anyone willing to help me debug this? the post-deployment configuration on the controller node is failing | 19:30 |
| openstackgerrit | Ryan Hallisey proposed openstack/tripleo-heat-templates: WIP - Ansible driven upgrade https://review.openstack.org/322591 | 19:31 |
| jmiu | it gives me this error code: | 19:31 |
| jmiu | Error: resources.ControllerNodesPostDeployment.resources.ControllerOvercloudServicesDeployment_Step4.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 1 | 19:31 |
| jmiu | this is the environment file i wrote to enable the post-deploy config | 19:31 |
| jmiu | https://paste.fedoraproject.org/375975/27781146/ | 19:31 |
| jmiu | this is the config file itself: | 19:32 |
| jmiu | http://paste.fedoraproject.org/375900/46531394/ | 19:32 |
| openstackgerrit | Stig Telfer proposed openstack/tripleo-heat-templates: Added support for pass-through iSER configuration https://review.openstack.org/324781 | 19:32 |
| jmiu | this is supposed to set the variables needed to enable domains (for Keystone v3) | 19:32 |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart: use the $OPT_WORKDIR to store playbooks https://review.openstack.org/326633 | 19:37 |
| *** pkovar has joined #tripleo | 19:37 | |
| *** diazjf has quit IRC | 19:39 | |
| beagles | jmiu: I'm just heading out, but have you seen http://hardysteven.blogspot.ca/2015/04/debugging-tripleo-heat-templates.html | 19:39 |
| *** anshul_behl has joined #tripleo | 19:40 | |
| openstackgerrit | Matt Young proposed openstack/tripleo-quickstart: Add clean virtualenv option to quickstart https://review.openstack.org/326586 | 19:40 |
| *** diazjf has joined #tripleo | 19:41 | |
| jmiu | beagles: no, i haven't! i was working off the documentation http://docs.openstack.org/developer/tripleo-docs/troubleshooting/troubleshooting-overcloud.html | 19:41 |
| jmiu | beagles: i'll definitely look at this. thank you so much | 19:41 |
| *** pkovar has quit IRC | 19:50 | |
| *** cwolferh has quit IRC | 19:52 | |
| *** mbound has joined #tripleo | 19:52 | |
| openstackgerrit | Merged openstack/tripleo-quickstart: use the $OPT_WORKDIR to store playbooks https://review.openstack.org/326633 | 19:54 |
| *** trown is now known as trown|brb | 19:55 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Add timeout to deploy.sh https://review.openstack.org/326634 | 19:56 |
| *** Goneri has quit IRC | 19:56 | |
| *** fultonj has joined #tripleo | 19:56 | |
| *** mbound has quit IRC | 19:57 | |
| *** trown|brb is now known as trown | 19:59 | |
| *** anshul_behl has quit IRC | 20:02 | |
| *** rwsu has joined #tripleo | 20:04 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: Add clean virtualenv option to quickstart https://review.openstack.org/326586 | 20:07 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: Gate ansible-role-tripleo-* roles https://review.openstack.org/326037 | 20:07 |
| *** jayg is now known as jayg|g0n3 | 20:09 | |
| *** cwolferh has joined #tripleo | 20:11 | |
| *** diazjf has quit IRC | 20:13 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Add timeout to deploy.sh https://review.openstack.org/326634 | 20:14 |
| *** diazjf has joined #tripleo | 20:18 | |
| *** diazjf has quit IRC | 20:28 | |
| *** aufi has joined #tripleo | 20:37 | |
| *** xinwu has joined #tripleo | 20:41 | |
| *** lblanchard has quit IRC | 20:44 | |
| *** ayoung has quit IRC | 20:44 | |
| *** jprovazn has quit IRC | 20:45 | |
| openstackgerrit | Ryan Hallisey proposed openstack/tripleo-heat-templates: Ansible driven upgrade https://review.openstack.org/322591 | 20:47 |
| *** rhallisey has quit IRC | 20:49 | |
| *** dprince has quit IRC | 20:55 | |
| *** saneax is now known as saneax_AFK | 20:59 | |
| *** aufi has quit IRC | 21:02 | |
| *** ayoung has joined #tripleo | 21:04 | |
| ayoung | jmiu, any luck? | 21:05 |
| *** trown is now known as trown|outtypewww | 21:05 | |
| *** panda is now known as panda|Zz | 21:06 | |
| *** diazjf has joined #tripleo | 21:06 | |
| *** mgarciam has quit IRC | 21:06 | |
| openstackgerrit | Merged openstack/tripleo-image-elements: adding libffi-dev to required packages https://review.openstack.org/312336 | 21:07 |
| *** Goneri has joined #tripleo | 21:07 | |
| *** ebalduf_ has quit IRC | 21:10 | |
| *** ccamacho has quit IRC | 21:11 | |
| *** yamahata has quit IRC | 21:13 | |
| *** tzumainn has joined #tripleo | 21:13 | |
| *** yamahata has joined #tripleo | 21:15 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Remove usage of ::nova class in THT https://review.openstack.org/325983 | 21:22 |
| jmiu | ayoung: running another deploy that i'm starting to wonder is dead or not | 21:22 |
| jmiu | ayoung: oh, it just died on me. :( | 21:22 |
| ayoung | jmiu, moment of silence.... | 21:23 |
| ayoung | what error? | 21:23 |
| openstackgerrit | Merged openstack/tripleo-puppet-elements: Fix puppet modules in mixed src/pkg environments https://review.openstack.org/285860 | 21:25 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Enable nova-api as a composable service https://review.openstack.org/326020 | 21:25 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Enable nova-scheduler as a composable service https://review.openstack.org/326096 | 21:25 |
| openstackgerrit | Merged openstack/tripleo-image-elements: making ansible version configurable https://review.openstack.org/312342 | 21:25 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Enable nova-consoleauth as a composable service https://review.openstack.org/326118 | 21:25 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Enable nova-vncproxy as a composable service https://review.openstack.org/326132 | 21:25 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Remove nova::db classes, moved to puppet-tripleo https://review.openstack.org/326028 | 21:26 |
| *** diazjf has quit IRC | 21:33 | |
| *** tzumainn has quit IRC | 21:34 | |
| *** jcoufal has quit IRC | 21:34 | |
| jmiu | ayoung: "status_reason": "deploy_status_code : Deployment exited with non-zero status code: 1", | 21:34 |
| jmiu | let me put up a paste with the full error message | 21:34 |
| jmiu | ayoung: http://paste.fedoraproject.org/376014/14653353/ | 21:35 |
| *** diazjf has joined #tripleo | 21:36 | |
| ayoung | jmiu, OK, steve Hardy's blog post is he best docs we have for debugging | 21:36 |
| jmiu | http://docs.openstack.org/developer/tripleo-docs/troubleshooting/troubleshooting-overcloud.html was also kinda helpful | 21:37 |
| *** rhallisey has joined #tripleo | 21:37 | |
| ayoung | heat stack-list --show-nested -f "status=FAILED" | 21:37 |
| jmiu | but i found steve hardy's post to more pertinent | 21:37 |
| jmiu | ^be | 21:37 |
| ayoung | Error: {\"/etc/keystone/domains\"=>nil} is not an absolute path. at /etc/puppet/modules/keystone/manifests/init.pp:1007 on node overcloud-controller-0.localdomain\u001b[0m\n", | 21:37 |
| jmiu | yup. | 21:38 |
| ayoung | EmilienM, we have a puppet question for you. jmiu is trying to use the Keystone puppet module, and geting that error ^^ | 21:38 |
| EmilienM | let me investigate | 21:40 |
| ayoung | I wonder if that is puppet or Heat complaining | 21:40 |
| ayoung | EmilienM, thanks. jmiu had links to what she used to call it | 21:41 |
| ayoung | http://paste.fedoraproject.org/375900/46531394/ and | 21:41 |
| ayoung | EmilienM, this one https://paste.fedoraproject.org/375975/27781146/ | 21:42 |
| EmilienM | ah | 21:42 |
| EmilienM | domain_config_directory: '/etc/keystone/domains/' | 21:42 |
| EmilienM | try again without the last / | 21:42 |
| EmilienM | domain_config_directory: /etc/keystone/domains | 21:42 |
| jmiu | hang on, let me try that | 21:43 |
| EmilienM | it might be a bug in puppetlabs-stdlib/lib/puppet/parser/functions/validate_absolute_path.rb | 21:43 |
| * EmilienM stop moving | 21:43 | |
| jmiu | i actually don't have that in the environment config file | 21:45 |
| jmiu | it is '/etc/keystone/domains' indeed | 21:45 |
| EmilienM | mhhh ok let me continue to look | 21:46 |
| jmiu | i removed it sometime ago and didn't update the paste | 21:46 |
| EmilienM | well, show me your env then | 21:46 |
| *** rasca has quit IRC | 21:46 | |
| jmiu | sure | 21:46 |
| EmilienM | the one you actually have | 21:46 |
| *** morazi has quit IRC | 21:46 | |
| jmiu | here is my paste | 21:47 |
| jmiu | not much different | 21:47 |
| jmiu | http://paste.fedoraproject.org/376018/33601414/ | 21:47 |
| ayoung | EmilienM, is there a faster way to test this other than a full redeploy? | 21:48 |
| EmilienM | run puppet manually | 21:49 |
| EmilienM | that's weird, the function just check if the path is absolute | 21:49 |
| EmilienM | which is the case | 21:49 |
| ayoung | EmilienM, yeah...an feed it the Hiera data somehow... | 21:49 |
| EmilienM | jmiu: can I see keystone.conf? | 21:50 |
| jmiu | yes | 21:53 |
| *** ayoung has quit IRC | 21:54 | |
| jmiu | the entire thing, or just the part i've edited? | 21:54 |
| EmilienM | jmiu: the thing generated by Puppet | 21:54 |
| jmiu | http://paste.fedoraproject.org/376020/36618146/ | 21:57 |
| jmiu | i've made some edits to it | 21:57 |
| jmiu | basically doing the same thing that puppet was supposed to do, but i can comment it out | 21:57 |
| jmiu | wait hang on you said the one generated by puppet. i just gave you the one for undercloud, not overcloud | 21:59 |
| jmiu | this is the right keystone.conf | 22:03 |
| jmiu | https://paste.fedoraproject.org/376022/36985146/ | 22:03 |
| jmiu | got to grab the bus, be online in ~2 hours | 22:03 |
| *** aufi has joined #tripleo | 22:05 | |
| *** diazjf has quit IRC | 22:06 | |
| EmilienM | the error you have is weird | 22:07 |
| EmilienM | sounds like the stdlib function does not validate the path | 22:07 |
| EmilienM | you might have found a bug | 22:07 |
| EmilienM | we have unit tests for it, weird bis | 22:08 |
| *** weshay has quit IRC | 22:08 | |
| *** ayoung has joined #tripleo | 22:08 | |
| EmilienM | ayoung: like I just said, something weird is happenning | 22:09 |
| ayoung | EmilienM, maybe because the directory does not exist? | 22:10 |
| EmilienM | no | 22:10 |
| EmilienM | let me show you the code a sec | 22:10 |
| EmilienM | https://github.com/puppetlabs/puppetlabs-stdlib/blob/master/lib/puppet/parser/functions/validate_absolute_path.rb | 22:11 |
| EmilienM | looking at what does function_is_absolute_path now | 22:11 |
| EmilienM | ayoung: I think I found the code | 22:15 |
| EmilienM | ayoung: https://github.com/puppetlabs/puppet/blob/98d22be27447bcb34c33aa4a44610ab8768390f5/lib/puppet/util.rb#L267 | 22:16 |
| EmilienM | puppet actually tests if the file does exist :) | 22:16 |
| EmilienM | I'll send some doc in puppet-keystone to make sure others find out quickly next time | 22:16 |
| EmilienM | ayoung: or better, drop the validation | 22:18 |
| EmilienM | it's very risky actually | 22:18 |
| *** rasca has joined #tripleo | 22:19 | |
| EmilienM | ayoung: well in fact the function just run a regex on the path but does not test the path directory I was wrong | 22:23 |
| *** akrivoka has quit IRC | 22:25 | |
| *** aufi has quit IRC | 22:31 | |
| ayoung | jmiu, ** was really for you... | 22:34 |
| EmilienM | :) | 22:34 |
| EmilienM | ayoung: let's continue debug tomorrow | 22:35 |
| ayoung | EmilienM, sure, just make sure you ping jmiu with answers, as she is the one battling this directly | 22:35 |
| EmilienM | ayoung: sure | 22:36 |
| ayoung | jmiu, so...drop that value for now. | 22:45 |
| ayoung | We can come back to the path value, but the config file will use the default, so there is no burning need to specify it | 22:45 |
| ayoung | and long term, we don't even want to use the domain specific config in the files. We'd rather use the one from the database, so this is not an essential change. | 22:46 |
| pabelanger | so https://review.openstack.org/#/c/326634/ restores the timeout ability to openstack overcloud deploy but for what ever reason undercloud.tar.xz is empty | 22:51 |
| pabelanger | any pointers why that is | 22:51 |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Don't skip errors in postci https://review.openstack.org/326759 | 22:57 |
| *** cwolferh has quit IRC | 22:58 | |
| EmilienM | pabelanger: yes ^ I was about proposing the same too | 22:59 |
| EmilienM | for the undercloud, there must be something similar | 22:59 |
| pabelanger | EmilienM: Ya, it is looking like the centos-7 failures are not related to centos-7 but inconsistent testing | 23:00 |
| openstackgerrit | Ryan Hallisey proposed openstack/tripleo-heat-templates: Ansible driven upgrade https://review.openstack.org/322591 | 23:01 |
| *** Goneri has quit IRC | 23:02 | |
| *** cwolferh has joined #tripleo | 23:02 | |
| EmilienM | pabelanger: I can't find why undercloud archive is not correctly generated | 23:02 |
| pabelanger | EmilienM: I think the ssh command is failing | 23:04 |
| EmilienM | pabelanger: http://logs.openstack.org/34/326634/4/experimental-tripleo/gate-tripleo-ci-centos-7-ha/54e614f/logs/postci.txt.gz#_2016-06-07_22_30_06_000 | 23:04 |
| pabelanger | EmilienM: better solution would be to ssh into undercloud, generate logs, and wait for devstack-gate to pull them using ansible | 23:04 |
| *** absubram has quit IRC | 23:04 | |
| pabelanger | today it is trying to pipe logs to tarball over SSH | 23:05 |
| *** saneax_AFK is now known as saneax | 23:22 | |
| *** tosky has quit IRC | 23:27 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: Enabled pacemaker by default in minimal deployment https://review.openstack.org/323878 | 23:27 |
| *** cwolferh has quit IRC | 23:29 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Remove loadbalancer profile https://review.openstack.org/325893 | 23:41 |
| *** cwolferh has joined #tripleo | 23:43 | |
| *** panda|Zz has quit IRC | 23:45 | |
| *** panda|Zz has joined #tripleo | 23:46 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Import ::nova class from THT https://review.openstack.org/325982 | 23:49 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Implement Nova API profile https://review.openstack.org/325937 | 23:49 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Implement Nova Scheduler profile https://review.openstack.org/326092 | 23:49 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Implement Nova Consoleauth profile https://review.openstack.org/326112 | 23:49 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Implement Nova vncproxy profile https://review.openstack.org/326130 | 23:49 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Deploy nova::db classes in api profile https://review.openstack.org/326027 | 23:52 |
| *** dciabrin has quit IRC | 23:56 | |
| *** saneax is now known as saneax_AFK | 23:58 | |
| *** rohitpagedar__ has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
