Wednesday, 2017-11-15

« Tuesday, 2017-11-14 Index Thursday, 2017-11-16 »
openstackgerritAlan Pevec proposed openstack/tripleo-quickstart-extras master: Avoid github mirror  https://review.openstack.org/51983300:03
*** artom has joined #tripleo00:03
*** adarazs|ruck has quit IRC00:09
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432800:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103200:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106300:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)00:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]00:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]00:10
*** adarazs has joined #tripleo00:11
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Add version and git sha to debug modal  https://review.openstack.org/51983500:12
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Add version and git sha to debug modal  https://review.openstack.org/51983500:14
*** Vijayendra__ has joined #tripleo00:18
*** Vijayendra_ has quit IRC00:20
*** ramishra has joined #tripleo00:22
*** bfournie has joined #tripleo00:27
*** bfournie has quit IRC00:28
*** raildo has quit IRC00:45
*** zshi has joined #tripleo00:45
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements  https://review.openstack.org/51918900:48
*** zshi has quit IRC00:51
*** tosky has quit IRC00:51
*** zshi_ has joined #tripleo01:04
*** rlandy|rover|afk is now known as rlandy|rover01:09
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432801:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103201:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106301:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)01:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]01:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]01:10
*** ecerquei has joined #tripleo01:11
*** myoung|alt has joined #tripleo01:12
*** cshastri has joined #tripleo01:13
*** myoung|bbl has quit IRC01:16
*** dsneddon has quit IRC01:16
*** myoung|alt has quit IRC01:17
*** rhallisey has quit IRC01:22
*** arxcruz__ has quit IRC01:30
*** myoung has joined #tripleo01:34
*** akrivoka has quit IRC01:34
*** absubram has quit IRC01:37
*** psahoo has joined #tripleo01:44
*** psahoo has quit IRC01:44
openstackgerritzhongshengping proposed openstack/puppet-tripleo stable/newton: Switch to Zuul v3 testing  https://review.openstack.org/51989501:49
openstackgerritzhongshengping proposed openstack/puppet-tripleo stable/ocata: Switch to Zuul v3 testing  https://review.openstack.org/51989701:49
openstackgerritzhongshengping proposed openstack/puppet-tripleo stable/pike: Switch to Zuul v3 testing  https://review.openstack.org/51989801:49
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs  https://review.openstack.org/51479301:49
EmilienMweshay: see the refacto ^01:50
EmilienMit's not finished yet but good progress01:50
*** yamahata has quit IRC01:50
*** links has joined #tripleo02:03
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432802:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103202:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106302:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)02:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]02:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]02:10
*** itlinux has joined #tripleo02:12
*** myoung has quit IRC02:13
*** itlinux has quit IRC02:17
openstackgerritKeith Schincke proposed openstack/puppet-tripleo master: Update cephx keys with ACLs for openstack services.  https://review.openstack.org/51953102:24
*** dougbtv_ has joined #tripleo02:34
*** jmelvin has quit IRC02:43
*** myoung has joined #tripleo02:50
openstackgerritSteve Baker proposed openstack/tripleo-docs master: Document use of 'tag discover' command  https://review.openstack.org/51995302:55
*** bfournie has joined #tripleo02:55
*** fzdarsky_ has joined #tripleo02:55
*** fzdarsky|afk has quit IRC02:57
*** rlandy|rover has quit IRC02:57
-openstackstatus- NOTICE: Due to an unexpected outage with Zuul (1 hour), you'll need to recheck any jobs that were in progress. Sorry for the inconvenience.02:58
*** fragatina has quit IRC03:10
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432803:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103203:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106303:10
*** ooolpbot has quit IRC03:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]03:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]03:10
*** gvrangan has joined #tripleo03:11
*** artom has quit IRC03:12
*** artom has joined #tripleo03:12
*** fragatina has joined #tripleo03:13
*** japestinho has joined #tripleo03:13
*** itlinux has joined #tripleo03:15
*** fragatina has quit IRC03:17
*** gbarros has quit IRC03:24
*** atoth has quit IRC03:24
*** psahoo has joined #tripleo03:34
*** rcernin has quit IRC03:34
*** rcernin has joined #tripleo03:36
*** mdnadeem has joined #tripleo03:39
*** artom has quit IRC03:39
*** udesale has joined #tripleo03:41
*** gvrangan has quit IRC03:52
*** gvrangan has joined #tripleo03:52
*** yamahata has joined #tripleo03:57
*** gvrangan has quit IRC04:00
*** dougbtv_ has quit IRC04:04
*** myoung is now known as myoung|hex04:05
*** myoung has joined #tripleo04:07
*** aditya_r has joined #tripleo04:09
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432804:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103204:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106304:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)04:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]04:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]04:10
*** anilvenkata has joined #tripleo04:11
openstackgerritRabi Mishra proposed openstack/tripleo-common master: Raise stored HTTPError when fetching swift object  https://review.openstack.org/51996204:12
*** ykarel has joined #tripleo04:13
*** myoung|hex has quit IRC04:21
*** dbecker has quit IRC04:24
*** psachin has joined #tripleo04:28
*** gvrangan has joined #tripleo04:29
*** etingof has quit IRC04:35
*** dbecker has joined #tripleo04:37
*** masco has joined #tripleo04:38
*** skramaja has joined #tripleo04:43
*** abregman|afk has quit IRC04:48
*** etingof has joined #tripleo04:52
*** pgadiya has joined #tripleo05:00
*** aditya_r has quit IRC05:01
*** aditya_r has joined #tripleo05:01
*** shreshtha has joined #tripleo05:04
*** shreshtha has quit IRC05:04
*** shreshtha has joined #tripleo05:04
*** pdeore has joined #tripleo05:06
*** aditya_r has quit IRC05:07
*** aditya_r has joined #tripleo05:07
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432805:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103205:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106305:10
*** ooolpbot has quit IRC05:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)05:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]05:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]05:10
*** dmacpher has joined #tripleo05:11
*** aditya_r has quit IRC05:12
*** jaganathan has joined #tripleo05:13
*** akane has joined #tripleo05:16
*** jaosorior has quit IRC05:24
*** itlinux has quit IRC05:27
*** aditya_r has joined #tripleo05:28
*** cmyster has joined #tripleo05:29
*** cmyster has quit IRC05:29
*** cmyster has joined #tripleo05:29
*** aditya_r has quit IRC05:31
*** tzumainn has quit IRC05:31
*** aditya_r has joined #tripleo05:31
*** janki has joined #tripleo05:33
*** janki has quit IRC05:40
*** moshele has joined #tripleo05:43
Tenguhello there!05:48
*** yprokule has joined #tripleo05:57
*** aditya_r has quit IRC06:00
*** aditya_r has joined #tripleo06:00
*** dparkes has quit IRC06:06
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs  https://review.openstack.org/51479306:10
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432806:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103206:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106306:10
*** ooolpbot has quit IRC06:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)06:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]06:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]06:10
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: CI: remove useless templates  https://review.openstack.org/51842306:11
*** janki has joined #tripleo06:12
*** gvrangan has quit IRC06:15
openstackgerritJuan Badia Payno proposed openstack/tripleo-heat-templates master: logging: merge fluentd-client and fluentd-base  https://review.openstack.org/50145706:18
*** nyechiel has joined #tripleo06:18
*** paramite has joined #tripleo06:19
*** gvrangan has joined #tripleo06:19
*** jprovazn has joined #tripleo06:21
*** daidv has joined #tripleo06:26
*** suuuper has joined #tripleo06:32
*** threestrands has quit IRC06:34
*** jfrancoa has joined #tripleo06:40
*** dmacpher has quit IRC06:46
*** masco has quit IRC06:52
*** ramishra has quit IRC06:53
*** moshele has quit IRC06:53
*** moshele has joined #tripleo06:55
*** ramishra has joined #tripleo06:56
*** jaosorior has joined #tripleo06:57
*** abregman has joined #tripleo06:57
*** dparkes has joined #tripleo06:57
openstackgerritmathieu bultel proposed openstack/instack-undercloud master: Install ansible during the upgrade/update in the UC  https://review.openstack.org/51966306:58
*** marios has joined #tripleo06:59
*** agurenko has joined #tripleo07:00
*** agurenko has quit IRC07:01
openstackgerritmathieu bultel proposed openstack/instack-undercloud master: Install ansible during the upgrade/update in the UC  https://review.openstack.org/51966307:02
*** masco has joined #tripleo07:04
*** dtrainor has quit IRC07:04
*** agurenko has joined #tripleo07:05
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432807:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103207:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106307:10
*** ooolpbot has quit IRC07:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]07:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]07:10
*** dtantsur|afk is now known as dtantsur07:11
openstackgerritmathieu bultel proposed openstack/instack-undercloud master: Install ansible-pacemaker during the upgrade/update in the UC  https://review.openstack.org/51966307:13
jaosoriormatbu_: thanks07:13
matbu_jaosorior: thank you for the review ;)07:13
*** spectr has joined #tripleo07:13
jaosoriormatbu_: while I'm alright with that change. Is there a reason why it wasn't added to instack-undercloud's package dependencies?07:14
matbu_jaosorior: well we can add the dependencies there, but in case of upgrade (already pike undercloud installed) we can miss this dependencies07:15
*** aditya_r has quit IRC07:16
matbu_jaosorior: but thats probably a better solution for long term07:16
*** aditya_r has joined #tripleo07:16
*** aufi has joined #tripleo07:17
Tenguhello! small question: in a 3-controllers setup with HA and so on, are we able to *replace* a controller?07:17
Tenguas apparently it's not possible to add a new one, I'm wondering about the replacement.07:18
jaosoriorTengu: shardy might know. Gotta ask him when he's online07:19
*** rcernin has quit IRC07:20
Tengujaosorior: ok, I'll do that then :).07:20
*** mdnadeem has quit IRC07:23
Tenguhmmm. I see puppet is provided by the delorea-deps repository. Any thought about consolidating the (in)famous "puppet-agent" from puppetlabs? That would allow to get a newer version, and for people wanting to get a hand on their nodes with some external puppet-server, to ensure a full compatibility :).07:24
Tengu(although I know tripleo is moving away from puppet)07:24
jaosoriorRight. I don't think folks have given much thought about it; given there are many efforts to move more towards ansible.07:26
Tenguhmm ok. I'll check if I can make both version live on the same server.07:26
TenguI think it should, as the puppet-agent package push all its content (executables, libs and so on) in /opt/puppetlabs and /etc/puppetlabs07:27
TenguI suspect we have a small compatibility issue.07:27
jaosoriorwhich?07:27
TenguSNI support, maybe, don't really know.07:28
Tenguwe have a puppet-server running on latest 4.x version from puppetlabs, with some TLS endpoint that needs SNI support. "fun" fact, the only nodes that are reporting SSL issues are the openstack ones.07:29
jaosoriorwha07:29
Tenguthe puppet version installed by delorean-deps is an "old" 4.8.2, while our standard agent is 4.10.607:29
jaosorioroh07:29
Tenguwe thought the 4.8.2 had full SNI support07:29
Tengubut seeing the errors, I'm not that sure…07:30
jaosoriorTengu: might not be related to puppet itself; but the ruby dependencies.07:30
TenguI'll create some VM in order to get the delorean puppet version and add the puppetlabs one. All paths are dedicated, and only the delorean-deps version adds the "puppet" executable in the system path.07:30
Tengujaosorior: maybe, yes. that's why puppetlabs bundles ALL deps in their rpm.07:31
jaosoriorI didn't know07:31
Tengujaosorior: all is installed in /opt/puppetlabs, and the libs are also there, in /opt/puppetlabs/lib or something like that.07:31
Tenguand the puppet exec is in /opt/puppetlabs/bin07:31
jaosoriorI see07:31
*** cshastri has quit IRC07:32
Tengumeaning we might make both version live together without harm.07:32
Tengubut I'd rather ensure it's the case ;). As for now puppet is still a key component of tripleO deploy.07:32
*** oidgar has joined #tripleo07:32
jaosoriormight, if the path bits work like they should07:32
Tenguhmm. now that I said the path thing… I wonder if puppetlabs doesn't add some "environment" file that adds the /opt/puppetlabs/bin directory to $PATH07:33
*** mdnadeem has joined #tripleo07:33
Tenguthat would be nasty and might override the other one if they add that directory to the beginning of the $PATH.07:33
*** ykarel is now known as ykarel|lunch07:33
Tenguchecked. at least on Debian, PATH is /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/puppetlabs/bin07:33
Tengumeaning a call to `puppet' will call the delorean-deps one.07:34
Tenguthat might actually work. hmm.07:34
*** gvrangan has quit IRC07:34
*** gvrangan has joined #tripleo07:35
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-quickstart master: Make FreeIPA supplemental node DNS server configureable  https://review.openstack.org/49589907:35
jaosoriorTengu: but then that would explain why the SNI doesn't work. It might be using the lib coming from delorean.07:38
*** cshastri has joined #tripleo07:38
*** spectr has quit IRC07:38
Tengujaosorior: according to a colleague, it's more that probable the ruby http lib is the issue.07:38
Tengualthough it's a ruby 2.007:38
jaosoriorthat makes sense07:39
*** SlickNik has joined #tripleo07:40
Tenguguess it's in the ruby-libs-2.0.0.648-30.el7.x86_64 package.07:41
Tenguas it provides /usr/share/ruby/net/http07:42
*** marios_ has joined #tripleo07:42
Tenguand https, of course.07:42
Tenguah, might also be the ruby "openssl" lib in fact.07:42
*** marios has quit IRC07:43
*** dpawar has joined #tripleo07:43
Tenguhmm. ok puppetlabs is based on ruby 2.1 apparently.07:44
Tenguat least on debian stretch07:44
*** marios_ is now known as marios07:45
Tenguand not based on the system's ruby. ook. so isolated.07:45
Tenguhaving both version might actually work better than expected.07:45
jaosoriorwonder if it's the same thing in CentOS07:45
Tengujust downloaded a cloud image, will start it in virtualbox shortly.07:45
*** ccamacho has joined #tripleo07:48
mosheleskramaja: hi07:49
*** marios has quit IRC07:50
*** pcaruana has joined #tripleo07:51
openstackgerritmathieu bultel proposed openstack/python-tripleoclient master: Implement Upgrade workflow via the tripleo cli  https://review.openstack.org/46372807:52
*** pdeore has quit IRC07:53
*** marios has joined #tripleo07:54
*** cylopez has joined #tripleo07:57
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-quickstart master: Make FreeIPA supplemental node DNS server configureable  https://review.openstack.org/49589907:57
openstackgerritmathieu bultel proposed openstack/python-tripleoclient master: Fix static-inventory option for minor update  https://review.openstack.org/51497507:58
*** bogdando has joined #tripleo07:58
*** fzdarsky_ is now known as fzdarsky07:58
Tengujaosorior: -.- wondering why I'm trying to run a qcow2 in virtualbox when I have an openstack where I can start whatever thing I want :D07:59
*** mmedvede has quit IRC07:59
*** mandre has quit IRC07:59
*** pdeore has joined #tripleo08:00
jaosoriorlol08:00
*** PagliaccisCloud has quit IRC08:01
*** gvrangan has quit IRC08:01
*** ebarrera has joined #tripleo08:03
apetrichjtomasek, btw I have the fix we agreed for https://review.openstack.org/#/c/511249/ but I'm waiting for the fix for the tests to pass on master so I can rebase on that08:03
jaosoriormarios: hey, could you check this out https://review.openstack.org/#/c/519256/ ?08:04
Tenguduh08:04
Tengujaosorior: can't work in the end: the puppet-agent package will replace facter.08:04
*** mandre has joined #tripleo08:05
*** Tripleo-User has joined #tripleo08:05
*** PagliaccisCloud has joined #tripleo08:05
Tengujaosorior: ah, and also: file /usr/lib/systemd/system/puppet.service from install of puppet-agent-1.10.7-1.el7.x86_64 conflicts with file from package puppet-4.8.2-1.el7.noarch08:05
jaosorioraaah bummer08:05
Tenguthat one is a show-stopper unfortunately.08:05
*** mmedvede has joined #tripleo08:06
Tenguthat said… IF tripleO receipts call puppet like "puppet apply <catalog>" without full path, it might still work.08:06
Tenguhmm. """might""", with a lot of emphasis.08:06
Tripleo-Userhi08:07
*** dtrainor has joined #tripleo08:07
Tripleo-Userpike 容器08:07
Tripleo-Userpike overcloud Containerized  deployment08:08
skramajamoshele: hi08:08
Tengujaosorior: fun part though: apparently, three nodes have constant failure due to the ssl (one controller, and two ceph-storage). others are transcient… I'm like "wtf" right now.08:09
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432808:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103208:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106308:10
*** ooolpbot has quit IRC08:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]08:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]08:10
jaosoriorTripleo-User: Are you having issues with the containerized deployment?08:10
Tripleo-Useryes08:10
*** ebarrera has quit IRC08:10
jaosoriorTripleo-User: what issue? Can you post logs on what you're seeing?08:10
*** ebarrera has joined #tripleo08:11
Tripleo-Userjaosorior:and you ?08:12
jtomasekapetrich: sounds good, thanks!08:12
Tripleo-User@jaosorior08:12
jaosoriorTripleo-User: what's the error?08:13
jaosoriorbogdando: hey, could you check this out https://review.openstack.org/#/c/519256/ ?08:14
mosheleskramaja: pci passthrough is broken due to this change https://github.com/openstack/puppet-nova/commit/d845b6f0aefaaab0b65b32cd22ccda8c2c8ee6ae#diff-bd867f545f9fb9f386289955e89cd74008:14
*** suuuper has quit IRC08:14
mosheleskramaja: this should be updated https://github.com/openstack/tripleo-heat-templates/blob/5f772c062596ac78ffb743e87e589b7eb263906d/puppet/services/nova-compute.yaml ? or we should revert the removal?08:14
jaosoriormarios: thanks! This is the t-h-t patch https://review.openstack.org/519361   and this other one actually uses it https://review.openstack.org/#/c/519362/08:15
Tripleo-Userdocker service can't start08:15
openstackgerritCarlos Camacho proposed openstack/tripleo-quickstart-extras master: Include 'queens' as target version for upgrade.  https://review.openstack.org/51664708:15
jaosoriorTripleo-User: what error does it output? is that an error in the undercloud or overcloud?08:16
bogdandojaosorior:  why do you drop imudp? it uses a local socket and we don't need the modules, right?08:16
jaosoriorTripleo-User: what command did you use for the deployment?08:16
bogdandoit communicates*08:16
jaosoriorbogdando: right, so  I actually gave it a go with the imudp module. Without net=host, and it was unable to bind for some reason08:17
*** tesseract has joined #tripleo08:17
Tripleo-Useropenstack overcloud deploy --templates /home/stack/openstack-tripleo-heat-templates -e /usr/share/openstack-tripleo-heat-templates/environments/docker.yaml -e ~/docker_registry.yaml08:17
jaosoriorbogdando: and it just plain refused to work with it unless I would start using net=host; which I don't want to do and pretty much defeats the purpose.08:17
skramajamoshele: i think we need to modify nova-compute to the new hiera mapping08:17
jaosoriorbogdando: so, I opted for dropping it. Either way, we do expose a unix socket, so that should be enough.08:18
mosheleskramaja: no problem I will push a fix soon08:18
bogdandojaosorior: sounds totally fair08:18
jaosoriorTripleo-User: what's the error output that you get?08:19
*** agurenko has quit IRC08:19
*** ykarel|lunch is now known as ykarel08:19
skramajaok thanks moshele.08:19
Tripleo-Userwhen i exec /usr/libexec/heat-config/hooks/ansible < /var/lib/heat-config/deployed/e0c22e2e-8348-4477-93e8-bfdae3df6e19.json  errors:08:20
Tripleo-User"2017-11-15 08:18:23,319 INFO: 311393 -- Pulling image: 172.24.27.2:8787/tripleoupstream/centos-binary-rabbitmq:latest",          "2017-11-15 08:18:26,742 ERROR: 311395 -- Failed running docker-puppet.py for horizon",          "2017-11-15 08:18:26,743 ERROR: 311395 -- + mkdir -p /etc/puppet",          "+ cp -a /tmp/puppet-etc/auth.conf /tmp/puppet-etc/hiera.yaml /tmp/puppet-etc/hieradata /tmp/puppet-etc/modules /tmp/puppet-etc/08:20
jaosorioruhm08:21
*** ebarrera has quit IRC08:21
jaosoriorTripleo-User: can you try doing08:21
jaosoriorTripleo-User: openstack stack failures list --long overcloud08:21
skramajamoshele: it affects queens only !! i was cross checking if we need to backport to pike, but puppet-neutron change is added in queens only.08:21
mariosjaosorior: ack08:22
*** ebarrera has joined #tripleo08:22
TenguTripleo-User: oh and, please, use http://paste.openstack.org/ instead of copying the output here :)08:25
Tripleo-UserOK08:26
*** ffiore has joined #tripleo08:27
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: test unbound config and log  https://review.openstack.org/52000708:27
jaosoriormarios: I could add a release note. Though I'm hoping to merge this first https://review.openstack.org/#/c/515282/ . That enables the rsyslog sidecar container image with the HAProxy and swift services. But here's the thing, we MIGHT start including that sidecar always...08:29
*** aditya_r has quit IRC08:29
*** aditya_r has joined #tripleo08:30
jaosoriormarios: So, for logging to files, swift currently relies on an rsyslog rule that comes with the package. This is how https://review.openstack.org/#/c/519577/ actually works currently. However, as we're trying to move away from having the packages in overcloud-full and rely more in the containers. I might just use the rsyslog sidecar image to redirect the output of swift to a file.08:30
jaosoriormarios: so the commit I linked to is meant to be backported to stable/pike. Then on top of that, I will start experimenting with making swift stop using the hosts' rsyslog08:31
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-quickstart master: Make FreeIPA supplemental node DNS server configureable  https://review.openstack.org/49589908:32
jaosoriorTripleo-User: what was the output of: openstack stack failures list --long overcloud08:33
*** hjensas has joined #tripleo08:33
*** hjensas has quit IRC08:33
*** hjensas has joined #tripleo08:33
mariosjaosorior: thanks for the extra context i've added the swift review to check later too (and re the release note yeah it isn't really an interface per se so no strong pushback hence I +2 already there)08:34
jaosoriormarios: but yeah, the summary is: Logging is a mess :D08:35
*** arxcruz__ has joined #tripleo08:35
jaosoriormarios: need reviews?08:35
mariosjaosorior: sure, this one https://review.openstack.org/510902 (its scary but repetition, just if you spot something in 2 mins thanks)08:38
mariosjaosorior: you already checked that one https://review.openstack.org/#/c/518328/08:39
*** amoralej|off is now known as amoralej08:40
jaosoriormarios: 518328 will keep failing until https://review.openstack.org/#/c/516136/ merges08:42
jaosoriormarios: so you might want to rebase on top of that08:42
mariosjaosorior: thanks08:43
jbadiapa_jaosorior, could you take a look at https://review.openstack.org/#/c/501457/?08:43
*** jbadiapa_ is now known as jbadiapa08:43
jbadiapajaosorior, I do not know why is not falling the job legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-container08:44
jaosoriorjbadiapa: looks fairly straight forward. You tried it out already, right? planning to backport it?08:45
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Layout components updates  https://review.openstack.org/51636108:45
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Modals with react-bootstrap  https://review.openstack.org/51636208:45
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Add/Update modal-related dependencies  https://review.openstack.org/51636308:45
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Rework ModalPanel to be based of react-overlays  https://review.openstack.org/51636408:45
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: ModalPanel vs Validations sidebar positioning  https://review.openstack.org/51636508:45
jaosoriorjbadiapa: failing? last I see from zuul it passed that job.08:45
jbadiapayes, that's why i'm asking, there is no fluentd container on the http://logs.openstack.org/57/501457/8/check/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-container/b8d13c3/logs/undercloud/home/zuul/containers-default-parameters.yaml.txt.gz08:46
jbadiapajaosorior^08:46
openstackgerritmathieu bultel proposed openstack/python-tripleoclient master: Implement Upgrade workflow via the tripleo cli  https://review.openstack.org/46372808:47
jaosoriorjbadiapa: Gotta add it to tripleo-common08:47
jaosoriorjbadiapa: needs to be here https://github.com/openstack/tripleo-common/blob/master/container-images/overcloud_containers.yaml.j2  , here https://github.com/openstack/tripleo-common/blob/master/container-images/overcloud_containers.yaml   and in the relevant unit tests08:47
*** fzdarsky is now known as fzdarsky|afk08:48
jaosoriorjbadiapa: I suggest you add a patch that merely adds the fluentd service here https://github.com/openstack/tripleo-common/blob/master/container-images/overcloud_containers.yaml.j2#L16608:48
jaosoriorand doesn't remove FluentdClient08:48
jbadiapajaosorior, I have the patch, BUt I think the job should've failed08:48
jaosoriorI think so too.08:48
openstackgerritJuan Badia Payno proposed openstack/tripleo-common master: Fluentd: rename fluentd service  https://review.openstack.org/52001108:48
jaosoriorjbadiapa: So, lets do a three step thing08:48
jaosoriorjbadiapa: first add it (don't rename it)08:49
jaosoriorthen rename it in t-h-t08:49
jaosoriorthen remove FluentdClient from tripleo-common08:49
jaosoriorthat way it should always work in CI08:49
jaosoriorjbadiapa: now, we gotta figure out why the hell it's not failing in CI08:49
jaosoriorjbadiapa: are we even using containerized fluentd for that test?08:50
jbadiapajaosorior,  https://review.openstack.org/520011 ^ this is the add on tripleo-common.08:51
jbadiapajaosorior, https://github.com/openstack/tripleo-heat-templates/blob/master/ci/environments/scenario001-multinode-containers.yaml#L2708:51
*** jpena|off is now known as jpena08:52
jbadiapajaosorior, is https://github.com/openstack/tripleo-heat-templates/blob/master/ci/environments/scenario001-multinode-containers.yaml#L2608:52
jaosoriorjbadiapa: commented08:52
jaosoriorjbadiapa: interesting08:53
openstackgerritJuan Badia Payno proposed openstack/tripleo-common master: Fluentd: rename fluentd service  https://review.openstack.org/52001108:54
*** jpich has joined #tripleo08:57
jaosoriorjbadiapa: rechecked the patch08:58
jaosoriorjbadiapa: lets see how it goes in CI08:58
jbadiapajaosorior, we will see, thx08:59
jaosoriorjbadiapa: no biggie. Any other commits you need reviews for?09:00
jbadiapajaosorior, no thx09:00
jaosoriorok09:01
*** florianf has joined #tripleo09:02
Tengujaosorior: btw, care to check that one? https://review.openstack.org/#/c/518695/ I know it's not directly related to tripleo, but as you already had some insight in oslo… :)09:02
*** agurenko has joined #tripleo09:02
*** gkadam has joined #tripleo09:02
jaosoriorTengu: aaah dude, sorry, I forgot about it09:02
jaosoriorlets check it out09:03
Tenguno problem, it's not urgent.09:03
openstackgerritCarlos Camacho proposed openstack/instack-undercloud master: Load undercloud DB password to a mistral environment  https://review.openstack.org/51829209:05
*** gfidente has joined #tripleo09:06
jaosoriormandre: any idea what might be causing this http://paste.openstack.org/show/626337/ ?09:06
*** mcornea has joined #tripleo09:06
Tripleo-Userhttp://paste.openstack.org/show/626345/09:06
*** shardy has joined #tripleo09:07
*** nyechiel has quit IRC09:07
mandrejaosorior: Could not find class ::tripleo::packages? this is new to me09:08
jaosoriormandre: same here. It's something Tripleo-User is seeing.09:09
jaosoriorTripleo-User: are you using the overcloud-full.qcow2 image? or are you using a custom one?09:09
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432809:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103209:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106309:10
*** ooolpbot has quit IRC09:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)09:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]09:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]09:10
openstackgerritDaniel Alvarez proposed openstack/puppet-tripleo master: Add support for OVN Metadata Agent  https://review.openstack.org/50294009:10
Tripleo-Userhttp://paste.openstack.org/show/626347/09:11
*** nyechiel has joined #tripleo09:13
mandreTripleo-User: can you check your version of puppet-tripleo on the overcloud nodes?09:15
jaosoriorTripleo-User: wait a minute... you're using /home/stack/openstack-tripleo-heat-templates for your templates.09:15
jaosoriorTripleo-User: you said you were deploying with pike09:15
jaosoriorTripleo-User: are the tripleo-heat-templates you're using pike as well?09:16
jaosoriorusing the latest templates won't work.09:16
Tripleo-Userovercloud no puppet-tripleo packages09:16
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates stable/pike: Fail on upgrade if legacy ceph::profile::params::osds hiera is found  https://review.openstack.org/52001709:16
Tripleo-Useryes09:16
jaosoriorTripleo-User: where did you get /home/stack/openstack-tripleo-heat-templates ? did you clone that from git?09:17
*** lucas-afk is now known as lucasagomes09:17
*** suuuper has joined #tripleo09:17
*** abregman has quit IRC09:18
jaosoriorTripleo-User: when you do "git branch"  from /home/stack/openstack-tripleo-heat-templates what does it output?09:18
jaosoriorTripleo-User: you can still use tripleo-heat-templates from git, just need to make sure that you're using the right branch.09:19
Tripleo-Userhttp://paste.openstack.org/show/626348/09:20
*** holser__ has joined #tripleo09:21
*** holser__ has quit IRC09:21
*** holser__ has joined #tripleo09:22
jaosoriorTripleo-User: right. So here's the tricky thing. That's the version of the heat-templates for the Heat API. It might still be queens or master regardless of that version. Hence why I asked you to post the output of "git branch"09:24
Tripleo-Userhttp://paste.openstack.org/show/626337/   the problems is Related with overcloud-full.qcow2 puppet-modules ?09:27
jaosoriorTripleo-User: I think the problem is that either the overcloud image  or the tripleo-heat-templates  are not of matching versions.09:28
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Refactor Register nodes dialog to use redux-form  https://review.openstack.org/50490309:28
jaosoriorso one could be queens and another one could be pike.09:28
Tripleo-Userthe overcloud image only according to http://tripleo.org/install/basic_deployment/basic_deployment_cli.html#get-images to make it ?09:29
jaosoriorTripleo-User: did you use this repo /etc/yum.repos.d/delorean-pike.repo https://trunk.rdoproject.org/centos7-pike/current/delorean.repo ?09:31
jaosoriorTripleo-User: also, did you build the images yourself?09:32
*** dsariel has joined #tripleo09:32
*** derekh has joined #tripleo09:34
*** suuuper has quit IRC09:34
jaosoriorI'll be back. Gonna have lunch09:34
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Use prepared instackenv.json if exists  https://review.openstack.org/52002309:37
Tripleo-Useri have build the images09:38
jaosoriormarios, mandre: Do you know the link for the mirrors where Tripleo-User might be able to download pike images?09:38
jaosoriormarios, mandre would https://images.rdoproject.org/pike/rdo_trunk/tripleo-ci-testing/ be OK ?09:39
mandrejaosorior: not sure, let's see what tripleo-quickstart uses09:41
mandrejaosorior: IIUC it's https://images.rdoproject.org/pike/cbs/cloudsig-stable/09:42
openstackgerritmathieu bultel proposed openstack/python-tripleoclient master: Implement Upgrade workflow via the tripleo cli  https://review.openstack.org/46372809:42
openstackgerritCarlos Camacho proposed openstack/instack-undercloud master: Load undercloud DB password to a mistral environment  https://review.openstack.org/51829209:42
*** arxcruz__ has quit IRC09:43
mandrejaosorior: but then I don't know what is the difference between https://github.com/openstack/tripleo-quickstart/blob/master/config/release/stable/pike.yml and https://github.com/openstack/tripleo-quickstart/blob/master/config/release/trunk/pike.yml09:44
Tripleo-Userhttps://images.rdoproject.org/pike/cbs/cloudsig-stable/   stable/  version ?09:45
mandreTripleo-User: yes, try with these images ^^09:45
Tripleo-Usertesting is not tu use it ?09:46
*** cylopez has quit IRC09:47
*** mdnadeem has quit IRC09:49
Tripleo-Usertonight i test it use https://images.rdoproject.org/pike/cbs/cloudsig-stable/   stable/  version09:50
*** abregman has joined #tripleo09:52
*** fzdarsky|afk is now known as fzdarsky09:57
*** ffiore has quit IRC09:57
*** benone has quit IRC09:58
*** ffiore has joined #tripleo09:59
*** jpena is now known as jpena|mtg09:59
*** benone has joined #tripleo10:00
openstackgerritCarlos Camacho proposed openstack/instack-undercloud master: Load undercloud DB password to a mistral environment  https://review.openstack.org/51829210:01
openstackgerritmathieu bultel proposed openstack/tripleo-common master: Implement upgrade workflow in mistral  https://review.openstack.org/46376510:01
openstackgerritafazekas proposed openstack/tripleo-heat-templates stable/pike: Enable ntp iburst  https://review.openstack.org/51977910:01
*** mdnadeem has joined #tripleo10:02
*** cylopez has joined #tripleo10:06
openstackgerritAnastasia Kravets proposed openstack/puppet-tripleo master: Merge from from Juniper repo for opencontrail  https://review.openstack.org/51665110:08
*** benone has quit IRC10:09
*** chem has quit IRC10:09
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432810:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103210:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106310:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)10:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]10:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]10:10
*** chem has joined #tripleo10:10
openstackgerritmathieu bultel proposed openstack/tripleo-common master: Add the major upgrade converge in the update workbook  https://review.openstack.org/46376510:11
*** chem has quit IRC10:11
*** paramite has quit IRC10:11
*** benone has joined #tripleo10:12
*** salmankhan has joined #tripleo10:13
*** chem has joined #tripleo10:13
shardyCan I get reviews on this tripleoclient pike backport please https://review.openstack.org/#/c/519352/ ?10:13
Tengushardy: hello!10:14
shardyTengu: Hi!10:14
*** adarazs is now known as adarazs|ruck10:14
Tengushardy: I have a small question regarding tripleo and multi-controller setup: is it possible to replace a controller?10:14
Tengushardy: as the doc says we can't ADD a new one, I'm wondering what would happen in the case a controller crashes.10:14
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Remove TLS-specific docker environment  https://review.openstack.org/52003710:15
cschwedeHi! Can someone review this backport, please? https://review.openstack.org/#/c/515448/ It's a blocker when upgrading from non-containerized to containerized environment10:15
*** benone has quit IRC10:16
shardyTengu: yes there are a couple of ways to do it, just trying to find the docs10:18
*** benone has joined #tripleo10:18
Tengushardy: thank you! I'll like them in our internal docs once I've reviewed them :).10:18
Tenguhmm. s/like/add/ of course.10:19
*** cmyster has quit IRC10:20
cschwedejaosorior, mandre: thx a lot for your quick review!10:20
shardyTengu: http://tripleo-docs.readthedocs.io/en/latest/post_deployment/replace_controller.html10:21
Tenguah, perfect!10:21
shardyTengu: also https://docs.openstack.org/tripleo-docs/latest/install/post_deployment/delete_nodes.html10:21
shardywhich is basically another way to set ControllerRemovalPolicies (or same for any other role)10:22
TenguI'll add those links in our internal doc so that my colleagues will have the knowledge.10:22
shardyTengu: there is one other way that's not well documented, which allows reuse of e.g existing fixed IPs10:22
Tenguahh10:22
Tenguthat one, for a controller, might be interesting.10:22
shardythat is to use heat's resource mark-unhealthy interface10:22
shardyyou mark the server resource in the nested stack unhealthy, then run a normal deploy command10:23
shardyand heat replaces the server10:23
Tengushardy: hmm meaning we can mark all resources … ok10:23
Tenguthat one is nice.10:23
*** benone has quit IRC10:23
shardyTengu: yeah, I saw some notes on it but it's not made it into the docs yes10:23
*** daidv has quit IRC10:23
shardyyet10:23
Tengu:]10:23
openstackgerritmathieu bultel proposed openstack/instack-undercloud stable/pike: Install ansible-pacemaker during the upgrade/update in the UC  https://review.openstack.org/52004010:24
shardyThanks jaosorior10:24
*** benone has joined #tripleo10:25
*** cmyster has joined #tripleo10:25
*** cmyster has quit IRC10:25
*** cmyster has joined #tripleo10:25
*** tosky has joined #tripleo10:26
Tengushardy: so basically, `openstack stack list --nested | grep -i controller' find the one talking about the controller I want to replace (say I want to replace controller-2, so overcloud-Controller-3yzkkobwl2ih-2-xewsnehsdhuj) and mark the resource "OS::TripleO::ControllerServer" as unhealthy?10:29
*** benone has quit IRC10:30
*** benone has joined #tripleo10:30
shardyTengu: yup that should do it10:30
openstackgerritMoshe Levi proposed openstack/tripleo-heat-templates master: Fix applying pci whitelist in nova compute  https://review.openstack.org/52004210:30
Tenguok. I'll document it as well. That one is probably my favorite since we can re-use the same IPs.10:30
*** jaganathan has quit IRC10:30
*** nyechiel has quit IRC10:30
shardyTengu: you could also do openstack stack resource list overcloud-Controller-3yzkkobwl2ih-2-xewsnehsdhuj and just mark the server resource unhealthy, but that could cause issues with the network resources10:31
shardyprobably best to test it if you have time :)10:31
mosheleskramaja: the fix was in 2 places https://review.openstack.org/#/c/520041/ and https://review.openstack.org/#/c/520042/10:31
jaosoriorshardy: If I want to pass a binary file as part of the overcloud deployment, is there an interface I could already use?10:31
skramajamoshele: thanks, will look at it.10:32
shardyjaosorior: yes you could use the DeployArtifacts interface and unpack either a tarball or install an RPM10:32
mosheleskramaja: thanks10:32
Tengushardy: yep, but I have to find the stack name :). That part is tricky, really.10:33
Tengunames aren't that human.10:33
Tengushardy: hmm. there's no "Server" resource name in fact, in that stack. am I aiming the right one?10:34
*** benone has quit IRC10:35
Tenguit's named "Controller" maybe.10:35
openstackgerritMoshe Levi proposed openstack/tripleo-heat-templates master: Adds environment file for OVS Hardware Offload  https://review.openstack.org/50740110:36
shardyTengu: http://paste.openstack.org/show/626354/ is probably the way I'd try it10:36
Tengushardy: ah, good. So I wasn't wrong. thanks for the confirmation!10:37
shardyTengu: http://paste.openstack.org/show/626355/ sorry, pasted too much10:37
Tengu:)10:37
*** benone has joined #tripleo10:37
*** ffiore has quit IRC10:38
Tenguok, yep, I wasn't that far. your way is easier. Much easier :).10:38
shardyopenstack stack resource list e6e59a63-79d1-4a8c-90fb-da17843b7a68 | grep Server will show "Controller" of type OS::TripleO::ControllerServer10:39
shardyso you could also just mark that (the nova server resource) unhealthy, but it's probably safer to mark the whole stack including the ports etc10:39
shardyand have them all created again10:39
Tengushardy: ok!10:40
*** daidv has joined #tripleo10:41
jaosoriorshardy: is that something that I could use on a per-role basis?10:41
*** benone has quit IRC10:42
jaosoriorshardy: say, I only want the binary to be installed in a ceratain role10:42
shardyjaosorior: currently no but I guess we could add a per-role interface10:42
*** raissa_ has quit IRC10:42
shardyjaosorior: in general I think we expect all binary things to live in RPMs tho10:42
*** ffiore has joined #tripleo10:43
shardyjaosorior: what's the data in this case?10:43
jaosoriorshardy: a PKCS12 file.10:43
*** benone has joined #tripleo10:44
*** benone has quit IRC10:48
*** holser__ has quit IRC10:49
*** holser__ has joined #tripleo10:49
shardyjaosorior: ack, can that be exported in an ascii encoded form then passed as a parameter?10:51
*** benone has joined #tripleo10:51
jaosoriorshardy: uhm... Actually that's not a bad idea. I'll look into that.10:51
*** daidv has quit IRC10:52
jaosoriorshardy: still, having per-role swift artifacts would be kinda nice :D\10:52
*** abregman is now known as abregman|mtg10:52
*** etingof has quit IRC10:53
*** raissa_ has joined #tripleo10:53
openstackgerritCarlos Camacho proposed openstack/instack-undercloud master: Load undercloud DB password to a mistral environment  https://review.openstack.org/51829210:53
shardyjaosorior: Yeah we could add it but that's supposed to be a developer interface really10:53
jaosorioruhm10:53
jaosoriorok10:53
*** aditya_ra has joined #tripleo10:54
*** paramite has joined #tripleo10:55
*** aditya_ra has quit IRC10:55
*** etingof has joined #tripleo10:55
*** benone has quit IRC10:56
*** aditya_r has quit IRC10:57
*** raissa_ is now known as raissa10:57
*** benone has joined #tripleo10:58
*** benone has quit IRC11:03
*** benone has joined #tripleo11:06
*** holser__ has quit IRC11:08
*** rfolco has joined #tripleo11:08
*** daidv has joined #tripleo11:09
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432811:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103211:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106311:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)11:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]11:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]11:10
*** sg has joined #tripleo11:10
*** benone has quit IRC11:11
*** sg is now known as Guest317011:11
*** Guest3170 has quit IRC11:11
*** nguyentrihai has joined #tripleo11:11
*** holser__ has joined #tripleo11:12
*** benone has joined #tripleo11:12
*** daidv has quit IRC11:12
*** mrunge has quit IRC11:15
*** benone has quit IRC11:16
*** benone has joined #tripleo11:17
*** benone has quit IRC11:21
*** cshastri has quit IRC11:22
*** benone has joined #tripleo11:24
*** pdeore has quit IRC11:25
*** bogdando has quit IRC11:26
jaosoriorjistr, mandre: can you check this out https://review.openstack.org/#/c/519577/ ?11:27
*** daidv has joined #tripleo11:27
*** holser__ has quit IRC11:28
*** benone has quit IRC11:29
*** benone has joined #tripleo11:30
*** atoth has joined #tripleo11:30
*** daidv has quit IRC11:31
*** benone has quit IRC11:35
*** holser has joined #tripleo11:35
*** holser has quit IRC11:37
*** holser has joined #tripleo11:37
openstackgerritMerged openstack/tripleo-heat-templates master: Fail on upgrade if legacy ceph::profile::params::osds hiera is found  https://review.openstack.org/51869811:37
*** mrunge has joined #tripleo11:37
openstackgerritMerged openstack/tripleo-heat-templates master: Update the services README with the ServiceData parameter  https://review.openstack.org/48430611:37
openstackgerritMerged openstack/diskimage-builder master: elements: Respect devpts mount options  https://review.openstack.org/51874911:44
*** daidv has joined #tripleo11:45
*** benone has joined #tripleo11:46
*** salmankhan has quit IRC11:48
*** salmankhan has joined #tripleo11:48
*** benone has quit IRC11:50
openstackgerritCarlos Camacho proposed openstack/instack-undercloud master: Load undercloud DB password to a mistral environment  https://review.openstack.org/51829211:51
openstackgerritDerek Higgins proposed openstack/tripleo-docs master: Add details about boot baremetal from cinder volume  https://review.openstack.org/52005411:53
jbadiapajaosorior, looking back to the https://review.openstack.org/#/c/501457/ I can say that the job "legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-container" is useless11:54
*** shreshtha has quit IRC11:55
jaosoriorjbadiapa: it depends11:55
*** shreshtha has joined #tripleo11:55
jaosoriorjbadiapa: if it builds the images. It's useless. If it uses the defaults, it might as well just work11:56
jbadiapajaosorior, I might be wrong but the fluentd docker definition is not at http://logs.openstack.org/57/501457/8/check/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-container/9de5e87/logs/undercloud/home/zuul/containers-default-parameters.yaml.txt.gz11:57
jaosoriorjbadiapa: how the hell is it passing11:57
jaosoriorjbadiapa: I now I agree with you on the "this job is useless" statement11:57
jbadiapajaosorior, it was able to "deploy" the overcloud "not with fluentd" though http://logs.openstack.org/57/501457/8/check/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-container/9de5e87/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz11:58
jaosoriorjbadiapa: it does run the fluentd installation though http://logs.openstack.org/57/501457/8/check/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-container/9de5e87/logs/subnode-2/var/log/messages.txt.gz#_Nov_15_10_24_2911:59
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Collect unbound DNS server configs and logs  https://review.openstack.org/52000711:59
*** oidgar has quit IRC12:00
jaosoriorjbadiapa: what's the environment file it should be using?12:00
*** ukalifon has joined #tripleo12:00
*** bcafarel has quit IRC12:01
jbadiapajaosorior, It should use this https://review.openstack.org/#/c/501457/8/ci/environments/scenario001-multinode-containers.yaml12:03
jbadiapajaosorior, tbh I'm not sure12:04
*** holser has quit IRC12:04
*** holser has joined #tripleo12:04
openstackgerritNir Magnezi proposed openstack/tripleo-common master: Octavia post deployment mistral wrapper  https://review.openstack.org/51540212:05
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: debug unbound queries  https://review.openstack.org/52005512:05
jaosoriorjbadiapa: seems to be creating a fluentd container somehow http://logs.openstack.org/57/501457/8/check/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-container/9de5e87/logs/subnode-2/var/log/messages.txt.gz#_Nov_15_10_37_5312:05
jaosoriorI wonder what image it uses :/12:05
jaosoriorjbadiapa: there's even fluentd logs for the container http://logs.openstack.org/57/501457/8/check/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-container/9de5e87/logs/subnode-2/var/log/containers/fluentd/12:06
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements  https://review.openstack.org/51918912:06
*** lucasagomes is now known as lucas-hungry12:06
openstackgerritMerged openstack/python-tripleoclient stable/pike: Don't rewrite resource_registry values that aren't paths  https://review.openstack.org/51935212:09
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432812:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103212:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106312:10
*** ooolpbot has quit IRC12:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]12:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]12:10
*** janki is now known as janki|afk12:10
*** bcafarel has joined #tripleo12:14
*** ffiore has quit IRC12:18
*** stendulker has joined #tripleo12:18
openstackgerritJiri Stransky proposed openstack/tripleo-quickstart-extras master: Fix conditional to be compatible with new Ansible  https://review.openstack.org/52005712:20
jtomasekd0ugal, rbrady: hi, when I try to test https://review.openstack.org/#/c/509832/ I am getting this http://paste.openstack.org/show/626370/12:20
jtomasekd0ugal: is that still related to missing the yaml_parse in mistral? I've just installed new env12:20
d0ugaljtomasek: no, I don't think so.12:21
d0ugaljtomasek: nah, it is just broken12:21
d0ugalI'll comment on the review12:21
d0ugalguess it hasn't been tested yet12:21
jtomasekok, I'll comment on the patch, thanks12:21
rbradyjtomasek: I'll retest after the program call and get back to you12:21
jtomasekrbrady: thanks12:22
amoralejadarazs|ruck, would it be worthy to move on with https://review.openstack.org/#/c/519770/ until real fix is pushed?12:24
amoralejit's the only blocker for pike and master12:24
jistradarazs|ruck, panda, trown|outtypewww, sshnaidm: hi, looks like quickstart got broken on libvirt environments (probably not multinode/OVB, luckily). Please take a look at the fix when you have some bandwidth https://review.openstack.org/#/c/520057/12:24
amoraleji don't want to blindly recheck there12:24
*** ffiore has joined #tripleo12:25
sshnaidmjistr, did  you test it locally?12:25
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Update the .gitignore  https://review.openstack.org/51850612:25
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Correct the package name in the setup.cfg  https://review.openstack.org/52006112:25
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Add initial workflow tests for the stack.yaml workbook  https://review.openstack.org/52006212:25
jistrsshnaidm: yes i did, i got past the broken point12:26
jistr(still running though)12:26
sshnaidmjistr, thanks12:26
sshnaidmjistr, what is "new ansible"?12:26
adarazs|ruckamoralej: probably that's a good idea, yeah. I was hoping that https://review.openstack.org/519486 will fix it, but it didn't, as far as I can see from the gate job.12:27
*** janki|afk is now known as janki12:27
*** ecerquei has quit IRC12:28
jistrsshnaidm: i think 2.4.1.0 though i'm actually not sure what i'm running :D12:28
jistr^ that's what is printed from ansible --version12:29
*** karthiks has quit IRC12:29
sshnaidmjistr, hmm.. doesn't quickstart use ansible from its virtualenv directory?12:29
adarazs|ruckhmm, it's weird we didn't hit that error.12:29
adarazs|ruckwhen gating the bump in ansible version12:29
openstackgerritMerged openstack/tripleo-heat-templates stable/pike: Remove rsync from xinetd when upgrading to containerized deployment  https://review.openstack.org/51544812:30
openstackgerritMerged openstack/tripleo-quickstart master: Set quickstart to use ansible 2.4.1  https://review.openstack.org/51005912:30
openstackgerritMerged openstack/instack-undercloud master: Install ansible-pacemaker during the upgrade/update in the UC  https://review.openstack.org/51966312:30
jistrso there might be 2 things at play :)12:30
jistr1) i don't use quickstart.sh12:30
jistr2) CI might not excercise that code path at all12:30
adarazs|ruckjistr: it should when we run on centos ci and libvirt.12:31
adarazs|ruckbut the fix looks good so +212:31
sshnaidmadarazs|ruck, 2.4.1 didn't merge yet12:31
adarazs|rucksshnaidm: but we ran the centosci gate jobs on it.12:31
adarazs|ruckand they passed12:31
jistradarazs|ruck: ah ok so i guess 1) is the real cause :)12:31
sshnaidmadarazs|ruck, yeah, that's why I want to understand which ansible version exactly is used12:32
adarazs|rucksshnaidm: any objection from submitting that fix though?12:32
sshnaidmadarazs|ruck, I'd like to see CI jobs before12:32
sshnaidmjistr, so you don't use quickstart.sh?12:32
* jistr double checking my env what ansible i used12:32
*** abregman|mtg is now known as abregman12:32
*** ansmith has quit IRC12:33
slaglemarios: jistr : can you +2 this is you're fine with the naming as-is? or are there other things you'd like to see addressed?12:34
slaglehttps://review.openstack.org/#/c/514701/12:34
jistrslagle: ack, +2'd, there are probably no blockers, i just recall that i wondered about the other things marios and d0ugal brought up, but they don't seem like blockers either12:37
mariosslagle: ack +2 too12:37
marios:)12:37
*** stendulker has quit IRC12:37
slaglethanks. d0ugal's only other comment was about a change to one of the other interfaces not related to my patch12:38
slaglei do need to update the tripleo-common side though12:38
beaglesadarazs|ruck, hey, I just took a look at the patch you mentioned above regarding mtu. In some of the jobs that look like they are related it seems that the 1350 isn't getting configured but maybe I'm looking in the wrong spot?12:39
*** rhallisey has joined #tripleo12:40
jistrsshnaidm: so i guess i use system ansible. Yes i don't use quickstart.sh as the interface doesn't match my workflow. But i do source ~/.quickstart/bin/activate before running ansible. But still `which ansible` prints /usr/bin/ansible... I gotta be doing something wrong :)12:40
beaglesadarazs|ruck, I noticed you did a recheck so maybe next time it comes around and if it still fails, maybe you can point me exactly where to look? It seems like your fix should work12:40
*** rhallisey_ has joined #tripleo12:40
beaglesadarazs|ruck, "exactly where to look" = "the CI job that should affected"12:41
adarazs|ruckbeagles: we're changing the ovb settings for rdo cloud, so they are the gate-tripleo-ci-centos-7-ovb-... jobs posted by "RDO Third Party CI"12:42
adarazs|ruckbeagles: what I was looking at specifically is the "gate-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master-nv" job that runs full tempest; I was expecting it to fix the 29 failures that job has, but it stayed at 29 failures :/12:43
adarazs|ruckbeagles: I didn't check if the MTU was actually adjusted, just the end result :)12:43
*** abishop has joined #tripleo12:43
sshnaidmjistr, hm, it's strange, maybe you set virtualenv with system site packages?12:44
beaglesadarazs|ruck, okay cool - I'll see if I can sort out whether the setting took affect12:44
beaglesor effect12:44
*** stendulker has joined #tripleo12:44
*** rhallisey has quit IRC12:44
sshnaidmjistr, well, anyway we move to 2.4.1 right now, so it's great to have this fix12:44
beaglesor affect... whatever... more coffee12:45
*** karthiks has joined #tripleo12:45
*** Tripleo-User has quit IRC12:45
adarazs|ruckbeagles: I hope the coffee will affect you positively and in effect you'll figure things out. :)12:46
beagleslol12:46
*** qwebirc77837 has joined #tripleo12:46
jistrsshnaidm: i'm calling the bootstrap wrong somehow, ansible doesn't seem to be installed in the .quickstart virtualenv... I have additional automation around OOOQ. This is what i call, i did include the quickstart requirements so it's weird that Ansible isn't installed, anyway i'll figure it out :) https://paste.fedoraproject.org/paste/DQFJC5dI0q4ptADJC1kexw12:46
jistror i'll at least try to :D12:46
*** ffiore has quit IRC12:47
EmilienMhi12:48
*** ffiore has joined #tripleo12:49
adarazs|ruckEmilienM: good morning.12:49
openstackgerritJames Slagle proposed openstack/tripleo-common master: Add workflow for deployment via config-download  https://review.openstack.org/51287612:51
openstackgerritJames Slagle proposed openstack/tripleo-common master: Log Ansible actions to ansible.log  https://review.openstack.org/51478312:51
openstackgerritJames Slagle proposed openstack/tripleo-common master: Add work_dir for some actions  https://review.openstack.org/51478412:51
*** bobh has joined #tripleo12:52
slagled0ugal: can you check my comment on setup.cfg in https://review.openstack.org/#/c/512876/12:52
slaglethe ansible actions don't match the existing pattern. i can make mine match all the !ansible actions or the ansible actions :)12:53
*** ffiore has quit IRC12:54
openstackgerritBeth Elwell proposed openstack/tripleo-ui stable/pike: Fix Validations progress  https://review.openstack.org/52006912:56
*** karthiks has quit IRC12:57
*** jlabarre has joined #tripleo13:01
*** dprince has joined #tripleo13:02
*** jpena|mtg is now known as jpena|lunch13:03
*** athomas has quit IRC13:07
*** lucas-hungry is now known as lucasagomes13:07
*** absubram has joined #tripleo13:07
*** psahoo has quit IRC13:07
*** zshi_ has quit IRC13:08
*** absubram_ has joined #tripleo13:09
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432813:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103213:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106313:10
*** ooolpbot has quit IRC13:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)13:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]13:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]13:10
*** ffiore has joined #tripleo13:10
*** artom has joined #tripleo13:11
*** absubram has quit IRC13:11
*** absubram_ is now known as absubram13:11
beaglesgetting some failures in openstack-tox-py27 (and py35) jobs for tripleo_common.tests.image.test_kolla_builder.TestKollaImageBuilderTemplate.test_container_images_from_template_filter13:12
*** aputtur_ has joined #tripleo13:12
beaglesis this a known or is this something new?13:12
*** dprince has quit IRC13:12
*** dprince has joined #tripleo13:12
*** karthiks has joined #tripleo13:13
*** trown|outtypewww is now known as trown13:14
openstackgerritGiulio Fidente proposed openstack/tripleo-docs master: Update upgrade documentation with the Ceph related steps  https://review.openstack.org/51163513:15
*** dmacpher has joined #tripleo13:16
*** nguyentrihai has quit IRC13:16
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates master: Set file mode permission for Ceph keyrings in containers  https://review.openstack.org/50897513:16
openstackgerritRicardo Noriega proposed openstack/tripleo-heat-templates master: Using stevedore alias for BGPVPN/L2GW Service Plugin  https://review.openstack.org/51127713:18
openstackgerritDerek Higgins proposed openstack/tripleo-docs master: Add details about boot baremetal from cinder volume  https://review.openstack.org/52005413:20
*** brian83 has joined #tripleo13:21
*** links has quit IRC13:22
openstackgerritMoshe Levi proposed openstack/tripleo-heat-templates master: Adds environment file for OVS Hardware Offload  https://review.openstack.org/50740113:22
*** amoralej is now known as amoralej|lunch13:23
*** links has joined #tripleo13:24
*** tzumainn has joined #tripleo13:26
openstackgerritAlan Bishop proposed openstack/tripleo-heat-templates master: CI: Test multi-rbd backend for cinder in scenario001  https://review.openstack.org/51974913:27
*** bogdando has joined #tripleo13:27
*** stendulker has quit IRC13:28
*** stendulker has joined #tripleo13:29
*** bobh has quit IRC13:30
*** athomas has joined #tripleo13:31
*** rlandy has joined #tripleo13:32
*** nguyentrihai has joined #tripleo13:33
*** skramaja has quit IRC13:33
*** liverpooler has quit IRC13:34
*** ecerquei has joined #tripleo13:37
*** tosky has quit IRC13:37
*** toure_biab is now known as toure13:38
beagleslooks like "namespace" is being used as a reserved word or something in jinja?13:40
openstackgerritJiri Stransky proposed openstack/tripleo-quickstart master: Allow deploying Kubernetes worker nodes in fs026 but keep default at 0  https://review.openstack.org/52007313:41
*** moshele has quit IRC13:41
weshayarxcruz, fultonj, gfidente, trown FYI.. it would be interesting to see if the four of you guys synced for a sec re: the volumeboot pattern tempest bug to see if we can make some more progress today13:42
weshaymwhahaha, ^13:42
arxcruzweshay: ack13:42
fultonjweshay: ack13:42
weshaywe have a live environment up and trown afaik has been unable to get tempest to fail13:42
gfidenteweshay no I would like someone more familiar with ceilometer to check what is going on13:42
gfidentethere are a few services deployed only in scenario00113:43
gfidentecollectd, fluentd13:43
*** brault has quit IRC13:43
trownya I even ran a script that ran tempest 100 times in a row last night13:44
trownit did start failing around run 20... but only because it had knocked over the overcloud13:44
trownnot sure how to reproduce the tempest issue if it only happens in some clouds13:44
*** nyechiel has joined #tripleo13:45
weshaytrown, we need to confirm that w/ a recheck query13:45
gfidentetrown that was with the same set of enabled services?13:45
trowngfidente: that is with the exact job as run in CI ... ie I run the toci script on a pre-provisioned environment on rdocloud (similar to traas)13:46
gfidenteah ok13:46
gfidentesorry13:46
gfidentemissed that13:46
gfidentetrown can you check how long is the tempest run in your tests13:47
gfidenteand if it's much slower or faster than the CI cloud?13:47
*** bogdando has quit IRC13:47
*** ansmith has joined #tripleo13:48
trowngfidente: on the order of 15min13:48
trownfor 3 tests13:48
*** etingof has quit IRC13:48
*** brault has joined #tripleo13:49
fultonjarxcruz: in the tempest job there are 3 instances booted. the 3rd is up for 1 minute or less. Do we know if the test consists of trying to ssh into the 3rd. What if the 3rd just isn't up yet and we need to give it more time?13:49
fultonjcan tempest ping first or sleep or retry before it SSH's?13:49
gfidentefultonj from the logs looks like tempest did 23 attempts13:50
gfidenteat each it waited 1sec longer, starting from 1sec13:51
sshnaidmEmilienM, which repos do have influence on undercloud installation?13:51
fultonjgfidente: ok, thanks13:51
EmilienMsshnaidm: what do you mean13:51
EmilienMsshnaidm: instack-undercloud?13:51
gfidentefultonj but I don't know if it tries on all vms at the same time13:51
*** mdnadeem has quit IRC13:52
fultonjdo we know which VM it tries to SSH into?13:52
gfidentefultonj in one of the comments13:52
sshnaidmEmilienM, for example instack-undercloud, are there another repos that could affect undercloud installation?13:52
gfidenteowalsh was pointing out 2 out of 3 guests were reachable13:52
sshnaidmEmilienM, for example I have an undercloud box and want rebuild it every time something changes in undercloud13:52
sshnaidmEmilienM, so which repos should I track to rebuild it every time?13:53
*** links has quit IRC13:53
fultonjjust saying the 3rd had a very short life13:53
beaglesmwhahaha, just a heads up, I just filed this bug https://bugs.launchpad.net/tripleo/+bug/1732453 - not sure if this was an already known issue13:53
openstackLaunchpad bug 1732453 in tripleo "jinja treating 'namespace' as a keyword in tripleo_common tests" [Critical,Triaged]13:53
EmilienMsshnaidm: a lot13:53
EmilienMsshnaidm: all openstack packages13:53
gfidentefultonj ah sorry I didn't get that13:53
EmilienMsshnaidm: all puppet modules13:53
EmilienMetc13:53
fultonjnp, i did ask a few questions that were already in the bug report that i should have seen13:54
sshnaidmEmilienM, these will be provided with dlrn repos13:54
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Remove 'Refresh results' button on the nodes page  https://review.openstack.org/51547413:54
arxcruzfultonj: checking the code13:54
beaglesmwhahaha, I've just started looking at the issue - just getting to the "thinking about how to deal with" part :)13:54
fultonjit's the common trend i see in non-containerized and containerized jobs as per the libvirt logs for instance313:55
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Install dstat when a ready undercloud image is not used.  https://review.openstack.org/51972513:55
fultonjarxcruz: it would be good to know it at least answered pings, not sure if the job can do that before SSH'ing13:55
*** udesale has quit IRC13:56
sshnaidmEmilienM, well, maybe it's enough to track delorean-current repo where are newest tripleo packages?13:56
arxcruzfultonj: checking13:56
* fultonj tries to correlate log from ssh failure with instance 3 uptime13:56
*** jpena|lunch is now known as jpena13:56
EmilienMsshnaidm: probably13:56
sshnaidmEmilienM, I thought maybe we can run ovb jobs on prepared new undercloud image which includes installed undercloud.. Not sure though how to keep it updated. Do you think it could be helpful?13:57
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Install dstat when a ready undercloud image is not used.  https://review.openstack.org/51972513:57
*** udesale has joined #tripleo13:58
EmilienMsshnaidm: you can run undercloud upgrade commands13:58
EmilienMsshnaidm: it's documented, let me show you13:58
arxcruzfultonj: https://github.com/openstack/tempest/blob/master/tempest/scenario/test_volume_boot_pattern.py#L82-L9213:58
gfidentefwiw we just hit the issue again http://logs.openstack.org/36/516136/4/gate/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-container/a9f1d07/13:58
EmilienMsshnaidm: https://docs.openstack.org/tripleo-docs/latest/install/installation/installation.html#updating-undercloud-components13:58
arxcruzfultonj: I'm checking if it's failing in the first vm or in the third one13:59
fultonjarxcruz: thanks13:59
jtomasekhonza: I am sorry the patch is huge, but I am not sure I can split it. The reason why it is relatively large is because it is a result of several iterations on solving problems which whole patchset this patch is part of solves. I would rather iterate fast and merge fast but I am not sure we have a suitable environment for that. Splitting that patch is going to cause major pain to keep the dependent patches up to date13:59
honzajtomasek: Yes, agreed; sorry, I didn't mean for you to change it, just something to keep in mind for the future14:00
fultonjhttps://github.com/openstack/tempest/blob/master/tempest/scenario/test_volume_boot_pattern.py#L137-L15514:01
honzajtomasek: Yes, I prefer lots of tiny patches merged fast to long-running patches14:01
fultonj^ so the 3rd instance was booted from the snapshot14:01
honzajtomasek: The -1 was for that wrapper.  Is there no other way?14:01
honzajtomasek: I feel we're approaching INCEPTION14:02
jtomasekhonza: I am going to look into it. my goal was to make AuthenticatedContent lighter because it had too much logic in it.14:02
*** shreshtha has quit IRC14:02
gfidentetrown upstream looks like it took only 10 minutes for a much bigger set of tests? http://logs.openstack.org/36/516136/4/gate/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-container/a9f1d07/logs/undercloud/home/zuul/tempest_output.log.txt.gz#_2017-11-15_13_05_1814:02
*** shreshtha has joined #tripleo14:02
trowngfidente: look at the bottom... only 3 test actually run14:03
weshaydprince, mind if I change the dependencies on https://review.openstack.org/#/c/518454/ and https://review.openstack.org/#/c/519441/  I think we can flip them and may see it work in ci that way14:03
trowngfidente: the numbers in my env are in the same ballpark as the ones at the bottom of that job14:03
trowngfidente: though the failing test completes in ~150 seconds when it doesnt timeout14:04
dprinceweshay: I don't think that is the issue14:04
dprinceweshay: what are you seeing in the latest CI logs that indicates flipping them might work?14:04
weshaydprince, well.. I see undercloud-containers run w/ https://review.openstack.org/#/c/519441/ but it's not reporting back w/ logs14:05
weshayhttps://review.openstack.org/#/c/518454/ is reporting w/ undercloud-containers but doesn't have the tq patch14:05
openstackgerritJohn Fulton proposed openstack/tripleo-heat-templates master: Set file mode permission for Ceph keyrings in containers  https://review.openstack.org/50897514:05
weshaydprince, ah.. hearing that zuul3 was restarted14:06
weshaythat could be the reason jobs were not reporting14:06
weshay#lame14:06
*** rfolco is now known as rfolco_brb14:07
weshayhttps://twitter.com/openstackinfra/status/93063103484832563414:08
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Enable httpd to request certificates for the "external" network  https://review.openstack.org/52008114:08
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Dynamically generate FQDN hiera entries  https://review.openstack.org/52008214:08
*** pgadiya has quit IRC14:08
saimwhahaha: so, with pike can i deploy without containers by not passing the docker env files?14:08
fultonjsai: if you want to do that for ceph, then use puppet-ceph, not ceph-ansible14:09
*** Goneri has joined #tripleo14:09
saifultonj: hey, thanks, no ceph for now14:09
fultonjsai: and it will work upstream for centos14:09
fultonjsai: but downstream you have to update the image14:10
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432814:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103214:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106314:10
*** ooolpbot has quit IRC14:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)14:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]14:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]14:10
sshnaidmEmilienM, so for upgrade master-master I mostly need new repos, yum update for all packages and then "openstack undercloud upgrade", right?14:10
EmilienMsshnaidm: I think yum update is run for you14:11
EmilienMwhen doing openstack undercloud upgrade14:11
EmilienMlet me read the doc again14:11
*** fragatina has joined #tripleo14:11
sshnaidmEmilienM, so it will upgrade *all* packages, including non-openstack related, right?14:12
EmilienMyep14:12
numansEmilienM, Hi, your patch to enable tempest tests for ovn scenario007 job is passing now as ovs 2.8.1 is available :)  - https://review.openstack.org/#/c/494293/14:12
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Update the .gitignore  https://review.openstack.org/51850614:12
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Correct the package name in the setup.cfg  https://review.openstack.org/52006114:12
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Add initial workflow tests for the stack.yaml workbook  https://review.openstack.org/52006214:12
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Add the tempest command check-uuid to tox.ini  https://review.openstack.org/52008414:12
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Test the create_default_deployment_plan workflow  https://review.openstack.org/52008514:12
EmilienMwoot14:12
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Test create_deployment_plan with a git repo as a source  https://review.openstack.org/52008614:12
sshnaidmEmilienM, great, thanks14:12
arxcruzfultonj: it fails in the first instance: 2017-11-08 21:19:54 |     2017-11-08 21:13:44,278 18813 INFO     [tempest.scenario.test_volume_boot_pattern] Booting instance 1 from volume14:12
*** dpawar has quit IRC14:13
EmilienMnumans, weshay: where are tempest html results ? http://logs.openstack.org/93/494293/14/check/legacy-tripleo-ci-centos-7-scenario007-multinode-oooq/78ade4e/logs/14:13
EmilienMI thought this problem was fixed14:13
fultonjarxcruz: ok, thanks good to know14:13
arxcruzfultonj: now, it might not being able to connect because the cloud-init fails to get the dhcp (it happened in the past)14:13
weshayEmilienM, apparently the patch was about to land, but was bitten by the zuul reboot14:13
arxcruzfultonj: or something related to the cloud used on these jobs14:13
*** rfolco_brb is now known as rfolco14:13
weshayI also thought it was fixed14:13
EmilienMweshay: which patch14:14
honzajtomasek: also lol at the link to the 'use a render prop' article, why does everything in react need to be a 'story'? :)14:14
*** thrash|g0ne is now known as thrash14:14
weshayarxcruz, ^14:14
*** ratailor has joined #tripleo14:14
EmilienMnumans: please remove depends-on on https://review.openstack.org/#/c/494293/1414:14
arxcruzEmilienM: weshay https://review.openstack.org/#/c/517906/14:14
numansEmilienM, http://logs.openstack.org/93/494293/14/check/legacy-tripleo-ci-centos-7-scenario007-multinode-oooq/78ade4e/logs/undercloud/home/zuul/tempest_output.log.txt.gz14:14
numansEmilienM, oh yeah. ack will do that14:15
d0ugalI would like to setup a new CI job that has the undercloud only and runs tempest (testing tripleo workflows with tripleo-common-tempest-plugin). Anyone know how or where I would start doing that?14:15
openstackgerritPradeep Kilambi proposed openstack/instack-undercloud master: Fix panko ssl port  https://review.openstack.org/52008714:15
d0ugalEmilienM: ^ maybe you know?14:15
jtomasekhonza: same patch?14:15
numansEmilienM, this problem needed a fix in ovs as  there was a bug in ovs and ovs 2.8.1 is available now :)14:15
honzajtomasek: https://review.openstack.org/#/c/516362/3/src/js/components/ui/Modals/CloseModal.js14:15
EmilienMd0ugal: yes I know14:16
honzajtomasek: no, sorry, I thought it was14:16
EmilienMnumans: I don't like this workaround14:16
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Run rabbitmq_policy,rabbitmq_user with pacemaker  https://review.openstack.org/52008814:16
numansEmilienM, which one ? you mean the Depends-On patch ?14:16
*** fragatina has quit IRC14:16
EmilienMnumans: yes.14:16
numansEmilienM, that is not required.14:16
honzahttps://cdb.reacttraining.com/use-a-render-prop-50de598f11ce14:16
numansEmilienM, i will remove it.14:16
EmilienMd0ugal: it's a bit of complicated now but we should be able to experiment something14:16
EmilienMd0ugal: I think you want to reach out tripleo ci squad14:17
d0ugalAh, good point. I should have looked them up.14:17
EmilienMd0ugal: but ++ for a ci job for that14:17
EmilienMd0ugal: right now I'm working on zuul v3 transition, so we'll be able to setup CI jobs more quickly than before14:17
d0ugalGreat.14:18
EmilienMd0ugal: with the layout in-tree14:18
*** dpawar has joined #tripleo14:18
dprinceweshay: ah, should we recheck it then?14:18
*** pblaho has quit IRC14:19
*** ramishra has quit IRC14:19
weshaydprince, already did14:19
*** amoralej|lunch is now known as amoralej14:20
weshaydprince, http://zuulv3.openstack.org/static/stream.html?uuid=0baa4242b6a5413b8162047d11bcb8fb&logfile=console.log14:20
openstackgerritNuman Siddique proposed openstack/tripleo-quickstart master: Switch scenario007 to run Tempest  https://review.openstack.org/49429314:20
*** cdearborn has joined #tripleo14:22
*** rlandy is now known as rlandy|rover14:23
ratailorhttps://review.openstack.org/#/c/511772/14:23
ratailorhttps://review.openstack.org/#/c/511773/14:23
ratailorhttps://review.openstack.org/#/c/519610/14:23
ratailorhttps://review.openstack.org/#/c/519611/14:23
ratailorEmilienM, jaosorior could you pl review these ^^ patches, and help me in getting these merged ASAP, we need to do this before GA.14:23
*** ramishra has joined #tripleo14:23
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs  https://review.openstack.org/51479314:24
openstackgerritmathieu bultel proposed openstack/puppet-tripleo stable/pike: HAproxy should get full response from ironic-inspector  https://review.openstack.org/52008914:24
*** tosky has joined #tripleo14:24
jaosoriorratailor: you need the containerized bits though. Have you had a chance to check that out?14:24
ratailorjaosorior, I was on sick leave today, so I couldn't get it done.14:25
EmilienMd0ugal: once i'm done with tripleo-ci repo transition I'll do tripleo-common so you can start working on a new job14:25
*** dpawar has quit IRC14:25
EmilienMd0ugal: but in a nutshell, you'll probably want a new featureset in quickstart14:25
ratailorjaosorior, I will check that tomorrow. in meantime would you pl get it review others.14:25
EmilienMd0ugal: describing what you want to deploy and test, you'll probably need someone from oooq to help you14:25
d0ugalEmilienM: great, that would be fantastic.14:26
d0ugalEmilienM: most likely, I don't know much about quickstart - time to learn :)14:26
weshayd0ugal, https://github.com/openstack/tripleo-quickstart-extras/blob/master/playbooks/quickstart-extras-validate.yml and https://github.com/openstack/tripleo-quickstart-extras/tree/master/roles/validate-tempest14:26
weshayd0ugal, you'll need to point tempest config at the undercloud14:26
ratailorjaosorior, I think the containerzation part is just a matter of creating env file. right ?14:26
ratailorjaosorior, so that won't take much time, I think.14:26
*** jmelvin has joined #tripleo14:27
d0ugalweshay: oh, nice - so I think most of the hard work has been done.14:28
d0ugalI assume I need to get an rpm for my tests, so they are installed and then they can be executed with all the others14:28
jaosoriorratailor: check docker/services/glance-api.yaml14:28
jaosoriorratailor: it requires a separate container.14:28
thrashd0ugal: I was actually just thinking about it this morning. I'm thinking it should be based on scenario00314:29
jaosoriorratailor: where we'll run httpd (the TLS proxy)14:29
weshayd0ugal, ya.. you will see in the tq/config/general_config/featureset yaml files that different jobs install different sets of tempest plugins14:29
weshay#woot almost out of the dog house w/ infra http://status.openstack.org/openstack-health/#/14:29
*** udesale has quit IRC14:29
weshayonly 1 tripleo project left14:29
fultonjarxcruz: for https://bugs.launchpad.net/tripleo/+bug/1731063 is it worth it to update the test to verify that instance001 answers pings before the SSH attempt so we can rule out network issues?14:29
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]14:29
d0ugalweshay: thanks!14:30
d0ugalthrash: I'll take a look.14:30
thrashd0ugal: but weshay et al can help you more. :)14:30
fultonjor even have the job connect to port 22 so we can also rule out a security group issue and then feel more confident that it's cloud-init?14:30
openstackgerritFlorian Fuchs proposed openstack/tripleo-validations master: Fix the Network Gateway Validation  https://review.openstack.org/51861914:30
ratailorjaosorior, something like this. https://github.com/openstack/tripleo-heat-templates/commit/71efc9fcecf7b9b6939a3f8a9a8a1ab0c6d702e814:30
weshayd0ugal, apetrich can help you out.. you'll want to get setup through the deployment, then you can follow https://docs.openstack.org/tripleo-quickstart/latest/getting-started.html#validate-the-tripleo-overcloud-is-functional14:31
arxcruzfultonj: what would worth, is on that cases, get the output from console-log14:31
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Remove too opinionated parameters from kubernetes.yaml environment  https://review.openstack.org/52009114:31
apetrichweshay, aye I'm already working with him on it14:31
arxcruzfultonj: actually...14:32
arxcruzfultonj: 2017-11-08 21:19:54 |     2017-11-08 21:18:51,668 18813 INFO     [paramiko.transport] Connected (version 2.0, client dropbear_2012.55)14:32
arxcruzfultonj: so, it's the cloud-init that is not injecting the ssh key14:32
*** lblanchard has joined #tripleo14:33
arxcruzfultonj: http://paste.openstack.org/show/626387/14:33
fultonjarxcruz: nice find14:33
arxcruzfultonj: in the past we had this https://bugs.launchpad.net/tripleo/+bug/170225714:34
openstackLaunchpad bug 1702257 in tripleo "VM's cannot have ssh key injected because can't access metadata service" [Critical,Fix released]14:34
fultonjarxcruz: where is that log?14:34
* fultonj clicking around in http://logs.openstack.org/81/508881/2/gate/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-puppet/f9e4b6b/ 14:34
arxcruzfultonj: http://logs.openstack.org/79/516979/2/gate/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-container/13841da/logs/undercloud/home/zuul/tempest_output.log.txt.gz#_2017-11-08_21_19_5414:34
arxcruzfultonj: the one in the bug14:34
fultonjundercloud/home/zuul/tempest_output.log.txt not in http://logs.openstack.org/81/508881/2/gate/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-puppet/f9e4b6b/logs/undercloud/home/zuul/tempest/14:36
fultonjbut we have tempest.log.txt.gz14:36
*** dpawar has joined #tripleo14:36
fultonjis same?14:36
* fultonj looks14:37
fultonjarxcruz: so you think 1731063 is 1702257 reborn?14:39
fultonjseems that 1702257 wasn't solved but that it just went away, you think it's back?14:40
arxcruzfultonj: well, I'm setting up an environment on rdocloud to test, but according trown he has one yesterday and weren't able to reproduce, so, it might be a problem with the cloud as well14:40
honzajtomasek: what is the problem you were trying to solve?  It's not indicated in the commit message.14:40
arxcruzfultonj: because before was consistent14:40
arxcruznow it's not consistent14:41
arxcruzit pass some time14:41
*** liverpooler has joined #tripleo14:41
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs  https://review.openstack.org/51479314:41
fultonjalso, sorry but I can't find http://paste.openstack.org/show/626387 in http://logs.openstack.org/81/508881/2/gate/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-puppet/f9e4b6b/logs/undercloud/home/zuul/tempest/14:41
*** ratailor has quit IRC14:41
trownarxcruz: ya I made a new env this morning, already up to 8 in a row... I dont think we will recreate it on rdocloud14:41
*** lblanchard has quit IRC14:42
arxcruztrown: fultonj weshay so, what we know so far: it's the metadata service that is not injecting the ssh key, and that's why it's getting timeout, because ssh service is started after the metadata service finishes the env14:42
arxcruzit's not happening consistently14:43
arxcruzcan't be reproduced in rdocloud14:43
jtomasekhonza: that first patch's main goal is to make validations sidebar toggleable by trigger in navbar and change the validations sidebar responsive behaviour14:43
fultonjis the thought then to migrate the job to rdocloud with the thought that the hardware upgrade will improve chances of not hitting the timeout?14:43
weshaymwhahaha, ^14:44
*** udesale has joined #tripleo14:45
arxcruzfultonj: the timeout is because the metadata tries to connect for a few seconds, what were supposed to be a fraction of second14:45
*** tvignaud has quit IRC14:45
*** liverpooler has quit IRC14:45
arxcruzso, the test do: boot vm, when it's active, connect through ssh14:46
*** tvignaud has joined #tripleo14:46
arxcruzthe test assums that metadata service is faster, and was supposed to be14:46
arxcruzsince the ssh service in the vm spawned starts after metadata, and vm, even active still stucked there, we get the timeout14:46
jtomasekhonza: the goal of the whole patchset is to fix responsivity problems between validations sidebar, modal panels and rest of the content. make validations sidebar toggleable, ensure that there are no collisions between modal panel and modals and they can be freely nested14:47
honzajtomasek: +114:47
*** liverpooler has joined #tripleo14:47
fultonjthanks for figuring that out arxcruz , i updated the bug with some of this14:47
mwhahahaSounds like a performance regression14:48
*** lblanchard has joined #tripleo14:48
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs  https://review.openstack.org/51479314:48
jtomasekhonza: + add transitions to validations sidebar toggling, modal panel open/close, app content adjustments when modal panel or sidebar is open. -> note how modal panel and app content adjusts when window size changes when sidebar is open14:49
jtomasek(+ clicking modal backdrops closes it, as well as using esc key)14:50
fultonjas a hacky workaround can the tempest job sleep wait until after the metadata service finishes the env and then do the ssh attempt?14:50
*** gbarros has joined #tripleo14:50
honzajtomasek: see how that's too much? :)14:50
fultonjor the metadata service logs somewhere tempest can read so it knows it can now try the ssh14:51
*** noslzzp has quit IRC14:51
fultonjerr, the client of the metadata service is should say14:51
jtomasekhonza: I know, If I could merge one patch per day, the patches would look totally different. Problem is that instead of that I have 4 patches which I maintain for 2 months and keep rebasing (deeply) as I go and add stuff during those 2 months, while waiting for CI to ever pass:)14:52
openstackgerritMerged openstack/tripleo-ui master: Add message to spinner on Deployment Config modal  https://review.openstack.org/51947814:52
arxcruzfultonj: pretty sure is the metadata service :)14:52
*** noslzzp has joined #tripleo14:52
*** moshele has joined #tripleo14:52
honzajtomasek: I like the summaries in your commit messages because they give me a table of contents of sorts.  But what they often lack is the 'why'.14:53
fultonji agree, i'm just throwing ideas out on how to deal with its underperformance14:53
*** trown is now known as trown|brb14:53
jtomasekhonza: yeah, that's fair point. I think we should try to discuss stuff more on daily scrums, do you think that would help?14:54
*** anilvenkata has quit IRC14:54
honzajtomasek: Yes, it would certainly help me.  But that doesn't replace better commit messages (and I am as guity as anyone).14:54
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Add initial workflow tests for the stack.yaml workbook  https://review.openstack.org/52006214:55
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Test the create_default_deployment_plan workflow  https://review.openstack.org/52008514:55
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Test create_deployment_plan with a git repo as a source  https://review.openstack.org/52008614:55
jtomasekhonza: basically all of that work was to make validations sidebar properly toggleable and allow moving with nodes registration to modalPanel to be ready for nodes autoregistration (as we discussed some time ago) :) + dozen of bonus things which improve stuff14:56
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Start lines with timestamps in errors for logstash  https://review.openstack.org/52009614:56
*** etingof has joined #tripleo14:56
jtomasekhonza: ok, agree14:56
*** trown|brb is now known as trown14:57
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: Add noop-deploy-steps.yaml environment  https://review.openstack.org/52009715:01
shardylyarwood: ^^ untested but something like that15:01
shardymarios: FYI ^^ related to https://review.openstack.org/#/c/495658/15:01
dmsimardsshnaidm: (continuing discussion here because meeting in rdo)15:01
lyarwoodshardy: thanks15:02
dmsimardsshnaidm: I think the images we would end up using should be driven out of the promotion process15:02
dmsimardsshnaidm: so that the images we end up using are properly tested15:02
*** rhallisey_ is now known as rhallisey15:02
*** etingof has quit IRC15:03
shardylyarwood: if folks are OK with that approach I can propose pike/ocata/newton versions, which will use some slightly different resources I think15:03
openstackgerritHonza Pokorny proposed openstack/tripleo-ui master: Remove 'Refresh results' button on the nodes page  https://review.openstack.org/51547415:03
*** almondjoy has joined #tripleo15:03
sshnaidmdmsimard, well, there is a problem, because in jobs we use the newest tripleo packages, not from promotion. (and not tested well, actually)15:04
lyarwoodshardy: well, a newton version would be useful for FFU, and a pike version would be useful for upgrades15:04
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Update the .gitignore  https://review.openstack.org/51850615:04
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Correct the package name in the setup.cfg  https://review.openstack.org/52006115:04
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Add the tempest command check-uuid to tox.ini  https://review.openstack.org/52008415:04
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Add initial workflow tests for the stack.yaml workbook  https://review.openstack.org/52006215:04
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Test the create_default_deployment_plan workflow  https://review.openstack.org/52008515:04
openstackgerritDougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Test create_deployment_plan with a git repo as a source  https://review.openstack.org/52008615:04
lyarwoodshardy: outside of that I can't think of a reason we would need an Ocata version at the moment15:04
dmsimardsshnaidm: we can't realistically build new images (and make them used) every time there's a new commit, DLRN builds more than 100 commits every day15:04
shardylyarwood: ack, Ok let me take a first pass at a newton-only version but I'll mark it WIP.15:05
*** ramishra has quit IRC15:05
lyarwoodshardy: cool thanks, I've got an env ready to test it btw15:05
shardyI was just thinking we'd normally backport through all the branches15:05
lyarwoodshardy: right but we are going to need the Newton env in Queens for FFU15:06
shardylyarwood: Ok - when we start the process will we point --templates to the new queens tht?15:06
lyarwoodshardy: yes, that was the plan15:07
shardyif so we can just have a -newton version in the queens tht I guess15:07
lyarwoodshardy: ack, yeah, as queens tht contains the FFU logic15:07
lyarwoodor will at least15:08
sshnaidmdmsimard, we will build only when commit is merged into tripleo related repos, and even not to all of them. I think it's about 15-20 min of upgrading image15:08
mariosshardy: ack thanks will check it (sorry on a call right now)15:09
shardyHmm, thinking about it, just a queens version should be enough then, as the deploy stack update will update the stack to match the queens tht15:09
beaglesshardy, how do you reference a rendered port/network file for a resource registry entry in an environment file that is *outside* of the templates directory?15:09
shardybeagles: you can't j2 render anything outside the --templates directory15:09
sshnaidmdmsimard, and in case of promotion of course15:09
beaglesshardy, e.g. if I have /home/stack/work/foo.yaml and I want to specify something to have an external port15:10
shardybeagles: sorry you mean the env file is outside15:10
beaglesshardy, yeah15:10
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432815:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103215:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106315:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173214015:10
*** ooolpbot has quit IRC15:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)15:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]15:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]15:10
openstackLaunchpad bug 1732140 in tripleo "Kolla tests failing for tripleo-common master" [Critical,In progress] - Assigned to Adriano Petrich (apetrich)15:10
beaglesshardy, I figured it was just a matter of finding the right relative path, but I'm not sure what that would be :)15:10
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs  https://review.openstack.org/51479315:10
shardybeagles: Ah, yeah, you just specify the rendered file e.g http://paste.openstack.org/show/626394/15:11
*** links has joined #tripleo15:11
dmsimardsshnaidm: I'm too multithreaded right now, I'll get back to you asap15:11
sshnaidmdmsimard, ok15:11
*** dhill_ has quit IRC15:11
shardybeagles: the trick is to use an absolute path that matches --templates15:11
beaglesshardy, ah okay15:11
shardybeagles: we do relpath resolution so relative should work, but I heard of some issues with downstream testing around that15:12
beaglesshardy, abregman was having an issue with this and I was like "wait, what?" :)15:12
shardyI wasn't able to reproduce tho15:12
beaglesshardy, ack15:12
*** dhill_ has joined #tripleo15:12
shardybeagles: yeah I discussed it with abregman, I couldn't reproduce the error15:12
shardyI used that env file I just pasted, it worked OK15:12
shardyif you can reproduce problems I'm happy to help debug15:12
beaglesshardy, I've never had a problem so I never thought about it ...15:12
shardypossible we're missing a backport somewhere15:12
beaglesshardy, oh maybe so15:12
*** janki has quit IRC15:13
weshayrlandy|rover, adarazs|ruck fyi https://review.openstack.org/#/c/520007/2 https://review.openstack.org/#/c/520055 to address https://bugs.launchpad.net/tripleo/+bug/173093115:13
openstackLaunchpad bug 1730931 in tripleo "Github.com is used in TripleO CI for cloning distgit repos" [Critical,Triaged]15:13
*** pradk has joined #tripleo15:15
*** etingof has joined #tripleo15:16
honzajtomasek: I'm sorry about the late reviews :(15:16
jtomasekhonza: you're actually doing great with reviews, I really appreciate it!15:17
*** ffiore has quit IRC15:18
moshelebeagles: hi can you review https://review.openstack.org/#/c/520041/ and https://review.openstack.org/#/c/520042/15:20
beaglesmoshele, yep15:20
*** rbowen has joined #tripleo15:21
moshelebeagles: fixes  for SR-IOV that is broken now15:21
*** salmankhan has quit IRC15:22
moshelebeagles:  also if you can review the switchdev path for ovs hardware offload https://review.openstack.org/#/c/507100/. that will be great :)15:22
*** hjensas has quit IRC15:23
beaglesmoshele, yeah - sorry I've been meaning to get to them. I'll work at em now15:23
moshelebeagles: cool thank :)15:23
openstackgerritmathieu bultel proposed openstack/python-tripleoclient master: Add option for ceph ansible playbook  https://review.openstack.org/51543615:25
*** moshele has quit IRC15:25
dmsimardsshnaidm: ok so... bear with me here because I'm not super knowledgeable with TripleO. Right now we have build, test and promote undercloud images... right ? Now, we want to have an image for the overcloud and boot from that ? Or we want to boot from the undercloud image ? (or both!?)15:27
lyarwoodshardy: so the noop env works against my newton env as is15:28
EmilienMdmsimard: afik we don't build undercloud images15:28
openstackgerritOliver Walsh proposed openstack/tripleo-common master: Include required packages for libvirt SASL auth  https://review.openstack.org/52010715:28
EmilienMdmsimard: we build / promote overcloud images15:29
EmilienMbut I might have missed something15:29
dmsimardEmilienM: http://images.rdoproject.org/master/rdo_trunk/current-tripleo/ there's undercloud everywhere15:29
*** ffiore has joined #tripleo15:29
dmsimardunless they're named wrong15:29
openstackgerritOliver Walsh proposed openstack/tripleo-puppet-elements master: Include required packages for libvirt SASL auth  https://review.openstack.org/52010815:29
dmsimardor maybe it's just a tripleo-quickstart thing15:29
weshaydmsimard, the undercloud image itself is only there for legacy users of that image like opnfv15:29
weshayit's no longer consumed in upstream or downstream ci.. and we'll age it out and remove but need to give folks like trozet enough time to do that15:30
*** pblaho has joined #tripleo15:30
sshnaidmdmsimard, bluejeans..?15:32
trozetweshay, dmsimard: well we could switch to use overcloud image, but it needs to have the proper delorean repo in it so we can install packages that are missing like THT15:32
dmsimardsshnaidm: ok, after meetings in ~30 minutes15:32
shardylyarwood: Ok, nice!15:33
sshnaidmdmsimard, ok15:33
*** salmankhan has joined #tripleo15:33
dmsimardtrozet: what's stopping you from adding it in yourself ?15:33
*** stendulker has quit IRC15:33
shardylyarwood: I'm not entirely clear why the tripleoclient patch didn't work then, but whatever works :)15:33
trozetdmsimard: because we need the specific delorean repo used from the build, otherwise we have no way to know which repo to pull from packages from that was used to promote the images15:33
shardyslagle: Hey wanted to chat some more about config download, and if we'd be able to perhaps extend the approach from config-download-environment.yaml to enable large scale deployments (e.g $many computes)15:34
shardyslagle: the two ideas I had were either multiple compute stacks, to build up to hundreds/thousands of compute nodes, with each configured via config download15:35
weshaymatbu_, join https://bluejeans.com/42262127515:35
weshaymatbu_, need you in the escalation mtg15:35
*** tosky has quit IRC15:35
shardyslagle: or, possibly externally provisioned nodes that we wire in via a manually generated ansible inventory15:35
dmsimardtrozet: hmm, surely there's a hint in the image where it was built from.. if not, it should be easy to add one in15:35
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs  https://review.openstack.org/51479315:35
shardythe second approach would probably require getting all the config-download-environment bits ported to pure ansible though, right?15:36
trozetdmsimard: last time weshay and I spoke there wasnt, but I think he added a trello card or something to add the delorean build repo into the OC image, then we could switch to that15:36
weshayarxcruz, did you detect if volumebootpattern is hit in a particular cloud15:36
arxcruzweshay: not yet15:36
*** udesale has quit IRC15:37
slagleshardy: sure15:37
slagleshardy: what are you thinking, that Heat would not know about the thousands of computes?15:38
shardyslagle: yeah that was my first thought, but if we need heat then we could also deploy blocks of computes via multiple stacks, e.g a series of compute-only stacks?15:38
*** pblaho has quit IRC15:39
shardyslagle: the motivation is to work around some scale issues I'm aware have been apparent for very large environments15:39
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: debug unbound queries  https://review.openstack.org/52005515:39
openstackgerritmathieu bultel proposed openstack/tripleo-common master: Update ceph-ansible playbook path in parameter default for update  https://review.openstack.org/51544215:39
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Dynamically generate FQDN hiera entries  https://review.openstack.org/52008215:39
ccamachofolks can we merge this simple cherrypick in pike? https://review.openstack.org/#/c/519309/15:39
slagleshardy: i dont see why multiple stacks wouldn't work with what we have now15:39
ccamachothe dependency is already merged15:39
shardyslagle: Ok cool, I may give it a try then, and we could consider the pre-provisioned case after15:40
slagleshardy: well, i guess the configs that assume all hostnames (/etc/hosts configuration) would not be complete15:40
shardyslagle: ultimately it'd be cool to just do config download, then point ansible at an operator generated inventory15:40
slagleshardy: if you're using pre-provisioned nodes via deployed-server, they will still be in the inventory generated from tripleo-ansible-inventory15:41
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Dynamically generate FQDN hiera entries  https://review.openstack.org/52008215:41
shardyslagle: yeah, I was just thinking if we pre provision the nodes then configure via ansible, there might be an argument for doing it outside of heat15:41
shardythe existing deployed-server stuff would be fine as a first step though15:41
arxcruzweshay: what's the name of the project to run queries in command line ?15:42
slagleshardy: right. that's like longer term where I'd like to get to. Figure out how to use the ansible from config-download with no Heat15:42
matbu_weshay: its about rdo-cloud ?15:42
weshaymatbu_, ya.. rdo cloud upgrade15:43
slagleshardy: at a minimum, we might need to write an ansible role that can apply a nic config template15:43
slagleshardy: as well as one to parse Heat environments and set hieradata, etc15:44
weshaymatbu_, thank you :)15:44
slagleshardy: that's assuming we want to maintain the existing template interface15:44
shardyslagle: cool, yeah I was thinking we'd need to basically refactor the things which currently get run via group_vars incrementally, but I expect that will take a while15:45
shardyslagle: yeah would be nice if we could maintain the templates, even if it's only for a transition15:45
slagleshardy: yea, it's not so much about refactoring that stuff to be native ansible, as much as it is figuring out how to maintain the template interface + get ansible to do the orchestration things that are currently provided by Heat (picking IP's, etc)15:46
shardyslagle: yup, I was thinking we may want to ultimately say no more ipam, and that only FixedIPs are supported, which would make it easier15:47
openstackgerritPradeep Kilambi proposed openstack/instack-undercloud stable/pike: Fix panko ssl port  https://review.openstack.org/52011615:48
slagleshardy: yea, it would :). and would also help move away from neutron15:48
shardyslagle: the other idea I had was to stop merging all the role_data in heat, and instead write json files for each service on the host filesystem15:48
slagleas an uc dependency anyway15:48
shardye.g the outputs from all the service templates15:49
shardythen the templates could be more easily converted into ansible roles via some tool15:49
shardye.g a bunch of roles which just write the per-service data15:49
shardythen some other roles that do the configuration15:49
shardywe could do that while still maintaining the heat interfaces I think, but it'd make a switch to ansible roles easier15:49
openstackgerritDerek Higgins proposed openstack/tripleo-docs master: Add details about boot baremetal from cinder volume  https://review.openstack.org/52005415:50
shardytools/services-to-roles.py ;)15:50
slaglemakes sense15:50
shardyslagle: Ok thanks for the feedback, provided nobody is already looking into this I'll spend some time on it15:51
openstackgerritLuke Hinds proposed openstack/puppet-tripleo master: Introduces puppet module for `/etc/login.defs`  https://review.openstack.org/45798915:51
slagleshardy: np. i think if we can get the initial pass of the config-download workflow landed, and ansible driving all the transport of configuration, then it will make these follow on steps easier to implement/adopt15:52
slaglethat's kind of why i started the way I did anyway :)15:53
shardyslagle: ack yeah agreed, I've been mostly testing locally with config download for a while now15:53
slaglenot trying to get to ansible-nirvana in the first pass :)15:53
shardyhehe :)15:53
*** stendulker has joined #tripleo15:55
weshaymwhahaha, https://trello.com/c/UURXfoxO15:55
*** dhill_ has quit IRC15:56
openstackgerritMichele Baldessari proposed openstack/puppet-tripleo master: WIP Make sure rabbitmq is fully up before creating any rabbitmq resources  https://review.openstack.org/52011915:56
openstackgerritMichele Baldessari proposed openstack/tripleo-heat-templates master: WIP DNR quick test  https://review.openstack.org/52012015:59
*** ykarel has quit IRC15:59
*** itlinux has joined #tripleo16:01
*** dhill_ has joined #tripleo16:02
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates master: Run Octavia configuration on the overcloud  https://review.openstack.org/50819516:03
openstackgerritBrent Eagles proposed openstack/tripleo-heat-templates master: Add Octavia to scenario002 (containers only)  https://review.openstack.org/51833116:03
*** dsneddon has joined #tripleo16:06
openstackgerritFlorian Fuchs proposed openstack/tripleo-validations stable/pike: Add missing module documentation  https://review.openstack.org/52012316:07
openstackgerritEmilien Macchi proposed openstack/tripleo-ui master: CI test - never merge  https://review.openstack.org/52012416:07
lyarwoodshardy: urgh, so in another env I'm seeing the same --setup-heat-outputs error again - http://paste.openstack.org/show/626414/16:08
trownsshnaidm: I think we must not have a job for https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset026.yml ... do you know?16:08
trownsshnaidm: reason I say that is that we have a non-container scenario in that config, but container args16:09
openstackgerritJustin Kilpatrick proposed openstack/tripleo-quickstart-extras master: Add stack update role  https://review.openstack.org/52012516:09
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432816:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103216:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106316:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173214016:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)16:10
*** ooolpbot has quit IRC16:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]16:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]16:10
openstackLaunchpad bug 1732140 in tripleo "Kolla tests failing for tripleo-common master" [Critical,In progress] - Assigned to Adriano Petrich (apetrich)16:10
shardylyarwood: have you configured a nameserver so the overcloud nodes can reach pool.ntp.org?16:10
shardyntpdate[180526]: no server suitable for synchronization found16:11
shardyhttps://docs.openstack.org/tripleo-docs/latest/install/basic_deployment/basic_deployment_cli.html#configure-a-nameserver-for-the-overcloud16:11
sshnaidmtrown, acc. to project-config we have it in experimental16:12
shardyI'd probably try running ntpdate on the node to be sure?16:12
lyarwoodshardy: hmmm odd, this is an oooq virt env so I assumed that was already taken care of16:13
* lyarwood looks16:13
trownsshnaidm: hmm do we have any recent runs of it?16:13
openstackgerritmathieu bultel proposed openstack/python-tripleoclient master: Add option for ceph ansible playbook  https://review.openstack.org/51543616:13
sshnaidmtrown, I don't see it in status page, seems like not16:14
lyarwoodah, wait, don't we block non RH NTP within the corp network or something16:14
trownsshnaidm: hmm ok... does look like https://github.com/openstack/tripleo-heat-templates/blob/master/ci/environments/scenario006-multinode.yaml is actually containers, but looks like it needs 2nodes multinode job16:15
sshnaidmtrown, well, I don't track it in sova..16:15
openstackgerritJustin Kilpatrick proposed openstack/tripleo-quickstart-extras master: Add disruption detector for Overcloud endpoint monitoring  https://review.openstack.org/52012616:15
trownsshnaidm: I am convinced that job could not pass as is16:15
sshnaidmtrown, most likely it doesn't, just enabled its tracking, we'll see a bit later its runs16:16
trownsshnaidm: it will run if I do "check experimental" on a tripleo-ci patch?16:16
sshnaidmtrown, should run16:16
trownk, will give that a go16:16
trownbtw fs19 is fixed by the same solution as 1616:17
sshnaidmtrown, but it may run only of specific file changes, better to run it on change of its featureset16:17
sshnaidmtrown, and in specific repo..16:17
trownsshnaidm: ok I will put up a patch to THT that touchs the scenario16:18
trownsshnaidm: if that does not run the job... it should16:18
sshnaidmtrown, yeah, should work16:19
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs  https://review.openstack.org/51479316:19
openstackgerritJohn Trowbridge proposed openstack/tripleo-heat-templates master: DNM testing scenario006-multinode  https://review.openstack.org/52012816:20
*** dparkes has quit IRC16:24
*** rbowen has quit IRC16:24
*** tosky has joined #tripleo16:25
*** shreshtha has quit IRC16:25
*** aufi has quit IRC16:29
pradkmwhahaha, can we fast merge this into pike https://review.openstack.org/#/c/520116/ .. need it downstream asap16:32
mwhahahapradk: k16:33
sshnaidmmyoung, ping16:33
pradkmwhahaha, ty sir!16:33
myoungsshnaidm: o/16:33
sshnaidmmyoung, hi, do you know maybe why downstream jenkins save empty console file?16:34
sshnaidmmyoung, sova can't parse it and fails..16:34
myoungsshnaidm: I don't, will have a look now16:34
myoungsshnaidm: (to confirm) you're looking at thirdpartylogs right?16:34
openstackgerritOliver Walsh proposed openstack/tripleo-common master: Generate password for libvirt TLS SCRAM auth  https://review.openstack.org/52013516:35
sshnaidmmyoung, yeah, https://thirdparty.logs.rdoproject.org/jenkins-tqe-gate-rhos-11-current-passed-ci-ci-rhos-ovb-minimal-pacemaker-public-bond-279/16:35
*** ukalifon has quit IRC16:36
openstackgerritJuan Badia Payno proposed openstack/tripleo-common master: Fluentd: rename fluentd service  https://review.openstack.org/52001116:36
*** rbowen has joined #tripleo16:38
*** mrunge_ has joined #tripleo16:39
openstackgerritJuan Badia Payno proposed openstack/tripleo-heat-templates master: logging: merge fluentd-client and fluentd-base  https://review.openstack.org/50145716:39
*** mrunge has quit IRC16:39
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: add config to set neutron mtu in undercloud/overcloud  https://review.openstack.org/50966016:41
lyarwoodshardy: yeah sorry about that, this env was within the corp network so we can't use an external host for ntp16:41
*** hjensas has joined #tripleo16:41
*** ebarrera has quit IRC16:42
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Add MTU adjustment parameters to RDO Cloud settings  https://review.openstack.org/51948616:43
*** jpich has quit IRC16:44
*** moshele has joined #tripleo16:45
*** yamahata has quit IRC16:46
*** gbarros has quit IRC16:47
*** abregman has quit IRC16:49
EmilienMweshay: any progress on https://review.openstack.org/#/c/517444/ ?16:50
*** lucasagomes is now known as lucas-afk16:52
openstackgerritSteven Hardy proposed openstack/tripleo-docs master: Clarify composable services docs  https://review.openstack.org/52014616:54
*** rbowen has quit IRC16:57
*** ffiore has quit IRC16:58
*** dprince has quit IRC16:59
*** links has quit IRC17:00
*** dpawar has quit IRC17:01
*** psachin has quit IRC17:02
*** tesseract has quit IRC17:03
*** absubram has quit IRC17:03
myoungsshnaidm, rlandy|rover, adarazs|ruck: https://etherpad.openstack.org/p/no-console-logs-doh17:04
*** akane has quit IRC17:05
*** gbarros has joined #tripleo17:05
*** agurenko has quit IRC17:07
openstackgerritGiulio Fidente proposed openstack/instack-undercloud master: Do not set dhcp_domain in Nova from overcloud_domain_name  https://review.openstack.org/52015217:09
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432817:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103217:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106317:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173214017:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)17:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]17:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]17:10
openstackLaunchpad bug 1732140 in tripleo "Kolla tests failing for tripleo-common master" [Critical,In progress] - Assigned to Adriano Petrich (apetrich)17:10
*** yprokule has quit IRC17:10
*** d0ugal has quit IRC17:12
openstackgerritGiulio Fidente proposed openstack/instack-undercloud master: Do not set dhcp_domain in Nova from overcloud_domain_name  https://review.openstack.org/52015217:19
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Send graphite stats from ARA  https://review.openstack.org/48012117:20
*** dpawar has joined #tripleo17:23
*** dpawar has quit IRC17:24
*** dpawar has joined #tripleo17:24
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: Revert "Revert "Tag workflows created by the templates""  https://review.openstack.org/50376217:24
*** marios has quit IRC17:28
*** florianf has quit IRC17:29
*** cylopez has quit IRC17:31
weshayEmilienM, so.. in CI post-deployment is hanging on the containerized undercloud http://logs.openstack.org/41/519441/7/check/legacy-tripleo-ci-centos-7-undercloud-containers/0baa424/logs/undercloud/home/zuul/undercloud_install.log.txt.gz17:31
weshaymandre, you around?17:31
*** mcornea has quit IRC17:35
weshayEmilienM, fyi that was for the current undercloud-containers job fs27;  the updated way.. I need help/eyes w/ http://logs.openstack.org/18/518118/3/check/legacy-tripleo-ci-centos-7-undercloud-oooq/c56e673/logs/undercloud/home/zuul/undercloud_install.log.txt.gz17:36
*** gkadam has quit IRC17:37
*** moshele has quit IRC17:37
*** anilvenkata has joined #tripleo17:38
openstackgerritMerged openstack/tripleo-ui stable/pike: Fix Validations progress  https://review.openstack.org/52006917:41
*** masco has quit IRC17:42
*** akane has joined #tripleo17:42
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: DNM, update logging for undercloud containers  https://review.openstack.org/51897217:42
*** akane has quit IRC17:43
*** rbowen has joined #tripleo17:43
*** dpawar has quit IRC17:44
itlinuxhello guys.. I am using LDAP and when logging in (Horizon) that is pretty slow.. local account are way faster.. any tips? TY17:45
*** holser has quit IRC17:45
*** dtantsur is now known as dtantsur|afk17:46
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Upgrade UC and OC using tripleo-upgrade role  https://review.openstack.org/51564317:47
openstackgerritSven Anderson proposed openstack/tripleo-quickstart master: TEST DON'T MERGE - Enabling EC2-API Tempest tests.  https://review.openstack.org/51513917:48
*** shardy has quit IRC17:48
*** stendulker has quit IRC17:50
*** nyechiel has quit IRC17:50
*** holser has joined #tripleo17:50
*** dougbtv_ has joined #tripleo17:51
*** akane has joined #tripleo17:53
*** holser has quit IRC17:55
*** rhallisey has quit IRC17:58
*** masco has joined #tripleo17:58
*** derekh has quit IRC17:59
*** atoth has quit IRC18:00
*** fragatina has joined #tripleo18:01
*** fragatina has quit IRC18:01
openstackgerritLokesh Jain proposed openstack/tripleo-heat-templates stable/newton: Addition of Nuage bridge MTU parameter  https://review.openstack.org/52016318:01
*** fragatina has joined #tripleo18:02
*** jfrancoa has quit IRC18:02
openstackgerritMerged openstack/tripleo-heat-templates stable/pike: Enable horizon plugins by default in docker  https://review.openstack.org/51944518:02
*** yamahata has joined #tripleo18:02
*** bobh has joined #tripleo18:05
*** chlong has quit IRC18:06
*** fzdarsky is now known as fzdarsky|afk18:06
*** dpawar has joined #tripleo18:08
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432818:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103218:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106318:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173214018:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)18:10
*** ooolpbot has quit IRC18:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]18:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]18:10
openstackLaunchpad bug 1732140 in tripleo "Kolla tests failing for tripleo-common master" [Critical,In progress] - Assigned to Adriano Petrich (apetrich)18:10
*** dprince has joined #tripleo18:11
*** pcaruana has quit IRC18:11
*** dougbtv_ has quit IRC18:14
*** rbowen has left #tripleo18:15
*** fragatina has quit IRC18:15
*** jpena is now known as jpena|off18:16
mnaserEmilienM: left a few comments on the openstack tripleo-ci migration changes18:19
*** masco has quit IRC18:19
*** akane has quit IRC18:21
*** rhallisey has joined #tripleo18:23
*** dpawar has quit IRC18:25
*** trown is now known as trown|lunch18:26
*** salmankhan has quit IRC18:30
*** mrunge_ is now known as mrunge18:34
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements  https://review.openstack.org/51918918:34
*** fragatina has joined #tripleo18:34
*** tosky has quit IRC18:34
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Add subnet property to ctlplane network for server resources  https://review.openstack.org/47381718:36
*** dsariel has quit IRC18:37
*** cylopez has joined #tripleo18:38
*** dpawar has joined #tripleo18:39
*** akane has joined #tripleo18:39
*** paramite has quit IRC18:41
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: DNM: Update undercloud install options for containers  https://review.openstack.org/51744518:41
openstackgerritwes hayutin proposed openstack/instack-undercloud master: DNM, TESTING ONLY  https://review.openstack.org/51811818:41
*** pradk has quit IRC18:42
*** akane has quit IRC18:42
*** dpawar has quit IRC18:45
*** dsariel has joined #tripleo18:49
fultonjThis change adds comments to a file that isn't even tested by CI https://review.openstack.org/#/c/518650/ but CI is failing because of https://bugs.launchpad.net/tripleo/+bug/1731063 (which has a 20% error)  I can recheck it until green before it gets +w'd or should it just be +w'd to save resources?18:50
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]18:50
fultonjmwhahaha: ^ ?18:52
weshaytrown|lunch, please ping me w/ any updated news on fs1618:52
mwhahahafultonj: wait until we switch the job to nonvoting18:54
weshayarxcruz, any news on fs16?18:54
*** itlinux has quit IRC18:54
mwhahahafultonj: the problem is the impact of the failure in the gate causing backups18:54
fultonjmwhahaha: ok18:55
*** thrash is now known as thrash|biab18:58
arxcruzweshay: not yet19:01
*** tosky has joined #tripleo19:03
*** pradk has joined #tripleo19:06
EmilienMmnaser: ok thx19:06
*** atoth has joined #tripleo19:06
openstackgerritMerged openstack/tripleo-common stable/pike: Do not update the container registry if is None  https://review.openstack.org/51928719:08
openstackgerritMerged openstack/puppet-tripleo master: Add profile to configure the rsyslog sidecar container  https://review.openstack.org/51925619:08
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432819:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103219:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106319:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173214019:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]19:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]19:10
openstackLaunchpad bug 1732140 in tripleo "Kolla tests failing for tripleo-common master" [Critical,In progress] - Assigned to Adriano Petrich (apetrich)19:10
*** rodolof has joined #tripleo19:12
*** moshele has joined #tripleo19:13
*** mwhahaha changes topic to "CI status: MASTER: YELLOWish, STABLE/*: GREENish, see alerts and read ML. scenario001 is non-voting :( | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest/"19:15
*** moshele has quit IRC19:16
*** pradk_ has joined #tripleo19:17
*** pradk has quit IRC19:19
*** paramite has joined #tripleo19:20
*** d0ugal has joined #tripleo19:21
*** trown|lunch is now known as trown19:21
trownweshay: at a loss on how to actually reproduce the problem... my script is on run 26 today without failing once19:22
weshaytrown, k19:23
openstackgerritManuel Buil proposed openstack/diskimage-builder master: Add other OS to the documentation  https://review.openstack.org/51896819:24
*** rwsu has quit IRC19:24
EmilienMweshay: sorry back. So you need a bit of help, let me see the link19:24
EmilienM2017-11-13 20:03:59 |         "Error: Could not run: Could not find file /var/lib/tripleo-config/puppet_step_config.pp"19:25
weshayEmilienM, which one do you want to look at? getting undercloud-containers back to voting, or moving the standard undercloud setup to use containers?19:25
weshayEmilienM, ya /me just updated the logging to pull in that dir19:25
weshayEmilienM, https://review.openstack.org/#/c/518972/2/toci-quickstart/config/collect-logs.yml19:26
EmilienMweshay: well, 2) comes after 1)19:26
EmilienMfirst we need something stable19:26
weshayEmilienM, aye19:26
EmilienMand we need to use the correct workflow19:26
weshayEmilienM, so that works for me locally, but hangs in CI at http://logs.openstack.org/41/519441/7/check/legacy-tripleo-ci-centos-7-undercloud-containers/0baa424/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2017-11-15_14_55_0419:27
weshayEmilienM, if you want to log into my working instance let me know19:28
EmilienMweshay: let me finish my current patchset for tripleo-ci/zuulv319:28
EmilienMweshay: and I'm yours :P19:28
* weshay goes and puts bells on19:29
*** itlinux has joined #tripleo19:30
openstackgerritKeith Schincke proposed openstack/puppet-tripleo master: Update cephx keys with ACLs for openstack services.  https://review.openstack.org/51953119:35
*** Lokesh_Jain__ has quit IRC19:36
*** cylopez has quit IRC19:38
arxcruzweshay: it doesn't seems to be a node provider problem...19:40
*** pradk_ is now known as pradk19:40
arxcruzweshay: http://paste.openstack.org/show/626433/19:41
arxcruzweshay: the failure is very spread19:41
weshayhrm.. 100% on master though19:41
weshayarxcruz, what is the query19:41
arxcruzweshay: http://paste.openstack.org/show/626435/19:42
weshayarxcruz, thank you19:43
*** dprince has quit IRC19:45
arxcruzweshay: maybe worth involve someone from network ?19:46
mwhahahagfidente, fultonj: is this a known issue on ovb? http://logs.openstack.org/99/509199/7/check-tripleo/legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024/03fa13f/logs/undercloud/var/log/mistral/ceph-install-workflow.log.txt.gz#_2017-11-13_19_45_43_165 i couldn't find why docekr wasn't running19:47
*** chlong has joined #tripleo19:48
gfidentemwhahaha I think that job isn't using docker19:48
*** chlong is now known as chlong|lnch19:48
mwhahahagfidente: did we fix that in the last 2 days?19:48
gfidentemwhahaha that job should be fixed by https://review.openstack.org/#/c/519301/19:48
gfidenteyes19:48
mwhahahaah ok19:48
mwhahahaah so known issue, thanks19:48
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs  https://review.openstack.org/51479319:51
*** marrusl has joined #tripleo19:57
*** amoralej is now known as amoralej|off19:59
weshayarxcruz, wouldn't hurt19:59
weshayarxcruz, were you also able to recreate fs16?20:00
arxcruzbecause the metadata service isn't being reached20:00
arxcruzweshay: working on that right now20:00
weshayarxcruz++20:00
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo master: zuul: switch to unit tests job template  https://review.openstack.org/52018920:01
*** gfidente has quit IRC20:02
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs  https://review.openstack.org/51479320:06
*** jprovazn has quit IRC20:06
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: DNM, update logging for undercloud containers  https://review.openstack.org/51897220:07
mwhahahaweshay: arxcruz: so those stats were only for the non-containerized scenario001, are the stats the same for the containerized version?20:08
weshayarxcruz, ya actually we want three jobs listed there20:09
arxcruzweshay: do you have the job names ?20:09
weshayarxcruz, the scen001 jobs from http://cistatus.tripleo.org:8000/20:10
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432820:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103220:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106320:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173214020:10
*** ooolpbot has quit IRC20:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]20:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]20:10
openstackLaunchpad bug 1732140 in tripleo "Kolla tests failing for tripleo-common master" [Critical,In progress] - Assigned to Adriano Petrich (apetrich)20:10
weshayarxcruz, so add the puppet and containers job too20:10
mwhahahait's just two i think20:10
mwhahahalegacy-tripleo-ci-centos-7-scenario001-multinode-oooq-container and  legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-puppet20:11
weshaymwhahaha, shitty thing is that our jobs are at 95% success rate and we're still dying20:11
mwhahahait just takes one late failure to drop the queue to 6+ hours20:11
weshaymwhahaha, that is assuming just master20:11
weshaymight as well include egacy-tripleo-ci-centos-7-scenario001-multinode-oooq for old branches20:11
mwhahahaah20:12
mwhahahayea20:12
arxcruzmwhahaha: weshay I believe we should check with someone from network, because we already know the root cause, the vm can't reach the metadata service and so, it doesn't inject the key and the ssh service start late20:13
mwhahahaarxcruz: but we use the same network config between 001 & 002 right?20:13
weshayarxcruz, rock.. so we do that via the card on the escalation board20:13
weshayif you can add the reason why.. to the card/bug we'll go grab someone20:14
arxcruzmwhahaha: I don't know the details...20:14
mwhahahahttps://github.com/openstack/tripleo-heat-templates/blob/master/README.rst#service-testing-matrix20:14
arxcruzweshay: http://paste.openstack.org/show/626440/20:14
arxcruzthe query is http://paste.openstack.org/show/626441/20:15
weshaymwhahaha, def a regression20:15
* mwhahaha hates being right20:15
weshaydo you hate yourself often?20:15
mwhahahayes, i am full of self loathing20:16
weshay:)20:16
weshayjust bash EmilienM when you feel crappy.. works for me20:16
EmilienMlol20:16
mwhahahahe taunts me with pictures of his lunch20:16
*** cylopez has joined #tripleo20:17
EmilienMI'm having https://en.wikipedia.org/wiki/Tartiflette20:17
*** cylopez has quit IRC20:17
openstackgerritDan Prince proposed openstack/tripleo-quickstart-extras master: Don't try to start docker for the UC  https://review.openstack.org/51944120:19
*** itlinux has quit IRC20:19
arxcruzmwhahaha: weshay if you check the card, there's an error also in phase2 same thing...20:22
rlandy|rovermwhahaha: we need another core reviewer on the following changes to clean up promotion blockers: https://review.openstack.org/#/c/513820/ ... https://review.openstack.org/#/c/519770/20:23
weshayarxcruz, is it the same thing?20:23
arxcruzweshay: yup20:23
weshayI haven't had a sec to look20:23
weshayarxcruz, ok.. /me looks now20:24
weshayarxcruz, thanks20:24
*** thrash|biab is now known as thrash20:24
*** d0ugal_ has joined #tripleo20:25
*** d0ugal has quit IRC20:26
*** paramite has quit IRC20:29
arxcruzweshay: add some detailed comments in the card20:31
weshayarxcruz, can you submit that elastic recheck query please20:31
arxcruzweshay: removing the build_name right ?20:32
weshayarxcruz, mwhahaha just set the job to non-voting.. it would be great if recheck took care of identifying the issue for folks20:32
weshayarxcruz, I think we can keep the build names in it 3 jobs20:32
mwhahahathat looks like a different error downstream20:32
mwhahahaconnection reset by peer20:32
arxcruzweshay: ok, I'm adding as bug #1731063 ok ?20:32
openstackbug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged] https://launchpad.net/bugs/173106320:32
*** jbadiapa_ has joined #tripleo20:33
arxcruzmwhahaha: same issue20:33
arxcruz2017-11-15 10:01:17,513 28453 INFO     [paramiko.transport] Connected (version 2.0, client dropbear_2012.55)20:33
arxcruz2017-11-15 10:01:17,607 28453 INFO     [paramiko.transport] Authentication (publickey) failed.20:33
mwhahahaarxcruz: no the network reason is different20:34
*** holser has joined #tripleo20:34
mwhahahadownstream is connection reset by peer20:34
weshaymwhahaha, that should be a release blocker for 1220:34
weshayimho20:35
mwhahahait's a different issue20:35
mwhahahasame test failure, different cause20:35
mwhahahaupstream is a no key so it's not having connectivity issues to the vm20:35
*** rwsu has joined #tripleo20:35
mwhahahadownstream was connectivity problems to the vm20:35
*** jbadiapa has quit IRC20:36
weshayah20:38
arxcruzmwhahaha: yeah, you're right20:39
arxcruzdownstream are being able to connect20:39
arxcruzwell, honestly, i have no idea how to fix it...20:39
*** itlinux has joined #tripleo20:39
arxcruzand since we can't reproduce on rdocloud, it's more harder to check20:40
*** d0ugal_ has quit IRC20:40
*** jmelvin has quit IRC20:40
*** d0ugal has joined #tripleo20:40
*** d0ugal has quit IRC20:40
*** d0ugal has joined #tripleo20:40
*** gbarros has quit IRC20:42
*** etingof has quit IRC20:46
pradkmwhahaha, hmm did we remove tripleoupstream containers from docker hub? ./overcloud-prep-containers.sh is failing with Tag current-tripleo-rdo not found in repository docker.io/tripleoupstream/20:46
mwhahahaaren't we using rdo now?20:46
pradkhmm quickstart seems to default to this20:47
mwhahahaweshay, trown -^ where'd the containers go?20:47
mwhahahadocs revert to rdoproject https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/overcloud.html#preparing-the-environment20:47
mwhahahas/revert/refer20:48
trownmwhahaha: pradk this is local or in ci?20:49
pradktripleo-quickstart-extras/roles/extras-common/defaults/main.yml:docker_registry_host: docker.io20:49
pradklocal20:49
trownfor jobs that run in upstream infra we have to use dockerhub, so they can mirror it20:49
trownfor everything else we should use rdoproject20:49
pradki did a quickstart with minimal_container.yml and on undercloud the ./overcloud-prep-containers.sh point to docker.io20:49
*** bogdando has joined #tripleo20:50
trownpradk: hmm k... I thought there was a patch for that... looking20:50
trownpradk: and this is libvirt?20:52
pradkyes20:52
*** d0ugal has quit IRC20:52
trownpradk: looks like we do need to add the vars to https://github.com/openstack/tripleo-quickstart/blob/master/config/environments/default_libvirt.yml that are in https://github.com/openstack/tripleo-quickstart-extras/blob/master/config/environments/rdocloud.yml#L30-L3120:52
openstackgerritDan Prince proposed openstack/tripleo-quickstart-extras master: Don't try to start docker for the UC  https://review.openstack.org/51944120:53
pradktrown, i guess this needs to be updated too? https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/extras-common/defaults/main.yml#L820:54
*** gbarros has joined #tripleo20:54
weshaypradk, mwhahaha sec.. in 1-120:54
trownpradk: ya that is the other way... update the defaults, and make sure our CI is correctly overrding where needed20:55
trownpradk: in any case, it must be... because those defaults are wrong20:55
pradkok cool, i can push a patch with update20:56
trownpradk: thanks20:58
openstackgerritPradeep Kilambi proposed openstack/tripleo-quickstart-extras master: Update the default docker registry to rdoproject  https://review.openstack.org/52020120:59
*** itlinux has quit IRC21:00
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Drop step_config as top level docker requirement  https://review.openstack.org/44271621:02
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: docker: Add unit tests on service_name  https://review.openstack.org/44275521:02
*** bogdando has quit IRC21:04
*** Goneri has quit IRC21:06
*** holser has quit IRC21:07
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432821:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103221:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106321:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173214021:10
*** ooolpbot has quit IRC21:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]21:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]21:10
openstackLaunchpad bug 1732140 in tripleo "Kolla tests failing for tripleo-common master" [Critical,In progress] - Assigned to Adriano Petrich (apetrich)21:10
*** jmelvin has joined #tripleo21:11
pradktrown, whats the namespace we use in rdo registry? tripleoupstream doesnt seem to work21:15
mwhahahatripleomaster21:15
mwhahahaand tripleopike i think21:15
pradkok trying21:15
openstackgerritMerged openstack/puppet-tripleo master: Add TLS for ec2api metadata service  https://review.openstack.org/51177221:17
jristmore eyeballs plz https://review.openstack.org/#/c/514795/21:18
*** itlinux has joined #tripleo21:19
*** dhill_ has quit IRC21:20
*** liverpooler has quit IRC21:20
trownpradk: ya was going to comment on the review that we should default that to tripleopike21:20
*** dhill_ has joined #tripleo21:21
pradkso the tag we're defaulting to is current-tripleo-rdo .. assume thats not right either?21:21
*** jtomasek has quit IRC21:21
mwhahahajrist: https://review.openstack.org/#/c/514794/ seems to be the one that needs more eyes :D21:21
jristfair.21:22
ansiwenshouldn't scenario002 contain ec2-api in the services? why is it missing here? http://logs.openstack.org/39/515139/3/check/legacy-tripleo-ci-centos-7-scenario002-multinode-oooq-container/2f68dda/logs/undercloud/home/zuul/tempest_output.log.txt.gz#_2017-11-15_20_04_0621:22
jristthanks mwhahaha21:22
mwhahahaand https://review.openstack.org/#/c/514757/21:22
mwhahahaansiwen: does tempestconf auto pickup ec2api?21:24
pradktrown, hmm still seems to fail to pull .. imagename: trunk.registry.rdoproject.org/tripleopike/centos-binary-aodh-api:latest21:25
pradkdocker pull failed: Error: image tripleopike/centos-binary-aodh-api:latest not found21:25
ansiwenmwhahaha: as I understand the script would at least list all services if they are there... maybe I misread the code though21:25
trownpradk: ah ya tag latest is not a thing also21:27
mwhahahaansiwen: thats just tempest config getting printed out. i don't think that it autodiscovers it. i thought we used tempestconf to fill it out21:27
trownpradk: but that should be coming from release21:27
ansiwenmwhahaha: you probably think I have to add ec2 here: https://github.com/redhat-openstack/python-tempestconf/blob/master/config_tempest/config_tempest.py#L8321:27
trownpradk: ie https://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/pike.yml#L2021:28
ansiwenmwhahaha: but when I read the code it looked to me that it should still detect it. let me check again...21:28
mwhahahaansiwen: it's possible. you might check with arxcruz21:28
openstackgerritMerged openstack/tripleo-heat-templates master: Enable TLS for ec2api metadata service  https://review.openstack.org/51177321:29
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs  https://review.openstack.org/51479321:29
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Migrate THT to zuul v3 jobs  https://review.openstack.org/51976121:29
ansiwenmwhahaha: this run depends on my extended python-tempestconf btw (I hope Depends-On also works for python-tempestconf?)21:29
mwhahahait should21:29
arxcruzansiwen: I believe it doesn't get ec2api yet21:30
arxcruzwe are planning a huge refactoring in the code21:30
arxcruzfor those cases that tempestconf don't detect, you need to explicitly add the configuration21:30
ansiwenmwhahaha: where can I double check again, that ec2 api is really deployed in scenario002? I always get lost in the myriads of CI repositories21:30
pradktrown, ah ok .. hmm so i guess current-tripleo would be the default then?21:31
pradktrown, or should we just set it to whats in pike as default21:31
ansiwenarxcruz: on what does it depend, if it detects it automatically?21:31
mwhahahaansiwen: that part is most likely in tripleo-quickstart in the scenario config for that21:31
arxcruzmwhahaha: ansiwen: also, python-tempestconf is now under openstack umbrella https://github.com/openstack/python-tempestconf21:31
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Add various functionality to nodepool role  https://review.openstack.org/51856021:31
*** threestrands has joined #tripleo21:32
*** threestrands has quit IRC21:32
*** threestrands has joined #tripleo21:32
ansiwenarxcruz: yes, this I found out already21:32
pradktrown, well, current_tripleo isnt working either.. is there a place i can look for existing tags in rdo registry21:32
trownpradk: current-tripleo is a sane default for that... you are not useing a release file?21:32
trownpradk: current_tripleo or current-tripleo?21:32
arxcruzansiwen: for scenarios that run services that are not being detected by tempestconf, we add it manually, let me check an example for you21:32
mwhahahaansiwen: asctually THT/ci/enviurnments/scenario002-multinode-containers21:33
pradktrown, current-tripleo .. docker pull failed: Error: image tripleopike/centos-binary-aodh-api:current-tripleo not found21:33
*** aputtur_ has quit IRC21:34
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Migrate THT to zuul v3 jobs  https://review.openstack.org/51976121:34
arxcruzansiwen: https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset019.yml#L9121:35
*** ansmith has quit IRC21:35
ansiwenarxcruz: no, I have to add it to tempestconf, becauce I have to create ec2 credentials with the openstack client. I don't think I can do that just within quickstart21:35
trownpradk: ugh... right they are in there by the actual hash... thought we had named tags too... but we have code in quickstart that turns the named link (current-tripleo) into a hash (ca0834685d9bf226816e49b195251bd553fe5d9f_c74c332d) ... https://github.com/openstack/tripleo-quickstart/blob/master/roles/repo-setup/tasks/get-dlrn-hash.yml21:36
arxcruzansiwen: cool, if you have a patch, I can review that for you :)21:36
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Migrate THT to zuul v3 jobs  https://review.openstack.org/51976121:36
trownpradk: weshay has shown me a UI for the rdoproject registry... but I have never been able to log in21:36
ansiwenarxcruz: I have to do something equivalent to the commend "openstack ec2 credentials create"...21:36
pradktrown, hmm ok21:36
ansiwenarxcruz: the patch is up there already since two days21:37
arxcruzansiwen: can you show me ?21:37
ansiwenarxcruz: https://review.openstack.org/#/c/519714/21:37
*** ecerquei has quit IRC21:37
ansiwenarxcruz: but I found a problem already... let me fix it21:37
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Migrate THT to zuul v3 jobs  https://review.openstack.org/51976121:37
openstackgerritMerged openstack/puppet-tripleo stable/pike: Add TLS for ec2api service  https://review.openstack.org/51961021:40
mwhahahasdoran: you around?21:40
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs  https://review.openstack.org/51479321:40
sdoranYup.21:41
*** trown is now known as trown|outtypewww21:41
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Migrate THT to zuul v3 jobs  https://review.openstack.org/51976121:41
mwhahahasdoran: is there an established patern for delivering ansible modules via separate rpms?21:41
sdoranNope.21:41
* mwhahaha facepalms21:41
*** itlinux has quit IRC21:42
sdoranCurrently distributing custom modules are done via your playbook repo or inside a role.21:42
sdoranNot great — we're working on better ways to distribute/share all plugin types, not just modules.21:42
sdoranBut the way most folks do it today is inside a role and listing that role as a dep on roles than need the custom modules.21:43
mwhahahaso there's no way to distribute an ansible module for others to consume without them shoving it in their playbooks21:43
mwhahahaie write software, here's an ansible module that you can use with this21:43
mwhahahaeveryone profits w/o having to copy-pasta21:43
sdoranThe best way is to put your custom modules inside the roles that use them.21:43
sdoranThen write a simple playbook to call the role.21:44
sdoranThe rub comes when you want to use said custom modules across multiple roles.21:44
openstackgerritPradeep Kilambi proposed openstack/tripleo-quickstart-extras master: Update the docker registry defaults to rdoproject  https://review.openstack.org/52020121:44
mwhahaharight that's why i'm saddened by this21:45
sdoranStill best in that case to put all custom modules in a single place outside the roles and set your module path, or have them in dirs adjacent to playbooks so they automatically get picked up.21:45
sdoranIs distributing inside a role not sufficient?21:45
mwhahahait's awkward21:46
sdoranMost of the time you want to share tasks that call the custom module as well, right?21:46
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: debug unbound queries  https://review.openstack.org/52005521:46
*** rwsu has quit IRC21:47
mwhahahasdoran: so in the case i'm asking about, it's adding something to parse results from another software tool getting added to quickstart. now quickstart doesn't share it's roles seperately so no one benefits from this21:47
sdoran(We are working on a better way to distribute/install plugins. Just seeing if we can solve it some way today)21:47
mwhahahano i don't think we can21:47
mwhahahait's just it goes against being able to properly seperate the functiuonality so that others can use it21:48
mwhahahait would seem like the other peice of software should own the ansible module/role21:48
mwhahahabut there isn't a good way for that to be distributed21:48
mwhahahasdoran: https://review.openstack.org/#/c/479882/12 the review in question21:49
sdoranHmm, yeah having a monolithic repo full of roles makes that hard to distribute. Why I recommend against the practice.21:49
mwhahahayea let's not go down that path21:49
sdoranBut we (Ansible) really need a bette way to distribute the custom plugins.21:49
sdoranAgreed.21:49
mwhahahaeither way, to me the .py should be owned by the other tool21:49
mwhahahacause it's functionality that goes with that tool21:50
mwhahahabut there doesn't appear to be a good way for me to import this into my custom role21:50
mwhahahaso like if that .py gets packaged in the rpm and i install ARA, how could ansible leverage that21:50
sdoranYou could install it into one of the default locations Ansible searches for plugins.21:51
openstackgerritMerged openstack/tripleo-heat-templates stable/pike: Enable TLS for ec2api service  https://review.openstack.org/51961121:51
openstackgerritMerged openstack/instack-undercloud stable/pike: Install ansible-pacemaker during the upgrade/update in the UC  https://review.openstack.org/52004021:51
openstackgerritMerged openstack/tripleo-heat-templates stable/pike: Enable ntp iburst  https://review.openstack.org/51977921:51
openstackgerritMerged openstack/instack-undercloud master: Fix panko ssl port  https://review.openstack.org/52008721:51
openstackgerritMerged openstack/tripleo-common master: Move defaults out of overcloud_containers.yaml.j2  https://review.openstack.org/51613621:51
openstackgerritMerged openstack/tripleo-common master: Add rsyslog sidecar image for container image prepare  https://review.openstack.org/51528221:51
openstackgerritMerged openstack/instack-undercloud stable/pike: Fix panko ssl port  https://review.openstack.org/52011621:51
openstackgerritMerged openstack/tripleo-heat-templates master: OVN: Provide the option to define NeutronBridgeMappings as a role parameter  https://review.openstack.org/51844021:51
mwhahahawooo merging21:51
EmilienMomg21:51
mwhahahasdoran: ah exactly, that's what i was wondering of. so there is a path that you could use in the other tool to distribute it21:51
sdoranYeah, I think that would work.21:52
weshaywooooot21:52
weshayMERGE PARTY21:52
sdoranAssuming Ansible is installed on the system (or the tool will make the dirs if they don't exist), just have the tool drop custom plugins into the default paths Ansible will search.21:52
mwhahahasdoran: are these paths documented somewhere?21:53
* sdoran looking21:53
jristweshay: :)21:53
sdoranI know what the relative ones are21:53
fultonjthe ansible.cfg will list them21:53
sdoranNeed to look at the absolutes.21:53
fultonjor your separate ansible job could reference your own ansible.cfg which is what the ceph-ansible integration does21:54
*** lblanchard has quit IRC21:54
openstackgerritMerged openstack/puppet-tripleo stable/pike: HAproxy should get full response from ironic-inspector  https://review.openstack.org/52008921:54
mwhahahathough i wonder how it would work if your play book has to install the package and so the function doesn't exist at execution time21:54
mwhahahaso i don't have my custom module foo when envoked because the playbook itself installs it21:55
*** dsariel has quit IRC21:55
mwhahahai have a feeling that would error when it goes to parse the tasks initially?21:55
sdoranLooks like `/usr/share/ansible/plugins/[plugin type]`21:56
*** brian83 has quit IRC21:56
mwhahahasdoran: so do you know if the parser would error if it wasnt installed prior to the playbook exeuction?21:57
sdoranIdeally your Ansible control node is configured _before_ you start running things.21:57
*** toure is now known as toure_biab21:57
sdoranBut it probably depends on dynamic or static includes, whether it will error.21:58
sdoranIf the custom module is used in a dynamic include _after_ the custom module installation tasks, you'll probably be ok.21:58
mwhahahainteresting21:58
sdoranThis seems like playing chicken, though. 🙂21:59
sdoranI would have to test.21:59
slagleis it normal for master and pike patches to be in the same gate jobs?22:00
mwhahahaslagle: yes22:00
slagleit doesn't make sense, but I accept it22:01
mwhahahaslagle: infra'd22:01
sdoranmwhahaha: Do you have an example repo of what you're doing? Is this TripleO quickstart stuff?22:01
mwhahahasdoran: i was asking because of that review i previously linked22:01
mwhahahaARA is software outside of quickstart so we're adding in an ansible module to shove ARA data to graphite22:02
mwhahahai would assume that ARA should own that ansible module part22:02
mwhahahaand we would just consume it22:02
mwhahahawhich is why I was asking about the distribution22:02
sdoranSorry, missed that. See it now.22:02
mwhahahaARA gets installed via rpm i think22:02
*** ccamacho has quit IRC22:02
mwhahahaso if it could drop the module some where and it would just work22:03
mwhahahanow the dangers come if you don't have it installed or whatever22:03
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: DNM, update logging for undercloud containers  https://review.openstack.org/51897222:03
mwhahahaI would think other's would want that functionality outside of quickstart22:03
mwhahahaso trying to figure out the best home for it22:03
mwhahahabecause otherwise we'll end up with our version and it'll just stay there for a long time22:03
*** rwsu has joined #tripleo22:08
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: Don't try to start docker for the UC  https://review.openstack.org/51944122:09
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432822:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103222:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106322:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173214022:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)22:10
*** ooolpbot has quit IRC22:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]22:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]22:10
openstackLaunchpad bug 1732140 in tripleo "Kolla tests failing for tripleo-common master" [Critical,In progress] - Assigned to Adriano Petrich (apetrich)22:10
*** itlinux has joined #tripleo22:11
*** rodolof has quit IRC22:11
mwhahahaslagle: i asked about splitting off the branches in infra the other week but I don't remember the reason. But yea queues are split on a repo basis and are not branch aware. It does seem like making them branch aware would be beneficial for merge times22:12
*** liverpooler has joined #tripleo22:12
openstackgerritPradeep Kilambi proposed openstack/python-tripleoclient master: Add support to generate a sample undercloud.conf  https://review.openstack.org/51859522:12
*** rwsu has quit IRC22:14
*** rhallisey has quit IRC22:14
owalshmwhahaha: another blocker? http://logs.openstack.org/35/520135/1/check/openstack-tox-py27/28ec230/job-output.txt.gz#_2017-11-15_16_46_01_39289522:17
mwhahahaowalsh: no22:17
mwhahahaowalsh: Bug 173214022:18
openstackbug 1732140 in tripleo "Kolla tests failing for tripleo-common master" [Critical,In progress] https://launchpad.net/bugs/1732140 - Assigned to Adriano Petrich (apetrich)22:18
owalshmwhahaha: bah, I should have scolled back22:19
*** itlinux has quit IRC22:23
openstackgerritOliver Walsh proposed openstack/tripleo-common master: Generate password for libvirt TLS SCRAM auth  https://review.openstack.org/52013522:23
*** rcernin has joined #tripleo22:23
owalshmwhahaha: re https://bugs.launchpad.net/tripleo/+bug/1731063 I don't see anything logged in the neutron metadata agent logs either, so looks like the VM can't contact it (or never tried)22:25
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]22:25
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo master: Migrate puppet-tripleo to zuul v3 jobs  https://review.openstack.org/52018922:25
*** rwsu has joined #tripleo22:25
mwhahahaowalsh: is that a missing log thing or are the logs empty22:25
owalshmwhahaha: have logs, but not entry for that VM http://logs.openstack.org/79/516979/2/gate/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-container/13841da/logs/subnode-2/var/log/containers/neutron/neutron-metadata-agent.log.txt.gz22:26
owalshmwhahaha: 10.100.0.4 is the IP I think22:26
owalshmwhahaha: timing issue maybe, ssh just starts working on the twentysomething-th retry22:27
owalshbut it smells like a network issue22:28
*** vpickard is now known as vpickard_22:29
mwhahahaowalsh: it clearly is but it's really hard to know why it would be 0001 and not any other scenario22:29
owalshmwhahaha: good point22:31
mwhahahaowalsh: so arx looked at it and said it gave up on the metadata service which may be why it doesn't log22:31
sdoranmwhahaha: Went and talked with some folks on this. Seems the best way, today, is to create a role on Galaxy that contains your module.22:34
sdoranhttps://galaxy.ansible.com/ansible/kubernetes-modules/ <= example22:34
owalshmwhahaha: makes sense22:34
owalshmwhahaha; not look into this yet but https://bugzilla.redhat.com/show_bug.cgi?id=134000122:35
openstackbugzilla.redhat.com bug 1340001 in rhosp-director "Keystone auth_url set wrongly in metadata_agent.ini" [Medium,New] - Assigned to beagles22:35
sdoranIRT a play that installs a module that is needed by later tasks, this is kinda risky. We don't have great mechanisms for refreshing the plugin cache mid-play today.22:35
sdoranIf it works at all, you would have to ensure that the tasks/roles using the custom module are dynamically included _after_ the installation tasks.22:35
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs  https://review.openstack.org/51479322:36
openstackgerritLee Yarwood proposed openstack/tripleo-quickstart master: config: Provide a Master UC and Newton OC release config  https://review.openstack.org/51919522:37
openstackgerritLee Yarwood proposed openstack/tripleo-quickstart master: config: Provide a Keystone only overcloud config  https://review.openstack.org/51919622:37
mwhahahasdoran: so i guess the assumption is that there will always be some $module that provides all the things for $software. so in this case we'd need an ara module that sets up/install/provides ansible functionality22:37
mwhahahaowalsh: i thought we fixed it22:38
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Migrate THT to zuul v3 jobs  https://review.openstack.org/51976122:38
mwhahahaowalsh: https://github.com/openstack/tripleo-heat-templates/blob/17e79c8b15d7746ce95f90a0b259481d6bd9302c/puppet/services/neutron-metadata.yaml#L9522:39
owalshmwhahaha: so IIUC that should be KeystoneAdmin22:40
mwhahahasdoran: so I guess in this case we need an ara-ansible-module that provfides it22:40
sdoranmwhahaha: I'm confused by your use of $module. Ansible module, or some other module? (that term gets used in a lot of contexts)22:40
mwhahahasdoran: ansible module/role22:41
mwhahahasdoran: so i guess i meant role, in puppet it's a module22:41
mwhahahasdoran: where role is the distributed ansible bundle22:41
mwhahahaa role can include tasks/module bits22:41
mwhahaharight?22:42
openstackgerritMerged openstack/tripleo-heat-templates master: Write readme.txt into old log directories  https://review.openstack.org/51858422:42
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo master: Migrate puppet-tripleo to zuul v3 jobs  https://review.openstack.org/52018922:42
sdoranYeah, that's correct. puppet module is more akin to Ansible role.22:42
sdoranAnsible roles can contain any number of custom plugins. It doesn't need to have any tasks.22:43
sdoranAnd if you set the role as a dependency for other roles that need the custom modules, it will work.22:43
sdoranIt's not great, but it's what we have today.22:44
mwhahahayea just trying to figure out the best overall structure for reusablity by others since oooq-extras isn't going to get split up22:44
*** itlinux has joined #tripleo22:44
sdoranWe're working an a more capable installer (not a package manager).22:44
mwhahahaowalsh: i think that was a v2/v3 problem. I thought the admin/internal endpoint is the same22:44
owalshmwhahaha: yea, looking a bit closer I doubt it's this as I've seen other VMs getting metadata with no problems22:45
owalsh... when this particular VM fails22:45
mwhahahasdoran: sorry the noob question, so how do folks pull in galaxy modules today? is it natively in the playbook structure where it'll go auto-fetch them22:45
mwhahahaowalsh: how does the metadata service get called, i wonder if it's an dnsmasq problem where maybe the vm doesn't get an IP in time?22:46
sdoranNo, your questions are perfectly valid. You just happen to hit one of the ones we've been wrestling with for a while. 😄22:46
sdoranYou list the roles in a `requirements.yml` file and run `ansible-galaxy install -r requirements.yml -p roles`22:47
owalshmwhahaha: black magic, or something to do with DHCP I think... not sure of the mechanics of this22:47
sdoranThat'll pull in all your roles, from any valid repo, and put it into place.22:47
mwhahahasdoran: meh, too pythony :D also are there version numbers?22:47
sdoranThen you can run your `ansible-playbook` command.22:47
sdoranYeah, Pythonisms do bubble up a bit.22:48
sdoranYou can version roles, which are just tags in the repo.22:48
mwhahahai think from our stand point, we'd want more packages and less galaxy fetching. is there a standard path structure for role lookups or not yet22:49
sdoranWell the requirements file doesn't have to touch Galaxy. Most customers have it point only to internal repos. Not sure if that's helpful.22:49
*** bobh has quit IRC22:50
openstackgerritPradeep Kilambi proposed openstack/puppet-tripleo master: DNM: test ceilo upgrade with gnocchi dep  https://review.openstack.org/52023422:50
sdoranMeaning if you wanted to distribute your roles in a rpm, for example?22:50
mwhahahasdoran: yea i mean being able to yum install ansible-galaxy-foo and it installs to /usr/share/ansible/roles22:50
mwhahahaand we can just ansible-playbook bar.yaml that references foo22:50
*** d0ugal has joined #tripleo22:51
sdoranYeah, that't work very nicely.22:51
mwhahahais that a thing already?22:52
pradkmwhahaha, can you +A this https://review.openstack.org/#/c/509199/22:52
mwhahahapradk: done22:52
sdoranNo, not really. But you could do it.22:52
pradkthx!22:52
mwhahahasdoran: why is that not a thing :(22:52
sdoranMost people don't mess with building packages.22:52
mwhahahaand they should be shot22:52
openstackgerritMerged openstack/tripleo-quickstart-extras master: update the master tempest skip list, remove old bugs  https://review.openstack.org/51382022:52
sdoranThey use monolithic repos, requirements file, or git submodules.22:53
mwhahahapip install is no way to go through life :D22:53
sdoranBut you could absolutely make a package. Would be really simple.22:53
mwhahahathese are probably things we'll want for flaper87's k8s roles22:54
sdoranJust install to `/usr/share/ansible/plugins/[type]`22:54
sdoranThat's the default. But you can change them all in `ansible.cfg`22:54
mwhahahaso does it have to be a plugin or is a plugin structure the same a s role22:55
mwhahahaas a role22:56
EmilienMif someone wants some cleanup https://review.openstack.org/#/c/518423/22:56
mwhahahaEmilienM: i think we still use thsoe?22:56
EmilienMmwhahaha: where?22:56
EmilienMwe should not22:56
EmilienMand if we do, it's a bug in the zuul layout22:56
EmilienMthat will be fixed by my stuff22:56
EmilienMI can WIP now22:57
EmilienMin case22:57
mwhahahaEmilienM: see http://logs.openstack.org/23/518423/4/check/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq-container/65c11a6/logs/undercloud/home/zuul/overcloud-deploy.sh22:57
sdoranmwhahaha: Plugins are usually just single files. Roles have a standard structure.22:57
sdoranYou can see the strucutre22:57
EmilienMmwhahaha: scenario001-multinode-containers.yaml, not scenario001-multinode.yaml22:57
mwhahahasdoran: right so i was talking about roles in that case not plugins22:57
sdoranby running `ansible-galaxy init myrole`22:57
mwhahahaEmilienM: but we still have the -puppet jobs on oooq22:57
sdoranThat'll create a skeleton for you22:57
EmilienMmwhahaha: fair enough, I'll WIP my stuff. Nice catch22:58
mwhahahasdoran: right so my quesiton was is there a system path for loading roles22:58
sdoran`/usr/share/ansible/roles`22:58
mwhahahaok you confused me when you went mack to plugins :D22:59
sdoranYou can see all the default settings as of Ansible 2.4 with `ansible-config dump`22:59
sdoran`ansible-config dump | egrep '(PLUGIN|MODULE|ROLES)_PATH'` will show you default search paths (assuming you haven't changed them)22:59
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/47260723:00
sdoranBut `/usr/share/ansible` is the system path we search by default for roles and plugins.23:00
mwhahahaok cool, so sounds like for any roles we should package them there and plugins can be dropped into /usr/share/ansible/plugins/<type>23:01
*** rwsu has quit IRC23:01
sdoranYup. To stay within the defaults.23:01
sdoranYou can always override with your own `ansible.cfg`, but then have have to make sure that travels around with your playbooks.23:02
*** dsneddon has quit IRC23:02
*** dsneddon has joined #tripleo23:04
mwhahahasdoran: thanks for the info23:06
sdoranNo problem.23:07
sdoranHope something was helpful.23:07
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: Migrate tripleo-common to zuul v3 jobs  https://review.openstack.org/52023823:08
mwhahahain this case, it would be beneficial to have a role for ARA and for us if it's distributed by rpm we could just use the stuff23:08
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs  https://review.openstack.org/51479323:09
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/172432823:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173103223:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173106323:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/173214023:10
openstackLaunchpad bug 1724328 in tripleo "Netwon to Ocata upgrade failure because of ceilometer-upgrade" [Critical,Triaged] - Assigned to mathieu bultel (mat-bultel)23:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1731032 in tripleo "CI: Deployment fails on controller timeout with task creation in heat timing out" [Critical,Triaged]23:10
openstackLaunchpad bug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged]23:10
openstackLaunchpad bug 1732140 in tripleo "Kolla tests failing for tripleo-common master" [Critical,In progress] - Assigned to Adriano Petrich (apetrich)23:10
mwhahahaoh noes23:12
mwhahahaBug 1731063 may have shown up in scenario00323:12
openstackbug 1731063 in tripleo "CI: tempest TestVolumeBootPattern tests fail due to not being able to ssh to the VM" [Critical,Triaged] https://launchpad.net/bugs/173106323:12
mwhahahahttp://logs.openstack.org/70/519770/2/gate/legacy-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/69e0c2c/logs/undercloud/home/zuul/tempest_output.log.txt.gz#_2017-11-15_22_38_4023:12
*** cdearborn has quit IRC23:12
mwhahahabut different test though23:13
mwhahahamaybe same root cause?23:13
EmilienMare we sure it was only on scenario001?23:14
EmilienMdo we have an elastic query?23:14
mwhahahayea23:14
EmilienMor a logstash query?23:14
*** itlinux has quit IRC23:15
*** dmacpher has quit IRC23:19
openstackgerritDan Prince proposed openstack/tripleo-quickstart-extras master: Don't try to start docker for the UC  https://review.openstack.org/51944123:21
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs  https://review.openstack.org/51479323:23
*** pradk has quit IRC23:24
mwhahahaEmilienM: when you look for http://paste.openstack.org/show/626441/ (without the build_name stuff) with elastic-recheck the top5 tests are networking-odl, neutron-dvr, scenario001-puppet, networking grendade and scenario001-containers23:25
EmilienMmaybe we don't run scenario003 enough to catch it?23:26
mwhahahanope just 001 if you restrict to just THT23:26
mwhahahasince it's a different test it probably logs differently23:27
mwhahahayea different line23:27
mwhahahalet me try just SSHTimeout:23:27
*** thrash is now known as thrash|g0ne23:27
openstackgerritEmilien Macchi proposed openstack/python-tripleoclient master: Migrate python-tripleoclient to zuul v3 jobs  https://review.openstack.org/52024123:28
mwhahahathis is another case where it doesn't end up in the logs in the same way i think23:29
EmilienMouch23:29
mwhahahagood ol' consistency or lack there of23:30
EmilienMmwhahaha: I see it coming :P23:30
openstackgerritSteve Baker proposed openstack/python-tripleoclient master: Use common defaults for image prepare CLI  https://review.openstack.org/51613923:32
*** rlandy|rover is now known as rlandy|rover|bbl23:33
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs  https://review.openstack.org/51479323:34
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs  https://review.openstack.org/51479323:36
openstackgerritSteve Baker proposed openstack/tripleo-common master: Finish deprecation of pull_source  https://review.openstack.org/51613823:45
openstackgerritSteve Baker proposed openstack/tripleo-common master: Memoize docker uploader and client  https://review.openstack.org/51920023:45
openstackgerritSteve Baker proposed openstack/tripleo-common master: Implement post-upload cleanup of docker images  https://review.openstack.org/51920123:45
*** leitan has joined #tripleo23:45
*** etingof has joined #tripleo23:48
*** abishop has quit IRC23:49
openstackgerritSteve Baker proposed openstack/tripleo-quickstart master: Silence repo-setup task  https://review.openstack.org/50766623:50
openstackgerritSteve Baker proposed openstack/tripleo-quickstart master: Fix undercloud inventory ansible_host  https://review.openstack.org/50544723:50
*** etingof has quit IRC23:53
openstackgerritSteve Baker proposed openstack/tripleo-quickstart-extras master: Make UI accessable by default on OVB  https://review.openstack.org/50796923:54
openstackgerritSteve Baker proposed openstack/tripleo-quickstart-extras master: Sudo fix overcloud-custom-tht-script.sh.j2  https://review.openstack.org/49901823:54
openstackgerritSteve Baker proposed openstack/tripleo-quickstart-extras master: Use os_keypair for key upload  https://review.openstack.org/44522423:54
openstackgerritSteve Baker proposed openstack/tripleo-quickstart-extras master: Allow OVB to attach undercloud to existing floating IP  https://review.openstack.org/50639723:54
*** rfolco has quit IRC23:55
*** lblanchard has joined #tripleo23:55
« Tuesday, 2017-11-14 Index Thursday, 2017-11-16 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!