| *** bobh has joined #tripleo | 00:09 | |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
|---|---|---|
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 00:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730179 | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 00:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 00:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 00:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 00:10 |
| openstack | Launchpad bug 1730179 in tripleo "overcloud deploy hangs causing CI job timeout" [Critical,Triaged] | 00:10 |
| *** dparkes has quit IRC | 00:22 | |
| *** oanson has quit IRC | 00:34 | |
| *** brault has quit IRC | 00:36 | |
| *** brault has joined #tripleo | 00:36 | |
| *** oanson has joined #tripleo | 00:42 | |
| *** radeks has joined #tripleo | 00:43 | |
| *** agopi has joined #tripleo | 00:54 | |
| *** noslzzp has quit IRC | 00:57 | |
| *** noslzzp has joined #tripleo | 00:57 | |
| *** noslzzp has quit IRC | 00:58 | |
| *** dparkes has joined #tripleo | 00:58 | |
| *** gbarros has joined #tripleo | 01:00 | |
| *** gbarros has quit IRC | 01:03 | |
| *** cshastri has joined #tripleo | 01:09 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 01:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730179 | 01:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 01:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 01:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 01:10 |
| openstack | Launchpad bug 1730179 in tripleo "overcloud deploy hangs causing CI job timeout" [Critical,Triaged] | 01:10 |
| *** dougbtv__ has quit IRC | 01:19 | |
| *** radeks has quit IRC | 01:30 | |
| *** weshay changes topic to "CI status: GREEN | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest/" | 01:49 | |
| *** dougbtv__ has joined #tripleo | 02:02 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements https://review.openstack.org/511633 | 02:06 |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 02:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730179 | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 02:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 02:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 02:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 02:10 |
| openstack | Launchpad bug 1730179 in tripleo "overcloud deploy hangs causing CI job timeout" [Critical,Triaged] | 02:10 |
| *** dougbtv__ has quit IRC | 02:15 | |
| *** bobh has quit IRC | 02:19 | |
| *** bobh has joined #tripleo | 02:21 | |
| *** bobh has quit IRC | 02:27 | |
| *** rfolco has quit IRC | 02:30 | |
| *** japestinho has joined #tripleo | 02:33 | |
| openstackgerrit | Flavio Percoco proposed openstack/tripleo-heat-templates master: WIP: integrating tht and ansible-role-k8s-* https://review.openstack.org/517844 | 02:38 |
| *** daidv has joined #tripleo | 02:46 | |
| *** shardy has joined #tripleo | 02:58 | |
| *** brault has quit IRC | 03:03 | |
| *** anilvenkata has joined #tripleo | 03:09 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 03:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730179 | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 03:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 03:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 03:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 03:10 |
| openstack | Launchpad bug 1730179 in tripleo "overcloud deploy hangs causing CI job timeout" [Critical,Triaged] | 03:10 |
| *** d0ugal has quit IRC | 03:14 | |
| *** bobh has joined #tripleo | 03:16 | |
| *** radeks has joined #tripleo | 03:20 | |
| *** shardy has quit IRC | 03:21 | |
| *** chlong__ has joined #tripleo | 03:31 | |
| *** chlong_ has quit IRC | 03:33 | |
| *** zshi_ has joined #tripleo | 03:35 | |
| *** ecerquei has quit IRC | 03:40 | |
| *** yolanda has joined #tripleo | 03:44 | |
| *** yolanda has quit IRC | 03:45 | |
| *** rwsu has joined #tripleo | 03:49 | |
| *** stendulker has joined #tripleo | 03:51 | |
| *** gbarros has joined #tripleo | 03:56 | |
| *** ykarel has joined #tripleo | 04:09 | |
| *** mdnadeem has joined #tripleo | 04:10 | |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 04:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730179 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 04:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 04:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 04:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 04:10 |
| openstack | Launchpad bug 1730179 in tripleo "overcloud deploy hangs causing CI job timeout" [Critical,Triaged] | 04:10 |
| *** radeks has quit IRC | 04:12 | |
| *** radeks has joined #tripleo | 04:13 | |
| *** bobh has quit IRC | 04:25 | |
| *** skramaja has joined #tripleo | 04:33 | |
| *** udesale has joined #tripleo | 04:37 | |
| *** bobh has joined #tripleo | 04:38 | |
| *** pgadiya has joined #tripleo | 04:39 | |
| *** jbadiapa_ has quit IRC | 04:41 | |
| *** pdeore has joined #tripleo | 04:42 | |
| *** bobh has quit IRC | 04:42 | |
| *** agopi has quit IRC | 04:47 | |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates master: Added NovaMigrationTarget service to SR-IOV and DPDK roles https://review.openstack.org/517857 | 04:48 |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates stable/pike: Added a new role definition for SR-IOV Compute role https://review.openstack.org/517858 | 04:48 |
| *** derekjhyang has joined #tripleo | 04:50 | |
| *** radeks has quit IRC | 04:52 | |
| *** gbarros has quit IRC | 04:53 | |
| *** janki has joined #tripleo | 04:54 | |
| *** psachin has joined #tripleo | 04:54 | |
| *** agopi has joined #tripleo | 04:56 | |
| *** jaganathan has joined #tripleo | 04:56 | |
| *** agopi has quit IRC | 05:02 | |
| *** achadha has joined #tripleo | 05:05 | |
| *** cylopez has joined #tripleo | 05:06 | |
| *** cylopez has quit IRC | 05:07 | |
| *** cylopez has joined #tripleo | 05:08 | |
| openstackgerrit | Merged openstack/tripleo-common stable/pike: Add retry loop for docker pull https://review.openstack.org/517305 | 05:08 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Speed up deployment by reusing facts https://review.openstack.org/514945 | 05:09 |
| *** achadha has quit IRC | 05:09 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 05:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730179 | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 05:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 05:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 05:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 05:10 |
| openstack | Launchpad bug 1730179 in tripleo "overcloud deploy hangs causing CI job timeout" [Critical,Triaged] | 05:10 |
| *** shreshtha has joined #tripleo | 05:19 | |
| *** agopi has joined #tripleo | 05:19 | |
| *** jaosorior has joined #tripleo | 05:27 | |
| *** agopi has quit IRC | 05:32 | |
| *** stendulker_ has joined #tripleo | 05:32 | |
| *** ratailor has joined #tripleo | 05:33 | |
| *** stendulker has quit IRC | 05:35 | |
| *** agopi has joined #tripleo | 05:42 | |
| *** karthiks has joined #tripleo | 05:43 | |
| *** udesale has quit IRC | 05:45 | |
| *** shreshtha_ has joined #tripleo | 05:45 | |
| *** agopi has quit IRC | 05:47 | |
| *** shreshtha has quit IRC | 05:48 | |
| *** dparkes has quit IRC | 05:52 | |
| Tengu | hello there :). | 05:52 |
| jaosorior | good morning!\ | 05:53 |
| Tengu | jaosorior: jaosorior ready for some logging configuration? :) | 05:53 |
| jaosorior | Tengu: did you see the last mail from the thread you started? | 05:54 |
| Tengu | yep, answering to Doug | 05:54 |
| jaosorior | Tengu: I think it would be good to pursue that instead. | 05:54 |
| jaosorior | seems like the solution that will require the least effort. | 05:54 |
| Tengu | exposing a configuration in oslo.log that enables JSON output right away? true. That was my first thought, but it was kind of smashed by Dims :) | 05:55 |
| Tengu | but I still think using the "long" logging.conf way would be more efficient, especially since this would allow people to provide their own output format. | 05:56 |
| *** dsneddon has quit IRC | 05:57 | |
| jaosorior | Tengu: well, you can already override the format https://github.com/openstack/oslo.log/blob/master/oslo_log/_options.py#L109 :P | 05:58 |
| Tengu | hmm yes but nope ;) | 05:58 |
| Tengu | was more speaking about the output itself (being plain or JSON, maybe people would like to output in… well… XML, after all) | 05:59 |
| Tengu | aaanyway. answering to the ML so that we can get some more insights and feedbacks. | 06:00 |
| jaosorior | Tengu: I see your point | 06:00 |
| jaosorior | Tengu: if we want to start using that file by default, and we could | 06:00 |
| jaosorior | we do need to make it backwards compatible | 06:00 |
| Tengu | yup. lemme finish my mail so that you can push an answer in that way :) | 06:01 |
| jaosorior | look at this file https://github.com/openstack/oslo.log/blob/master/oslo_log/_options.py and look at all the instances of _IGNORE_MESSAGE | 06:01 |
| jaosorior | ok | 06:01 |
| *** gbarros has joined #tripleo | 06:05 | |
| *** tvignaud has joined #tripleo | 06:07 | |
| *** jprovazn has joined #tripleo | 06:10 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 06:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730179 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 06:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 06:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 06:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 06:10 |
| openstack | Launchpad bug 1730179 in tripleo "overcloud deploy hangs causing CI job timeout" [Critical,Triaged] | 06:10 |
| *** suuuper has joined #tripleo | 06:12 | |
| *** zshi_ has quit IRC | 06:14 | |
| *** agopi has joined #tripleo | 06:14 | |
| *** chlong__ has quit IRC | 06:14 | |
| *** abregman has joined #tripleo | 06:14 | |
| *** benone has quit IRC | 06:14 | |
| *** abregman has quit IRC | 06:14 | |
| *** abregman has joined #tripleo | 06:14 | |
| *** shreshtha_ has quit IRC | 06:15 | |
| *** benone has joined #tripleo | 06:16 | |
| *** agopi has quit IRC | 06:17 | |
| *** shreshtha has joined #tripleo | 06:17 | |
| *** bandini has quit IRC | 06:17 | |
| *** bandini has joined #tripleo | 06:19 | |
| *** noslzzp has joined #tripleo | 06:21 | |
| *** chlong__ has joined #tripleo | 06:26 | |
| *** abregman has quit IRC | 06:31 | |
| *** psachin has quit IRC | 06:31 | |
| *** mdnadeem has quit IRC | 06:31 | |
| *** openstackgerrit has quit IRC | 06:31 | |
| *** zzzeek has quit IRC | 06:31 | |
| *** zoli has quit IRC | 06:31 | |
| *** rodrigods has quit IRC | 06:31 | |
| *** arxcruz- has quit IRC | 06:31 | |
| *** Slower has quit IRC | 06:31 | |
| *** ed_b has quit IRC | 06:31 | |
| *** timothyb89 has quit IRC | 06:31 | |
| *** bandini has quit IRC | 06:33 | |
| *** jbadiapa has joined #tripleo | 06:34 | |
| *** bandini has joined #tripleo | 06:36 | |
| *** agopi has joined #tripleo | 06:37 | |
| *** agurenko has joined #tripleo | 06:37 | |
| *** d0ugal has joined #tripleo | 06:38 | |
| *** bobh has joined #tripleo | 06:40 | |
| jaosorior | Tengu: started this https://etherpad.openstack.org/p/oslo-log-options-for-advanced-file | 06:41 |
| jaosorior | Tengu: set the files that would need equivalents and the default values there already so it's easier to set. I guess we could put the equivalent configuration there too. | 06:42 |
| *** dparkes has joined #tripleo | 06:44 | |
| jaosorior | Tengu: we would need to set the defaults explicitly, since we would no longer have access to the $::os_service_default value. | 06:44 |
| *** bobh has quit IRC | 06:45 | |
| jaosorior | The reason I'm doing this is because using the classic logging configuration and the advanced logging is mutually exclusive: https://github.com/openstack/oslo.log/blob/master/oslo_log/log.py#L259 | 06:45 |
| jaosorior | looking at the code, it seems that we would need an equivalent for the rate limiting options as well | 06:46 |
| *** agopi has quit IRC | 06:47 | |
| *** gbarros has quit IRC | 06:48 | |
| *** chlong__ has quit IRC | 06:49 | |
| Tengu | 2s, reading | 06:49 |
| Tengu | hmm. not sure the rate limit is available "as is" in the python logging lib. | 06:50 |
| Tengu | jaosorior: why wouldn't the fact/variable $::os_service_default be available? | 06:51 |
| jaosorior | Tengu: cause there are no defined openstack defaults in the "advanced logging file" | 06:52 |
| *** yolanda has joined #tripleo | 06:52 | |
| *** dsariel has joined #tripleo | 06:52 | |
| Tengu | ah, ok. well, we might push a default for that. it goes in the direction "generic and cool base config" I spoke about in the ML | 06:52 |
| jaosorior | Tengu: and even if they were, they probably are not the same to what we're setting in oslo.log's _options.py | 06:52 |
| Tengu | true for that | 06:52 |
| jaosorior | Tengu: that's why I started tracking the defaults in that etherpad | 06:53 |
| Tengu | yup | 06:53 |
| jaosorior | what was the docs site for the logging config? | 06:54 |
| Tengu | hmm the one for the python-logging lib, 2s | 06:54 |
| Tengu | https://docs.python.org/2/howto/logging.html#logging-advanced-tutorial | 06:55 |
| Tengu | more specifically: https://docs.python.org/2/howto/logging.html#configuring-logging | 06:55 |
| jaosorior | great | 06:55 |
| Tengu | few scroll down you'll get the example, but as said in the ML: the configuration file isn't that friendly. a good abstraction layer for beginners would be nice, i.e. good defaults and such | 06:56 |
| jaosorior | Tengu: is the loggers section mandatory? | 06:57 |
| jaosorior | Tengu: seems that it's highly tied with the code :/ | 06:57 |
| *** marios has joined #tripleo | 06:57 | |
| Tengu | hmm for what I understand: loggers list a bunch of logger configuration, and one logger links to the log handler, that one linking to the formatter class | 06:58 |
| Tengu | not sure it's that tight to the code itself. | 06:58 |
| Tengu | *tied | 06:58 |
| jaosorior | Tengu: ok, lets look for an example, is the getLogger call used by any openstack component? | 06:58 |
| Tengu | should be. 2s, cloning some project. | 06:59 |
| jaosorior | if so, we would probably need the value given to that call and set it there, would we? | 06:59 |
| jaosorior | I do see a bunch of calls to log.getLogger(__name__) in the keystone codebase | 06:59 |
| *** iranzo has joined #tripleo | 07:00 | |
| *** iranzo has joined #tripleo | 07:00 | |
| Tengu | humpf. apparently we have some connection issue in here -.- great. | 07:00 |
| jaosorior | bummer :/ | 07:00 |
| Tengu | peering issue apparently. | 07:01 |
| Tengu | nova's coming on my computer so that I can play with grep. | 07:01 |
| *** agopi has joined #tripleo | 07:01 | |
| *** jfrancoa has joined #tripleo | 07:01 | |
| jaosorior | bandini, marios: Could you check this out https://review.openstack.org/#/c/517039/ ? | 07:02 |
| Tengu | jaosorior: ok, logging.getLogger is used in apparently every class, like: nova/api/auth.py:LOG = logging.getLogger(__name__) | 07:02 |
| *** suuuper has quit IRC | 07:02 | |
| Tengu | jaosorior: and "logging" is just: from oslo_log import log as logging | 07:03 |
| jaosorior | right | 07:03 |
| Tengu | a nice way to make it compatible with previous way to log. | 07:03 |
| Tengu | jaosorior: pretty sure it's the same everywhere - the getLogger is the way to instanciate a logger. You can't output anything if you don't call that. So any service using logs will get that. | 07:04 |
| *** shreshtha has quit IRC | 07:05 | |
| *** suuuper has joined #tripleo | 07:06 | |
| jaosorior | Tengu: maybe we can make due with just using the root logger. | 07:06 |
| jaosorior | in that loggers configuration | 07:06 |
| Tengu | jaosorior: pretty sure that would be a good default. | 07:07 |
| Tengu | jaosorior: also, we'll want to check the keystone logging.conf that's shipped. | 07:07 |
| *** shreshtha has joined #tripleo | 07:07 | |
| Tengu | jaosorior: they define apparently two logger, a root and an "access". | 07:08 |
| *** brault has joined #tripleo | 07:08 | |
| Tengu | That's a cool idea, as it allows to set different log level depending on the type. | 07:08 |
| Tengu | jaosorior: may I copy the logging.conf in your pad? | 07:09 |
| jaosorior | Tengu: sure | 07:09 |
| *** spectr has joined #tripleo | 07:09 | |
| jaosorior | Tengu: I have no idea how you would enable or use the access logger though | 07:09 |
| Tengu | ah, better, point to the current file in the git. | 07:09 |
| Tengu | https://github.com/openstack/keystone/blob/master/etc/logging.conf.sample | 07:10 |
| *** udesale has joined #tripleo | 07:10 | |
| Tengu | jaosorior: first go with the root one | 07:10 |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 07:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730179 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 07:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 07:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 07:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 07:10 |
| openstack | Launchpad bug 1730179 in tripleo "overcloud deploy hangs causing CI job timeout" [Critical,Triaged] | 07:10 |
| Tengu | added in the pad with comment. | 07:10 |
| *** abregman has joined #tripleo | 07:12 | |
| *** psachin has joined #tripleo | 07:12 | |
| *** mdnadeem has joined #tripleo | 07:12 | |
| *** openstackgerrit has joined #tripleo | 07:12 | |
| *** zzzeek has joined #tripleo | 07:12 | |
| *** zoli has joined #tripleo | 07:12 | |
| *** rodrigods has joined #tripleo | 07:12 | |
| *** arxcruz- has joined #tripleo | 07:12 | |
| *** Slower has joined #tripleo | 07:12 | |
| *** ed_b has joined #tripleo | 07:12 | |
| *** timothyb89 has joined #tripleo | 07:12 | |
| Tengu | jaosorior: the cool thing is: OK, it will take a bit of time, especially in order to get the correct configuration file, but once it's done, it will be easier to set the "stdout" output you need in the containers :). | 07:12 |
| jaosorior | Tengu: maybe, or maybe not | 07:13 |
| jaosorior | Tengu: there's a catch :O | 07:13 |
| Tengu | :D | 07:13 |
| * jaosorior regrets not writing a blog post earlier | 07:13 | |
| Tengu | about what? | 07:13 |
| jaosorior | Tengu: so, lets take nova as an example | 07:13 |
| jaosorior | lets say we have the base nova services that run in a controller | 07:14 |
| *** jfrancoa has quit IRC | 07:14 | |
| jaosorior | and for this, lets take API, placement, consoleauth, scheduler into account | 07:14 |
| jaosorior | they all share the same nova configuration | 07:14 |
| Tengu | yup | 07:14 |
| jaosorior | API and placement run over httpd, while consoleauth and scheduler don't | 07:15 |
| *** derekjhyang has quit IRC | 07:15 | |
| jaosorior | the tricky part with httpd is that it redirects stdout/stderr of the services running on it | 07:15 |
| jaosorior | and it also overrides the ouputs with its own logging format | 07:15 |
| jaosorior | so we can't just output everything to stdout/stderr, cause we would get logs with: first, the httpd logging format, second, the openstack service's logging format. | 07:16 |
| jaosorior | so what I do is let it keep logging to a file, and track that file with a sidecar container. | 07:16 |
| Tengu | hmm ok. | 07:17 |
| jaosorior | effectively, what this ends up as is, for each service running over httpd | 07:17 |
| jaosorior | we have 3 logging containers | 07:17 |
| jaosorior | openstack service log, access log and error log | 07:17 |
| jaosorior | now, for consoleauth and scheduler, we don't have this limitation | 07:17 |
| *** agopi has quit IRC | 07:17 | |
| Tengu | I think this won't really change in fact. | 07:17 |
| jaosorior | but they share the nova.conf file | 07:17 |
| Tengu | they will as well share the logging.conf file. | 07:18 |
| jaosorior | so they would keep logging to a file, and then I would require one sidecar container for logging for each service, | 07:18 |
| *** Ng has quit IRC | 07:18 | |
| jaosorior | which is a bit of a waste | 07:18 |
| jaosorior | so what I do | 07:18 |
| jaosorior | is that I pass the --config-dir and --config-file options, which come from oslo.cfg, and eventually get used by oslo-logging | 07:19 |
| jaosorior | *oslo_log | 07:19 |
| *** Ng has joined #tripleo | 07:19 | |
| *** ChanServ sets mode: +v Ng | 07:19 | |
| *** hjensas has joined #tripleo | 07:19 | |
| *** hjensas has quit IRC | 07:19 | |
| *** hjensas has joined #tripleo | 07:19 | |
| jaosorior | this effectively lets me set the output to stdout | 07:19 |
| jaosorior | and thus I don't need that sidecar container | 07:19 |
| jaosorior | if we enable logging.conf, it will skip the --config-dir and --config-file options | 07:19 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart master: Cleaning up featureset files from tripleo-ci related variables https://review.openstack.org/512225 | 07:19 |
| jaosorior | and I wouldn't be able to do that | 07:19 |
| *** paramite has joined #tripleo | 07:19 | |
| Tengu | hmm. you might be able to do that. | 07:20 |
| jaosorior | how? | 07:20 |
| Tengu | if I understand, you already point to another nova.conf file using the --config-dir and --config-file right? | 07:20 |
| jaosorior | crap | 07:21 |
| jaosorior | wrong name | 07:21 |
| jaosorior | I meant --log-dir --log-file | 07:21 |
| Tengu | ah :/ | 07:21 |
| jaosorior | sorry | 07:21 |
| Tengu | darn, that changes indeed. | 07:21 |
| jaosorior | yepo | 07:21 |
| * jaosorior needs moar coffee | 07:21 | |
| Tengu | hmm. is there a way to pass another config file maybe? | 07:21 |
| * Tengu has his cup of tea | 07:21 | |
| jaosorior | Tengu: maybe, but then managing configurations becomes more of an issue. | 07:22 |
| *** bandini has quit IRC | 07:22 | |
| Tengu | jaosorior: ah, have an idea. instead of starting the service in the container CMD, push a script that might `sed' the configuration file for the append-log-file line, and point to a service-dedicated logging.conf? | 07:22 |
| jaosorior | Tengu: cause then we would need to be tracking two nova.conf, one for httpd services and another one for the rest | 07:22 |
| *** gfidente[m] has quit IRC | 07:23 | |
| Tengu | (and once it's sed-ed, start the service, of course) | 07:23 |
| *** bandini has joined #tripleo | 07:23 | |
| jaosorior | that MAY work | 07:23 |
| *** jfrancoa has joined #tripleo | 07:23 | |
| jaosorior | but at this point, like I said, I'm a bit more inclined on adding the JSON formatter option to be easily accessible, as that requires less maintenance work | 07:24 |
| Tengu | although one can find it ugly ,) | 07:24 |
| jaosorior | right | 07:24 |
| Tengu | yup. although we might regret that later when someone else comes asking for the support of the logging.conf file in order to provide his/hers own output module :). | 07:25 |
| Tengu | although I'm pretty sure this case won't happen | 07:25 |
| Tengu | but… | 07:25 |
| Tengu | you probably know my good friend Murphy. | 07:25 |
| jaosorior | maybe? | 07:25 |
| marios | jaosorior: ack | 07:26 |
| Tengu | jaosorior: the one with the terrific laws | 07:26 |
| Tengu | jaosorior: https://en.wikipedia.org/wiki/Murphy's_law :) | 07:26 |
| jaosorior | ah, haha | 07:26 |
| jaosorior | yep | 07:27 |
| Tengu | he's a dear, dear friend of mine, and he's never far away enough ;). | 07:27 |
| *** agopi has joined #tripleo | 07:28 | |
| jaosorior | lol | 07:28 |
| jaosorior | marios: I'll change the doc in a subsequent patch. | 07:29 |
| marios | jaosorior: ack | 07:29 |
| Tengu | jaosorior: anyway. I'm not a core, I'm just some external user with some ideas - you (Stackers) chose the best way to get something that can output in JSON :). I'll follow the community on its choice regarding that matter, as long as we can get JSON for fine ELK usage. | 07:30 |
| *** agopi has quit IRC | 07:32 | |
| jaosorior | Tengu: not being a core shouldn't be a limitation. All that being a core means is that you get to approve patches (with the added responsibility that you should be reviewing more than the usual dev). I still think your proposal is a valid use-case and a nice-to-have thing. However, I'm choosing to follow another path for the near-future (adding the JSON formatter as an option to oslo.log), as it requires | 07:33 |
| jaosorior | less work and less maintenance burden. | 07:33 |
| jaosorior | and effectively gets us to the same short-term goal. | 07:33 |
| jaosorior | of course, it does have the flexibility penalty. | 07:33 |
| Tengu | :) | 07:34 |
| jaosorior | Tengu: if you want to add the advanced logging file capability to TripleO, I would +2 your patches though (as long as it's still something that we can turn on/off and allows the current use cases) | 07:35 |
| Tengu | unfortunately, I don't really have much time in order to work on the "logging.conf integration", nor do I have enough insight of the logging part in containers :/ | 07:35 |
| Tengu | but who knows, it might change. | 07:36 |
| openstackgerrit | Juan Badia Payno proposed openstack/puppet-tripleo master: logging: support service_config_settings configuration mechanism https://review.openstack.org/500935 | 07:36 |
| Tengu | I'll keep that under the hood. | 07:36 |
| *** stendulker_ has quit IRC | 07:40 | |
| *** bobh has joined #tripleo | 07:41 | |
| *** radeks has joined #tripleo | 07:44 | |
| *** yprokule has joined #tripleo | 07:44 | |
| *** bobh has quit IRC | 07:45 | |
| *** stendulker_ has joined #tripleo | 07:46 | |
| *** pcaruana has joined #tripleo | 07:47 | |
| *** agopi has joined #tripleo | 07:50 | |
| *** ccamacho has joined #tripleo | 07:52 | |
| jaosorior | Tengu: got it | 08:00 |
| jaosorior | Tengu: looks something like this http://paste.openstack.org/show/625597/ | 08:00 |
| *** skramaja is now known as skramaja|afk | 08:00 | |
| Tengu | coooool | 08:01 |
| jaosorior | Tengu: just did a request | 08:01 |
| jaosorior | trying to fetch it fromt he logs | 08:01 |
| Tengu | :) | 08:02 |
| Tengu | format seems pretty consistent. would be cool to ensure stack traces are in json as well, but I think it should be, according to what I've read in the JSONFormatter. | 08:02 |
| *** agopi has quit IRC | 08:03 | |
| *** anshul has joined #tripleo | 08:04 | |
| jaosorior | Tengu: here's an output with a sample request https://paste.fedoraproject.org/paste/9bo47C~09in2vXF1k02CfA | 08:04 |
| jaosorior | Tengu: I'll cause a bug in keystone to try to get a stacktrace | 08:04 |
| Tengu | ok :) | 08:05 |
| Tengu | wow. verbosity++ | 08:05 |
| Tengu | :) | 08:05 |
| *** ebarrera has joined #tripleo | 08:05 | |
| jaosorior | Tengu: can you give an extra read to those logs just to make sure we don't leak any sensitive info? | 08:06 |
| Tengu | jaosorior: like password and so on? | 08:07 |
| jaosorior | Tengu: right | 08:07 |
| ccamacho | morning folks! | 08:08 |
| *** dsariel has quit IRC | 08:09 | |
| Tengu | hmmm. | 08:09 |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 08:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730179 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 08:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 08:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 08:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 08:10 |
| openstack | Launchpad bug 1730179 in tripleo "overcloud deploy hangs causing CI job timeout" [Critical,Triaged] | 08:10 |
| *** kbyrne has quit IRC | 08:11 | |
| d0ugal | We had a promotion?! | 08:11 |
| Tengu | jaosorior: I'd say it's ok like that. we actually have a "print" of the keystone token, but it's a kind of "session object" and not a password. I think it won't be worse than the standard "debug" output in plain format. | 08:11 |
| jaosorior | Tengu: this contains a stack trace https://paste.fedoraproject.org/paste/RA6AsS3SHPe2bGyfF6aggg | 08:12 |
| Tengu | d0ugal: I think I saw something about hash promotion on openstack-dev ML | 08:12 |
| *** kbyrne has joined #tripleo | 08:12 | |
| jaosorior | Tengu: it's in line 6 | 08:12 |
| d0ugal | Tengu: woo :) | 08:12 |
| Tengu | jaosorior: purrrfect :) | 08:12 |
| jaosorior | Tengu: what do you think? | 08:13 |
| Tengu | jaosorior: I think it's the right output. | 08:13 |
| *** ykarel is now known as ykarel|lunch | 08:15 | |
| jaosorior | Can someone check this out https://review.openstack.org/#/c/517349/ it merely adds an environment file to t-h-t | 08:16 |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Upgrade UC and OC using tripleo-upgrade role https://review.openstack.org/515643 | 08:16 |
| Tengu | jaosorior: apparently, an ELK won't like the list in the stacktrace nor in the standard logs (things between []) - would be better to get "\n" chars instead in the output. But that one is in oslo.log. | 08:17 |
| *** tesseract has joined #tripleo | 08:17 | |
| *** Vijayendra has quit IRC | 08:19 | |
| jaosorior | Tengu: that can be fixed | 08:20 |
| *** mcornea has joined #tripleo | 08:23 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/diskimage-builder master: diskimage_builder: lib: common-functions: Fix options for devpts mount https://review.openstack.org/517886 | 08:24 |
| *** gfidente has joined #tripleo | 08:28 | |
| *** gfidente has quit IRC | 08:28 | |
| *** gfidente has joined #tripleo | 08:28 | |
| *** aufi has joined #tripleo | 08:29 | |
| *** adac has joined #tripleo | 08:29 | |
| *** adac has quit IRC | 08:30 | |
| *** adac has joined #tripleo | 08:30 | |
| *** sri_ has joined #tripleo | 08:33 | |
| *** Vijayendra has joined #tripleo | 08:36 | |
| *** bobh has joined #tripleo | 08:42 | |
| *** jpena|off is now known as jpena | 08:42 | |
| mandre | jaosorior: hey, how good are you with zaqar? | 08:45 |
| jaosorior | mandre: I can try :D | 08:46 |
| jaosorior | not great though | 08:46 |
| jaosorior | mandre: why? | 08:46 |
| mandre | i'll need help debug https://bugs.launchpad.net/tripleo/+bug/1730179 | 08:46 |
| openstack | Launchpad bug 1730179 in tripleo "overcloud deploy hangs causing CI job timeout" [Critical,Triaged] | 08:46 |
| jaosorior | therve: might be around though | 08:46 |
| *** bobh has quit IRC | 08:46 | |
| jaosorior | mandre: sure, I can give it a try | 08:46 |
| mandre | jaosorior: FYI, it just happened for you in legacy-tripleo-ci-centos-7-scenario003-multinode-oooq for https://review.openstack.org/#/c/510464/ | 08:47 |
| jaosorior | Tengu: by the way, it seems the standard logs are not actually enclosed between [], only the stacktraces, and that's something that we can easily configure once we expose another parameter. | 08:47 |
| *** gfidente has quit IRC | 08:47 | |
| therve | Heya | 08:47 |
| mandre | therve: hey! | 08:47 |
| jaosorior | therve: seems we need zaqar help :D | 08:47 |
| therve | That looks like the IPv6 error | 08:48 |
| mandre | therve: how can you tell? | 08:48 |
| Tengu | jaosorior: hmm, right, although there's an "extra" key in the json that contains a hash - we'll test on our side how's the ELK eats that and open an issue against oslo.log once we have all the information. | 08:48 |
| therve | mandre, Because I fixed it last week :) | 08:48 |
| *** amoralej|off is now known as amoralej | 08:48 | |
| jaosorior | therve: isn't this the fix https://review.openstack.org/#/c/516983/ ? | 08:48 |
| Tengu | jaosorior: guess it's the best thing to do so that we ensure we can provide all the info and avoid possible useless issue ;) | 08:48 |
| therve | jaosorior, Only for containerized undercloud | 08:49 |
| therve | Hum.... | 08:49 |
| therve | Looks like something dumb happened | 08:49 |
| mandre | therve: the other one would be in instack-undercloud? | 08:49 |
| therve | mandre, Yeah | 08:49 |
| *** gfidente has joined #tripleo | 08:49 | |
| therve | See http://logs.openstack.org/61/517661/1/gate/legacy-tripleo-ci-centos-7-nonha-multinode-oooq/adad4df/logs/undercloud/etc/zaqar/zaqar.conf.txt.gz | 08:50 |
| mandre | therve: https://review.openstack.org/#/c/515123/ | 08:50 |
| therve | It says notification-bind instead of notication_bind ... | 08:50 |
| mandre | hmm | 08:50 |
| mandre | therve: are you working on a patch for puppet-zaqar? | 08:55 |
| *** dbecker has joined #tripleo | 08:55 | |
| mandre | therve: ok, I went ahead and submitted https://review.openstack.org/517889 | 08:59 |
| *** jidar has quit IRC | 09:00 | |
| *** jidar has joined #tripleo | 09:00 | |
| therve | mandre, Sorry you're too fast :) | 09:02 |
| mandre | therve: i added you as a coauthor, good spot on the dash vs underscore | 09:03 |
| jaosorior | mandre: you should probably update this patch https://review.openstack.org/#/c/516983/ and add a dependson to your patch | 09:03 |
| therve | mandre, Yeah... Too bad I introduced it... | 09:03 |
| mandre | jaosorior: that would be for *all* tripleo patches though | 09:04 |
| therve | But tested locally, that's the issue... | 09:05 |
| therve | mandre, It's only a problem in a cloud provider though | 09:05 |
| *** ykarel|lunch is now known as ykarel | 09:05 | |
| *** Vijayendra has quit IRC | 09:06 | |
| mandre | right, but that means that if we get a node that has ipv6 in the gate, the job will timeout and eventually fail | 09:06 |
| *** Vijayendra has joined #tripleo | 09:06 | |
| mandre | that's what killed us last week | 09:06 |
| mandre | well... part of it at least :) | 09:06 |
| therve | Yeah that's why I tried to fix it... | 09:06 |
| *** lucas-afk is now known as lucasagomes | 09:07 | |
| mandre | i'm tempted to kill the queue in order to merge https://review.openstack.org/#/c/517889/ in priority | 09:08 |
| jaosorior | mandre: do you have +2 for puppet-zaqar | 09:08 |
| jaosorior | ? | 09:08 |
| *** jpich has joined #tripleo | 09:09 | |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 09:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730179 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 09:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 09:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 09:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 09:10 |
| openstack | Launchpad bug 1730179 in tripleo "overcloud deploy hangs causing CI job timeout" [Critical,Triaged] | 09:10 |
| mandre | jaosorior: i don't :-/ | 09:10 |
| mandre | jaosorior: only the puppet-manager-core can, https://review.openstack.org/#/admin/groups/134,members | 09:11 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-heat-templates master: Start using nic-configs from THT repo instead of tripleo-ci https://review.openstack.org/476708 | 09:11 |
| mandre | jaosorior: we only have in puppet-tripleo i believe | 09:12 |
| jaosorior | that's right | 09:12 |
| *** gfidente[m] has joined #tripleo | 09:14 | |
| *** noslzzp has quit IRC | 09:14 | |
| mandre | jaosorior: i have a question for you... while debugging the zaqar issue I noticed that swift doesn't log to disk, is that something you've seen? | 09:15 |
| *** zshi has quit IRC | 09:16 | |
| jaosorior | mandre: yes, it's the default. It logs to syslog | 09:16 |
| *** zshi has joined #tripleo | 09:16 | |
| mandre | jaosorior: ah makes sense, thx | 09:16 |
| *** skramaja|afk is now known as skramaja | 09:16 | |
| jaosorior | mandre: they don't use oslo.log, instead they explicitly set the logger to syslog. It's not as trivial to modify unfortunately :/ | 09:17 |
| *** openstackgerrit has quit IRC | 09:18 | |
| *** pblaho has joined #tripleo | 09:19 | |
| Tengu | :] there are still projects that don't use oslo.log? baaad :) | 09:19 |
| * Tengu runs away | 09:19 | |
| *** moshele has joined #tripleo | 09:21 | |
| *** tosky has joined #tripleo | 09:22 | |
| jaosorior | Tengu: https://github.com/openstack/swift/blob/master/swift/common/utils.py#L1855 :( | 09:23 |
| Tengu | darn | 09:24 |
| mandre | jaosorior: another question for you, the /var/log/messages in CI is littered with "error: Could not load host key: /etc/ssh/ssh_host_dsa_key", do you have an idea where it's coming from | 09:25 |
| *** gfidente has quit IRC | 09:26 | |
| *** dsariel has joined #tripleo | 09:26 | |
| Tengu | probably the sshd service that didn't generate the proper dsa_key while trying to load it. | 09:26 |
| Tengu | dsa is deprecated. | 09:26 |
| Tengu | mandre: -^ | 09:26 |
| mandre | Tengu: maybe somewhat related to https://review.openstack.org/#/c/516785/? | 09:26 |
| Tengu | I doubt it. | 09:27 |
| therve | mandre, What's the sshd_config? | 09:28 |
| Tengu | mandre: the error you reported is for sshd_config, and has nothing to do with the authorized_keys :). | 09:28 |
| jaosorior | mandre: nope | 09:28 |
| *** derekh has joined #tripleo | 09:29 | |
| *** gfidente has joined #tripleo | 09:29 | |
| *** gfidente has quit IRC | 09:29 | |
| *** gfidente has joined #tripleo | 09:29 | |
| jaosorior | mandre: don't know where it's coming fro. | 09:29 |
| jaosorior | *from | 09:29 |
| Tengu | mandre: the error about /etc/ssh/ssh_host_dsa_key is a simple "warning" (that won't prevent sshd to start, although it's flagged as "error") and can be corrected by editing the /etc/ssh/sshd_config and dropping the dsa entry. | 09:29 |
| jaosorior | uhm, would be wroth checking that out | 09:29 |
| mandre | right it seems to be just a warning, and I think that's when zuul connects to the host when doing its ansible stuff | 09:30 |
| mandre | the sshd config, http://logs.openstack.org/64/510464/23/check/legacy-tripleo-ci-centos-7-scenario003-multinode-oooq/6519e2c/logs/undercloud/etc/ssh/sshd_config.gz | 09:30 |
| *** Vijayendra has quit IRC | 09:31 | |
| Tengu | jaosorior, mandre : you might want to fix the "HostKey" option to a more recent algo | 09:31 |
| *** Vijayendra has joined #tripleo | 09:32 | |
| Tengu | format is: "HostKey /etc/ssh/ssh_host_rsa_key" for example. Option can be added multiple time | 09:32 |
| mandre | Tengu: thanks, now the game is to find where it's set :) i'll check with infra | 09:33 |
| Tengu | :) | 09:33 |
| Tengu | by default, recent sshd generate the following keys: /etc/ssh/ssh_host_rsa_key /etc/ssh/ssh_host_ecdsa_key and, apparently, /etc/ssh/ssh_host_ed25519_key - might be good to check that in order to avoid having another error to replace the DSA ;) | 09:34 |
| Tengu | at least the RSA one is present everywhere. | 09:34 |
| Tengu | mandre: apparently it's configured with puppet. | 09:35 |
| Tengu | according to the header. | 09:35 |
| Tengu | might come from the puppet-tripleo directly | 09:36 |
| *** abregman is now known as abregman|mtg | 09:36 | |
| jaosorior | marios, bandini: Could you checkt his out https://review.openstack.org/#/q/topic:bug/1729293+status:open ? | 09:36 |
| Tengu | mandre: https://github.com/openstack/puppet-tripleo/blob/d5a23606eef9fcf64313a3d4e1b5455e8c0610dc/manifests/profile/base/sshd.pp probably the best guess I can make :) | 09:37 |
| jaosorior | Tengu: so, we should set it up here I guess https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/sshd.pp | 09:37 |
| jaosorior | lol | 09:37 |
| Tengu | jaosorior: «o/ | 09:37 |
| mandre | jaosorior, Tengu, you guys want to take care of it? | 09:37 |
| Tengu | guess we should provide a default value for "options". | 09:38 |
| Tengu | mandre: guess we can ;). jaosorior do you want me to provide a patch for that one? | 09:39 |
| *** gfidente has quit IRC | 09:39 | |
| Tengu | hmm. what puppet-ssh module is in use o_O | 09:40 |
| *** gfidente has joined #tripleo | 09:41 | |
| *** gfidente has quit IRC | 09:41 | |
| *** gfidente has joined #tripleo | 09:41 | |
| Tengu | ah, saz. | 09:41 |
| *** gfidente has quit IRC | 09:41 | |
| *** gfidente[m] has quit IRC | 09:42 | |
| *** gfidente[m] has joined #tripleo | 09:42 | |
| *** bobh has joined #tripleo | 09:43 | |
| Tengu | mandre: do you have some launchpad issue regarding this sshd noise? | 09:44 |
| * Tengu ready to do a commit + review | 09:44 | |
| mandre | Tengu: no, I didn't create one, and I haven't checked on LP either if one already existed | 09:45 |
| Tengu | apparently… nothing matches sshd hostkey in tripleo issues. | 09:45 |
| Tengu | yup, no issue. would be good to create one I guess? | 09:46 |
| mandre | yep | 09:46 |
| Tengu | care to create it, mandre ? | 09:46 |
| mandre | sure I can do it | 09:47 |
| *** bobh has quit IRC | 09:47 | |
| Tengu | cool. link it to me so that I can put its reference in my commit, and it's done :) | 09:47 |
| *** chem has joined #tripleo | 09:50 | |
| *** gfidente has joined #tripleo | 09:50 | |
| mandre | Tengu: https://bugs.launchpad.net/tripleo/+bug/1730351 | 09:52 |
| openstack | Launchpad bug 1730351 in tripleo "sshd complains about missing /etc/ssh/ssh_host_dsa_key" [Medium,Triaged] | 09:52 |
| Tengu | taken, updating unit tests, and I'll push it, mandre | 09:52 |
| d0ugal | apetrich: https://github.com/openstack/tripleo-common-tempest-plugin | 09:52 |
| mandre | awesome, thanks Tengu | 09:53 |
| Tengu | np, small change ;) | 09:53 |
| Tengu | running the unit test in order to ensure there isn't any error with that. | 09:54 |
| apetrich | d0ugal, woot! | 09:55 |
| Tengu | btw, my profile::base::apache thing will probably never merge, seeing how it goes with the ci :(. | 09:55 |
| apetrich | d0ugal, but still having issues with my env | 09:55 |
| d0ugal | apetrich: dang, I got one working. | 09:55 |
| *** salmankhan has joined #tripleo | 09:55 | |
| apetrich | d0ugal, I think this time it might work though. it passed where it was breaking | 09:55 |
| *** etingof has joined #tripleo | 09:56 | |
| d0ugal | apetrich: well, mostly, the overcloud deploy failed (but I don't need that today) | 09:56 |
| apetrich | but I forgot to git pull oooq though | 09:56 |
| d0ugal | nice | 09:56 |
| apetrich | d0ugal, oh I jinxed | 09:57 |
| apetrich | instant fail after saying that :) | 09:57 |
| d0ugal | lol | 09:57 |
| d0ugal | apetrich: where is yours failing? | 09:57 |
| apetrich | d0ugal, last week on the (sic) Run the validate IPMI overclcoud connection script | 09:58 |
| *** hjensas has quit IRC | 10:00 | |
| *** cshastri has quit IRC | 10:00 | |
| *** openstackgerrit has joined #tripleo | 10:01 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/puppet-tripleo master: Ensure sshd has proper configuration for its HostKey. https://review.openstack.org/517903 | 10:01 |
| Tengu | mandre: -^^ | 10:01 |
| *** mdnadeem has quit IRC | 10:01 | |
| Tengu | unit tests needed an update, and it's apparently working as expected. let's see what the CI has to say. | 10:01 |
| sri_ | Tengu: hi , quick question, is barbican working in stable/pike release ? | 10:02 |
| jaosorior | sri_: no | 10:03 |
| Tengu | sri_: I'd say "nope for prod" | 10:03 |
| Tengu | ah :). | 10:03 |
| Tengu | thank you jaosorior - wasn't 100% sure. | 10:03 |
| jaosorior | sri_: It'll come for queens. | 10:03 |
| *** pblaho has quit IRC | 10:03 | |
| jaosorior | no biggie | 10:03 |
| jaosorior | Tengu: thanks for the ssh commit | 10:03 |
| *** hjensas has joined #tripleo | 10:04 | |
| *** hjensas has quit IRC | 10:04 | |
| *** hjensas has joined #tripleo | 10:04 | |
| Tengu | np, was easy. if one could just confirm I do it "right" regarding override capabilities and so on | 10:04 |
| *** ukalifon has joined #tripleo | 10:05 | |
| sri_ | jaosorior: ok , is there any post-deployment scripts to make barbican work in pike release ? thanks | 10:05 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-docs master: Update upgrade documentation with the Ceph related steps https://review.openstack.org/511635 | 10:05 |
| jaosorior | sri_: not any official ones. | 10:05 |
| jaosorior | sri_: so you can deploy the barbican-api (non-containerized IIRC) but it doesn't deploy with any security-sensitive options, and it's not really supported. it will oficially come in queens. | 10:06 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart master: Set quickstart to use ansible 2.4.1 https://review.openstack.org/510059 | 10:08 |
| sri_ | jaosorior: I am trying with non-containerzed , I've successfully deployed my overcloud by enable service/barbican.yaml , but when I am tring to use barbican api I am gettting this error : Unable to establish connection to http://192.168.15.122:9311/secrets/: ('Connection aborted.', BadStatusLine("''",) | 10:09 |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 10:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subcriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 10:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 10:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 10:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 10:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 10:10 |
| sri_ | jaosorior: i see what you're saying , api is there but it's functional !! | 10:11 |
| jaosorior | sri_: REALLY suggest you don't use that unless you know in-depth how to configure barbican properly. | 10:11 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-heat-templates master: DONT REVIEW: this is dependency patch https://review.openstack.org/446592 | 10:11 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM. test ansible 2.4 with tht https://review.openstack.org/510061 | 10:11 |
| sri_ | jaosorior: got it . | 10:12 |
| Tengu | jaosorior: btw, barbican will use custodia as default backend, right? | 10:12 |
| *** gfidente has quit IRC | 10:13 | |
| jaosorior | Tengu: we're considering the options. | 10:13 |
| Tengu | jaosorior: ok. what's the other option? I'm thinking about "how will it clusterize?" mainly. | 10:13 |
| Tengu | jaosorior: another idea: dedicated DB in mysql with state-of-the-art encryption for data. | 10:13 |
| Tengu | as galera is already there for the clustering. | 10:14 |
| jaosorior | Tengu: it is possible to do that with the simple_crypto backend. The main drawback is that the KEK is in barbican.conf | 10:14 |
| jaosorior | I'm not a fan, but it's a possibility | 10:15 |
| Tengu | ah, yes, custodia might "wrap" all of that. hmm, "KEK"? already heard that, but brain's off. | 10:15 |
| jaosorior | Key Encryption Key | 10:15 |
| Tengu | i.e. the encryption key for encrypted data? | 10:15 |
| jaosorior | right | 10:16 |
| Tengu | in custodia it's a dedicated file that can be pointed to from its configuration file. | 10:16 |
| Tengu | so yep, that might be an issue. | 10:16 |
| Tengu | and double-encryption is overkill and might even lesser the security. | 10:17 |
| Tengu | hmm. "lower" | 10:17 |
| Tengu | … anyway. | 10:17 |
| *** bogdando has joined #tripleo | 10:17 | |
| jaosorior | anyway, trust me, I would prefer custodia there | 10:17 |
| Tengu | :) I bet you do. | 10:17 |
| jaosorior | but too much work to do and too little people to do it :/ | 10:17 |
| Tengu | hmm. on that I might provide some help maybe? | 10:18 |
| jaosorior | gotta ask raildo how's that going once he logs in | 10:18 |
| Tengu | as I already have a "custodia" client lib in python for certodia, as well as a simple ansible snippet for custodia deploy? | 10:18 |
| *** mdnadeem has joined #tripleo | 10:18 | |
| jaosorior | uhm... that might be useful. Let me poke raildo when he logs in. He knows better the status of that work. | 10:19 |
| Tengu | cool | 10:19 |
| Tengu | ping me back when needed :). | 10:19 |
| *** adac has quit IRC | 10:21 | |
| *** gfidente has joined #tripleo | 10:24 | |
| *** gfidente has quit IRC | 10:24 | |
| *** gfidente has joined #tripleo | 10:24 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates master: Add condition to os-net-config run during upgrade. https://review.openstack.org/517905 | 10:25 |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: Fix tempest html generation file https://review.openstack.org/517906 | 10:29 |
| *** adac has joined #tripleo | 10:34 | |
| *** ckyriakidou has joined #tripleo | 10:36 | |
| mandre | mwhahaha, EmilienM: please take a look at https://review.openstack.org/#/c/517889/ at your earliest convenience, it's supposed to fix the frequent timeouts we've seen in CI | 10:38 |
| *** ykarel is now known as ykarel|afk | 10:41 | |
| *** bobh has joined #tripleo | 10:43 | |
| *** udesale has quit IRC | 10:44 | |
| *** brault has quit IRC | 10:44 | |
| *** brault has joined #tripleo | 10:45 | |
| *** pblaho has joined #tripleo | 10:46 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates master: Fix wrong permission on database during mysql_init tasks. https://review.openstack.org/517909 | 10:46 |
| *** bobh has quit IRC | 10:47 | |
| *** brault has quit IRC | 10:49 | |
| *** mrch has joined #tripleo | 10:50 | |
| *** fzdarsky has joined #tripleo | 10:50 | |
| *** pblaho has quit IRC | 10:51 | |
| *** anilvenkata has quit IRC | 10:54 | |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: Fix tempest html generation file https://review.openstack.org/517906 | 10:54 |
| *** abregman|mtg is now known as abregman | 10:56 | |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: Fix tempest html generation file https://review.openstack.org/517906 | 10:59 |
| *** pgadiya has quit IRC | 10:59 | |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: Fix tempest html generation file https://review.openstack.org/517906 | 11:02 |
| *** noslzzp has joined #tripleo | 11:02 | |
| *** pblaho has joined #tripleo | 11:04 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-docs master: WIP: Upgrades developer documentation layout. https://review.openstack.org/517916 | 11:04 |
| *** pblaho has quit IRC | 11:06 | |
| *** pblaho has joined #tripleo | 11:06 | |
| *** ebarrera has quit IRC | 11:08 | |
| *** ebarrera has joined #tripleo | 11:09 | |
| *** rfolco has joined #tripleo | 11:10 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 11:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 11:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 11:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 11:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 11:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 11:10 |
| openstackgerrit | Moshe Levi proposed openstack/tripleo-heat-templates master: Adds environment file for OVS Hardware Offload https://review.openstack.org/507401 | 11:11 |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart master: Cleaning up featureset files from tripleo-ci related variables https://review.openstack.org/512225 | 11:11 |
| openstackgerrit | Damien Ciabrini proposed openstack/puppet-tripleo stable/pike: Add new MySQL server option to mysql_bundle https://review.openstack.org/517920 | 11:15 |
| jaosorior | jistr: could you review this if you have a chance https://review.openstack.org/#/q/topic:bug/1729293+status:open ? | 11:15 |
| *** atoth has quit IRC | 11:17 | |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: Fix tempest html generation file https://review.openstack.org/517906 | 11:17 |
| *** adac has quit IRC | 11:20 | |
| jpich | I'm trying to set up an ovb environment on RDO cloud with quickstart, however my deployments always fail with "FAILURE: instackenv validator found 2 errors\nFailure to contact below addresses IPMI Address : (etc)". Any tip on how I might go with debugging this? Is there something I should configure or check before running quickstart in this case? | 11:22 |
| *** stendulker_ has quit IRC | 11:24 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: Remove references to the current task from task() in baremetal.yaml https://review.openstack.org/516257 | 11:29 |
| *** shreshtha has quit IRC | 11:29 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common master: Migrate to the new Mistral context class https://review.openstack.org/506186 | 11:31 |
| *** adac has joined #tripleo | 11:32 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Disable live migration over TLS https://review.openstack.org/517927 | 11:33 |
| *** jaganathan has quit IRC | 11:34 | |
| *** florianf has joined #tripleo | 11:35 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Install tripleo Ci admins keys on hosts https://review.openstack.org/517928 | 11:37 |
| chem | jfrancoa: so for https://review.openstack.org/#/c/517916/1, I don't really know the best practice, but if we're going to split this up | 11:38 |
| chem | jfrancoa: we need some placeholder like # todo or something | 11:39 |
| chem | jfrancoa: and we create all empty review for each part under the same topic ? | 11:39 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: test dstat https://review.openstack.org/517929 | 11:41 |
| jfrancoa | chem: ok, I'll add it. But I wanted to get your feedback first, agree on the complete layout for it (I probably missed something) and then land it with a #TODO so we can split up the work (it's quite a lot of sections) | 11:43 |
| *** bobh has joined #tripleo | 11:44 | |
| *** jkilpatr has joined #tripleo | 11:45 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Install tripleo Ci admins keys on hosts https://review.openstack.org/517928 | 11:47 |
| *** bobh has quit IRC | 11:49 | |
| *** pkovar has joined #tripleo | 11:49 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Translate tripleo-ci logic of when to build images to ansible https://review.openstack.org/511940 | 11:50 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: cinder: switch CinderCronDbPurgeUser to 'cinder' https://review.openstack.org/517412 | 11:50 |
| *** rbrady has quit IRC | 11:51 | |
| Tengu | jaosorior: care to add a +2 on https://review.openstack.org/#/c/517903/ ? Not sure it has kicked in the CI for now. | 11:52 |
| jaosorior | Tengu: was waiting for CI | 11:52 |
| jistr | jaosorior: so how will it look when barbican is introduced? Will we have ceilometer_notification_topics, barbican_notification_topics, which will both be automatically enabled by including the service, and then extra_notification_topics for user configuration? | 11:52 |
| jaosorior | jistr: correct. | 11:53 |
| Tengu | jaosorior: ah, ok :) | 11:53 |
| Tengu | no problem | 11:53 |
| jaosorior | jistr: I wanted to have just one parameter; my initial commit was using hiera_array to fetch all the places where services might have set a notification queue topic, but it's currently not possible to do that via t-h-t since there is a deep map merge that overwrites this :/ | 11:54 |
| jaosorior | jistr: so I ended up going for that solution. | 11:54 |
| *** udesale has joined #tripleo | 11:58 | |
| *** udesale has quit IRC | 11:58 | |
| *** udesale has joined #tripleo | 11:58 | |
| *** pdeore has quit IRC | 11:59 | |
| *** noslzzp has quit IRC | 12:02 | |
| *** atoth has joined #tripleo | 12:02 | |
| *** raildo has joined #tripleo | 12:02 | |
| *** fzdarsky has quit IRC | 12:02 | |
| *** ykarel|afk is now known as ykarel | 12:03 | |
| *** salmankhan has quit IRC | 12:03 | |
| *** noslzzp has joined #tripleo | 12:04 | |
| *** ratailor has quit IRC | 12:04 | |
| jistr | jaosorior: left a comment there https://review.openstack.org/#/c/516969/2/manifests/profile/base/keystone.pp | 12:06 |
| jistr | jaosorior: there is an alternative but i'm not sure it's worth it, i left it for your consideration but i +2'd so if you think what we already have is better fit, we can +A | 12:07 |
| openstackgerrit | Wojciech Dec proposed openstack/puppet-tripleo master: Adding manifest for Cisco VTS ML2 mechanism driver configuration https://review.openstack.org/510645 | 12:07 |
| jaosorior | jistr: thanks | 12:08 |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 12:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 12:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 12:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 12:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 12:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 12:10 |
| *** zshi_ has joined #tripleo | 12:10 | |
| *** adarazs is now known as adarazs_lunc | 12:11 | |
| *** adarazs_lunc is now known as adarazs_lunch | 12:11 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: test dstat https://review.openstack.org/517929 | 12:12 |
| *** mdnadeem has quit IRC | 12:15 | |
| *** zshi_ has quit IRC | 12:15 | |
| *** salmankhan has joined #tripleo | 12:16 | |
| *** lucasagomes is now known as lucas-hungry | 12:16 | |
| *** noslzzp has quit IRC | 12:16 | |
| *** ansmith has quit IRC | 12:19 | |
| *** radeks has quit IRC | 12:20 | |
| Tengu | thanks mandre and jaosorior for the votes. mandre does it solve your issue for good? | 12:22 |
| Tengu | hmm apparently, no sshd complaining in the undercloud_install.log. | 12:23 |
| jaosorior | Tengu: thanks for the patch | 12:25 |
| mandre | Tengu: it does, yes | 12:25 |
| Tengu | mandre: cool :). This one was an easy fix. | 12:25 |
| mandre | Tengu: keep sending easy fixes :) we love them | 12:26 |
| openstackgerrit | Moshe Levi proposed openstack/tripleo-heat-templates master: Adds environment file for OVS Hardware Offload https://review.openstack.org/507401 | 12:26 |
| *** mdnadeem has joined #tripleo | 12:28 | |
| Tengu | ^^ | 12:29 |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-quickstart-extras master: Fix a typo https://review.openstack.org/517974 | 12:29 |
| *** suuuper has quit IRC | 12:31 | |
| *** anilvenkata has joined #tripleo | 12:34 | |
| *** ccamacho has quit IRC | 12:41 | |
| *** rhallisey has joined #tripleo | 12:41 | |
| *** jlabarre has joined #tripleo | 12:43 | |
| *** mdnadeem has quit IRC | 12:43 | |
| *** pblaho has quit IRC | 12:43 | |
| *** anilvenkata has quit IRC | 12:44 | |
| *** pdeore has joined #tripleo | 12:47 | |
| *** brault has joined #tripleo | 12:47 | |
| *** anilvenkata has joined #tripleo | 12:47 | |
| *** anilvenkata has quit IRC | 12:48 | |
| *** anilvenkata has joined #tripleo | 12:48 | |
| jaosorior | dciabrin: are we using the systemd wrapper for haproxy in the container? | 12:52 |
| *** pchavva has joined #tripleo | 12:52 | |
| dciabrin | jaosorior, yes we are | 12:52 |
| jaosorior | great | 12:52 |
| jaosorior | so I can just send a sighup and it'll work? | 12:53 |
| *** shreshtha has joined #tripleo | 12:53 | |
| *** skramaja has quit IRC | 12:53 | |
| *** jlabarre has quit IRC | 12:53 | |
| dciabrin | yes a docker kill -HUP should restart haproxy without restarting the container | 12:53 |
| *** jlabarre has joined #tripleo | 12:54 | |
| Tengu | khof | 12:54 |
| jaosorior | dciabrin: cool cool | 12:54 |
| Tengu | ah, you don't care about the connections already in palce. | 12:54 |
| Tengu | *place | 12:54 |
| Tengu | duh… timeout in the CI for my sshd patch :'( | 12:55 |
| *** brault has quit IRC | 12:55 | |
| *** brault has joined #tripleo | 12:56 | |
| *** ccamacho has joined #tripleo | 12:57 | |
| *** rbrady has joined #tripleo | 12:57 | |
| *** rbrady has quit IRC | 12:57 | |
| *** rbrady has joined #tripleo | 12:57 | |
| *** eck`gone is now known as eck` | 12:58 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Send ARA statistics to Graphite server https://review.openstack.org/479882 | 12:59 |
| *** abishop has joined #tripleo | 12:59 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Remove certificate before updating it https://review.openstack.org/517984 | 13:00 |
| *** brault_ has joined #tripleo | 13:00 | |
| *** brault has quit IRC | 13:01 | |
| *** jpena is now known as jpena|lunch | 13:01 | |
| *** percevalbot has quit IRC | 13:02 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Send ARA statistics to Graphite server https://review.openstack.org/479882 | 13:04 |
| *** adarazs_lunch is now known as adarazs|rover | 13:05 | |
| *** catintheroof has joined #tripleo | 13:06 | |
| *** percevalbot has joined #tripleo | 13:07 | |
| *** pcaruana has quit IRC | 13:08 | |
| *** tvignaud has quit IRC | 13:08 | |
| *** tvignaud has joined #tripleo | 13:08 | |
| *** florianf has quit IRC | 13:10 | |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 13:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 13:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 13:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 13:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 13:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 13:10 |
| *** brault_ has quit IRC | 13:10 | |
| *** pblaho has joined #tripleo | 13:11 | |
| *** trown|outtypewww is now known as trown | 13:11 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: send graphite stats from ARA https://review.openstack.org/480121 | 13:12 |
| *** catintheroof has quit IRC | 13:12 | |
| beagles | does anyone know of a way of finding whether a service has been deployed as a container either in heat templates or in a mistral workflow? | 13:13 |
| beagles | trying to find a clean way of adding a conditional depending on whether neutron is being deployed as a container or not | 13:13 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: send graphite stats from ARA https://review.openstack.org/480121 | 13:14 |
| *** lucas-hungry is now known as lucasagomes | 13:14 | |
| *** catintheroof has joined #tripleo | 13:14 | |
| beagles | d0ugal, do you know of a way to do this in the workflows ^ | 13:16 |
| d0ugal | beagles: no, sorry | 13:16 |
| beagles | d0ugal, cool, thanks! | 13:16 |
| d0ugal | beagles: I don't know very much about the container work - no idea how you'd manually check to see how a service is deploy. | 13:17 |
| d0ugal | beagles: if you can tell me that, maybe I can think of a way to do it in Mistral | 13:17 |
| beagles | d0ugal, ack I think we do a pretty good job of "hiding" whether a service is being deployed as a container or not :) | 13:18 |
| beagles | d0ugal, unfortunately it is working against me in this case | 13:18 |
| d0ugal | heh, I guess it makes generally tho' | 13:19 |
| beagles | yep :) | 13:19 |
| d0ugal | Can we "ask" k8s? | 13:19 |
| d0ugal | via an API or CLI call | 13:19 |
| *** percevalbot has quit IRC | 13:19 | |
| d0ugal | but maybe you are hoping to do this before the deploy | 13:20 |
| beagles | not part of the picture actually in this case - going to sift through the heat stack data to see if there is something there | 13:20 |
| *** pcaruana has joined #tripleo | 13:20 | |
| *** brault has joined #tripleo | 13:20 | |
| beagles | d0ugal, during the deploy actually | 13:20 |
| d0ugal | beagles: right. Well, let me know if you find something! I am curious now. | 13:21 |
| beagles | d0ugal, will do :) | 13:21 |
| *** agurenko has quit IRC | 13:21 | |
| *** percevalbot has joined #tripleo | 13:22 | |
| *** bryan_att has quit IRC | 13:23 | |
| *** NobodyCam has quit IRC | 13:23 | |
| *** gregwork has quit IRC | 13:23 | |
| *** sdake has quit IRC | 13:23 | |
| *** bryan_att has joined #tripleo | 13:24 | |
| *** NobodyCam has joined #tripleo | 13:24 | |
| *** gregwork has joined #tripleo | 13:24 | |
| *** sdake has joined #tripleo | 13:25 | |
| *** sdake has joined #tripleo | 13:25 | |
| *** agurenko has joined #tripleo | 13:26 | |
| *** vpickard_ is now known as vpickard | 13:26 | |
| *** brault has quit IRC | 13:27 | |
| *** brault has joined #tripleo | 13:28 | |
| *** owalsh has joined #tripleo | 13:28 | |
| *** panda|ruck|off is now known as panda|ruck | 13:29 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/instack-undercloud master: Make sure selinux permissions are correct on ~/.ssh. https://review.openstack.org/495157 | 13:30 |
| *** pkovar has quit IRC | 13:31 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-common master: Add Barbican worker and keystone notifier images https://review.openstack.org/517076 | 13:31 |
| *** cylopez has quit IRC | 13:33 | |
| *** cylopez has joined #tripleo | 13:33 | |
| *** brault has quit IRC | 13:33 | |
| bogdando | folks PTAL https://review.openstack.org/#/c/513669 | 13:33 |
| bogdando | a partial fix for selinux enforcing. The remaining work is t-h-t only | 13:34 |
| bogdando | those docker/services | 13:34 |
| *** dougbtv__ has joined #tripleo | 13:35 | |
| *** dougbtv__ has quit IRC | 13:35 | |
| jaosorior | bogdando: excellent! | 13:36 |
| *** dougbtv__ has joined #tripleo | 13:36 | |
| bogdando | jaosorior: thanks | 13:36 |
| bogdando | and the follow-up is the *hard one* https://review.openstack.org/#/c/517383 | 13:37 |
| bogdando | I have no a clue how to make -v /run:/run and /dev:/dev selinux friendly | 13:37 |
| openstackgerrit | Or Idgar proposed openstack/tripleo-common master: Add ansible playbook for configuring octavia https://review.openstack.org/487450 | 13:38 |
| *** dougbtv_ has quit IRC | 13:39 | |
| *** brault has joined #tripleo | 13:39 | |
| *** udesale has quit IRC | 13:40 | |
| *** jaosorior has quit IRC | 13:40 | |
| *** pchavva has quit IRC | 13:40 | |
| *** dougbtv_ has joined #tripleo | 13:41 | |
| *** jaosorior has joined #tripleo | 13:42 | |
| *** zshi has quit IRC | 13:42 | |
| *** dougbtv__ has quit IRC | 13:43 | |
| *** aputtur_ has joined #tripleo | 13:43 | |
| *** liverpooler has joined #tripleo | 13:44 | |
| *** ansmith has joined #tripleo | 13:47 | |
| *** janki has quit IRC | 13:47 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/instack-undercloud stable/newton: undercloud/upgrade: change ownership for ironic-dbsync.log https://review.openstack.org/517356 | 13:47 |
| *** brault has quit IRC | 13:47 | |
| *** brault has joined #tripleo | 13:48 | |
| *** ykarel is now known as ykarel|afk | 13:48 | |
| *** lblanchard has joined #tripleo | 13:51 | |
| *** brault has quit IRC | 13:53 | |
| *** jpena|lunch is now known as jpena | 13:55 | |
| *** ykarel|afk has quit IRC | 13:55 | |
| *** lblanchard has quit IRC | 13:56 | |
| *** brault has joined #tripleo | 13:57 | |
| *** toure_biab is now known as toure | 13:58 | |
| *** akrivoka has joined #tripleo | 14:02 | |
| *** pcaruana has quit IRC | 14:05 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Allow containerized undercloud deploy with SELinux https://review.openstack.org/513669 | 14:05 |
| *** thrash|g0ne is now known as thrash | 14:06 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-docs master: Add cleanup checks in the upgrade doc https://review.openstack.org/389609 | 14:06 |
| *** dhill_ has joined #tripleo | 14:07 | |
| openstackgerrit | Andy Smith proposed openstack/tripleo-heat-templates master: WIP Separate rpc and notify messaging backends https://review.openstack.org/507963 | 14:07 |
| *** ecerquei has joined #tripleo | 14:07 | |
| *** cshastri has joined #tripleo | 14:08 | |
| *** amoralej is now known as amoralej|lunch | 14:09 | |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 14:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 14:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 14:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 14:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 14:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 14:10 |
| *** zshi_ has joined #tripleo | 14:10 | |
| jaosorior | dciabrin: you around? | 14:11 |
| jaosorior | or bandini | 14:13 |
| trown | sshnaidm: my issue is around needing to do some tasks only on subnode-0 | 14:16 |
| trown | sshnaidm: the repo installation is 1 | 14:16 |
| trown | sshnaidm: since we dont have any task that would remove epel from other subnodes, we only want to put it on subnode-0 | 14:16 |
| trown | sshnaidm: the other is to do with installing ssh keys, since we want to install the key from one host on the others | 14:17 |
| sshnaidm | trown, some of multinode bootstrapping scripts remove epel from subnodes too | 14:17 |
| trown | sshnaidm: did you solve these things already? with delegate? | 14:17 |
| *** pcaruana has joined #tripleo | 14:17 | |
| trown | sshnaidm: ah ok... so that one maybe easy | 14:17 |
| trown | sshnaidm: the ssh key one is still tricky though | 14:18 |
| *** jmelvin has joined #tripleo | 14:18 | |
| bandini | jaosorior: sortof :) | 14:19 |
| sshnaidm | trown, take a look https://github.com/sshnaidm/various/tree/master/roles/nodepool-simulation | 14:19 |
| sshnaidm | trown, I'm testing right now with that | 14:19 |
| jaosorior | bandini: oh :/ | 14:19 |
| sshnaidm | trown, not sure we need everything there, but it works so far | 14:19 |
| bandini | jaosorior: fire away :) | 14:19 |
| jaosorior | bandini: need some help with the pacemaker bundle stuff. It's pretty hard to debug. | 14:19 |
| trown | sshnaidm: ok I will try that out thanks | 14:19 |
| bandini | jaosorior: ack. what's broken? | 14:19 |
| jaosorior | bandini: TLS updates :/ | 14:20 |
| jaosorior | bandini: in OSP12 | 14:20 |
| bandini | jaosorior: minor updates? | 14:20 |
| jaosorior | right | 14:20 |
| *** shreshtha has quit IRC | 14:21 | |
| jaosorior | and I have the feeling that upgrades as well | 14:21 |
| bandini | jaosorior: ack. do we have logs or more info on the issue? | 14:21 |
| jaosorior | bandini: I'm reproducing it locally right now | 14:21 |
| jaosorior | bandini: that's why I asked if you were around, if we could open a tmate session to debug it or something of the sort. | 14:21 |
| bandini | jaosorior: ack. I have a call n 30mins, but then I can free up some cycles to team-tag it | 14:22 |
| jaosorior | that would be nice | 14:22 |
| bandini | just ping me and I'll make some time | 14:23 |
| jaosorior | well, I'm available at any time :D except in three hours | 14:24 |
| *** spectr has quit IRC | 14:29 | |
| *** spectr has joined #tripleo | 14:31 | |
| openstackgerrit | Raoul Scarazzini proposed openstack/tripleo-quickstart-extras master: Add overcloud-ssl task to bm undercloud playbook https://review.openstack.org/517996 | 14:31 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-common master: Add ansible playbook for configuring octavia https://review.openstack.org/487450 | 14:38 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-common master: Octavia post deployment mistral wrapper https://review.openstack.org/515402 | 14:38 |
| *** dhill_ has quit IRC | 14:39 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates master: Run Octavia configuration on the overcloud https://review.openstack.org/508195 | 14:40 |
| *** ukalifon has quit IRC | 14:44 | |
| *** abregman has quit IRC | 14:46 | |
| *** dhill_ has joined #tripleo | 14:46 | |
| *** aufi has quit IRC | 14:46 | |
| *** cylopez has quit IRC | 14:47 | |
| *** amoralej|lunch is now known as amoralej | 14:48 | |
| openstackgerrit | Alfredo Moralejo proposed openstack-infra/tripleo-ci master: Install openstack-selinux with openvswitch https://review.openstack.org/518008 | 14:52 |
| *** zshi_ has quit IRC | 14:52 | |
| *** cylopez has joined #tripleo | 14:54 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Set host name explicitly for telemetry https://review.openstack.org/509199 | 14:54 |
| *** yamahata has joined #tripleo | 14:56 | |
| *** yamahata has quit IRC | 14:57 | |
| *** yamahata has joined #tripleo | 14:57 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-validations master: Add per-network bind ips to inventory https://review.openstack.org/517051 | 14:58 |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/472607 | 15:00 |
| *** yamahata has quit IRC | 15:01 | |
| *** yamahata has joined #tripleo | 15:01 | |
| jaosorior | bandini: this is one of the related fixes https://review.openstack.org/#/c/517984/ | 15:06 |
| *** bobh has joined #tripleo | 15:07 | |
| *** d0ugal has quit IRC | 15:08 | |
| *** cshastri has quit IRC | 15:10 | |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 15:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729881 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 15:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 15:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 15:10 |
| openstack | Launchpad bug 1729881 in tripleo "401 unauthorized in overcloud-validate on ovb-containers job" [Critical,Triaged] | 15:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 15:10 |
| *** moshele has quit IRC | 15:11 | |
| *** dsariel has quit IRC | 15:12 | |
| *** dmarlin has quit IRC | 15:14 | |
| jaosorior | marios: is it fine to merge this https://review.openstack.org/#/c/516969/ ? | 15:17 |
| honza | jpich: Where does 'openstack undercloud install' come from? I thought it was in openstackclient? | 15:20 |
| *** trown is now known as trown|brb | 15:20 | |
| jpich | honza: should be in python-tripleoclient I think | 15:20 |
| honza | jpich: ugh, thanks | 15:20 |
| honza | jpich: i'm dumb | 15:20 |
| jpich | You're alright ;) | 15:20 |
| honza | jpich: i got confused, and thought that those two were one and the same :( | 15:21 |
| jpich | honza: Nope! tripleoclient is an openstack client plugin, just adds a bunch of new exciting commands when installed | 15:22 |
| marios | jaosorior: sure i am not blocking it I +1 , didn't get a chance to revisit to be honest | 15:24 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-common master: Octavia post deployment mistral wrapper https://review.openstack.org/515402 | 15:24 |
| *** marrusl has joined #tripleo | 15:25 | |
| *** mrch has quit IRC | 15:25 | |
| *** brault has quit IRC | 15:26 | |
| *** marrusl has quit IRC | 15:26 | |
| *** marrusl has joined #tripleo | 15:26 | |
| *** marrusl has quit IRC | 15:27 | |
| *** trown|brb is now known as trown | 15:27 | |
| *** marrusl has joined #tripleo | 15:27 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo master: Remove Ceilometer Api puppet classes https://review.openstack.org/515502 | 15:28 |
| *** cylopez has quit IRC | 15:29 | |
| *** d0ugal has joined #tripleo | 15:34 | |
| *** d0ugal has quit IRC | 15:34 | |
| *** d0ugal has joined #tripleo | 15:34 | |
| *** pdeore has quit IRC | 15:43 | |
| *** brault has joined #tripleo | 15:44 | |
| *** brault has quit IRC | 15:49 | |
| *** fragatina has joined #tripleo | 15:50 | |
| *** fragatina has quit IRC | 15:51 | |
| *** fragatina has joined #tripleo | 15:51 | |
| *** psachin has quit IRC | 15:53 | |
| *** amodi has joined #tripleo | 15:53 | |
| *** dparkes has quit IRC | 15:55 | |
| *** dparkes has joined #tripleo | 15:56 | |
| openstackgerrit | Justin Kilpatrick proposed openstack/tripleo-quickstart-extras master: Disruption detection and some stack update roles https://review.openstack.org/497950 | 15:57 |
| openstackgerrit | Martin André proposed openstack/instack-undercloud stable/pike: Add one level of verbosity for memcached https://review.openstack.org/518020 | 15:59 |
| openstackgerrit | Dmitry Tantsur proposed openstack/instack-undercloud stable/newton: undercloud/upgrade: change ownership for ironic-dbsync.log https://review.openstack.org/517356 | 15:59 |
| *** aufi has joined #tripleo | 16:02 | |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 16:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 16:10 |
| *** brault has joined #tripleo | 16:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 16:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 16:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 16:10 |
| Slower | slagle: can you tell me what shardy is trying to say on this review? https://review.openstack.org/#/c/515967/10 | 16:13 |
| Slower | slagle: I'm guessing I need to update the commit message, but I don't want to lose all the +'s :) | 16:13 |
| jaosorior | Slower: your commit depends on a patch that's set as DNM | 16:14 |
| *** dsariel has joined #tripleo | 16:15 | |
| *** dparkes has quit IRC | 16:15 | |
| jaosorior | Slower: it won't merge unless you remove that depends-on :/ | 16:15 |
| jaosorior | Slower: if you want, udpate the commit message and I'll just add the +2 and +A | 16:15 |
| Slower | jaosorior: ok cool, thanks! | 16:18 |
| *** dsariel_ has joined #tripleo | 16:18 | |
| Slower | hopefully it doesn't rerun CI! | 16:18 |
| *** catintheroof has quit IRC | 16:19 | |
| openstackgerrit | Ian Main proposed openstack/python-tripleoclient master: Refactor undercloud deploy to use Ansible. https://review.openstack.org/515967 | 16:19 |
| slagle | Slower: right, looks like you can remove all the depends-on | 16:19 |
| Slower | doh! | 16:19 |
| *** dsariel has quit IRC | 16:19 | |
| Slower | yeah I could | 16:19 |
| Slower | the others landed though so I think they should be ok? | 16:20 |
| Slower | it's rerunning CI :/ | 16:20 |
| trown | sshnaidm: was the issue you were hitting to do with the vxlan tunnel not working? | 16:20 |
| Slower | there's a friday/saturday of coaxing it through in there.. | 16:20 |
| sshnaidm | trown, yeah, pings don't work between nodes | 16:21 |
| trown | sshnaidm: ok, I am at the same place... I can ping the private ips of the nodes, but pinging across the vxlan tunnel is failing | 16:21 |
| *** dsariel__ has joined #tripleo | 16:23 | |
| *** jprovazn has quit IRC | 16:24 | |
| jaosorior | Slower: bummer :/ yeah, seems it will run again in CI. Well, at least it'll merge now. hopefully. | 16:26 |
| *** dsariel_ has quit IRC | 16:26 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Correct the .gitreview project https://review.openstack.org/518025 | 16:27 |
| weshay | jrist, so for ocata should keystone be 'https://ip/keystone/v2/', | 16:28 |
| Slower | jaosorior: thanks! :) | 16:29 |
| jaosorior | Slower: no biggie. Let me know if you need any more reviews. | 16:29 |
| jrist | weshay: yes | 16:29 |
| jrist | weshay: is that 11? yeah | 16:29 |
| jrist | I think so | 16:29 |
| *** dsariel_ has joined #tripleo | 16:30 | |
| jaosorior | larsks: uploaded the patch to kolla for the sidecar container using the configuration you came up with. | 16:31 |
| *** dsariel_ has quit IRC | 16:31 | |
| *** dsariel_ has joined #tripleo | 16:32 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-heat-templates master: ironic upgrade: change ownership for ironic-dbsync.log https://review.openstack.org/518027 | 16:32 |
| dtantsur | can someone please tell me if this ^^^ is the right direction to go? | 16:32 |
| *** yamahata has quit IRC | 16:32 | |
| dtantsur | mwhahaha: maybe ^^^ | 16:32 |
| *** dsariel__ has quit IRC | 16:33 | |
| mwhahaha | dtantsur: yea that's the correct spot to put that stuff going forward | 16:33 |
| dtantsur | k | 16:33 |
| *** fragatina has quit IRC | 16:34 | |
| mwhahaha | dtantsur: but that won't work <pike so those go in those .sh files. i really thing the best thing to do is a post upgrade script on the package to ensure perms | 16:34 |
| mwhahaha | but that might have some additional fallout | 16:34 |
| dtantsur | >_< | 16:34 |
| dtantsur | remind me please, why couldn't we do it in puppet-ironic? | 16:34 |
| dtantsur | particularly given that its change was breaking in the end? | 16:35 |
| mwhahaha | you could do a file ensure in puppet-ironic | 16:35 |
| mwhahaha | prior to the exec | 16:35 |
| mwhahaha | dtantsur: so that's an option which would cover tripleo/puppet/packstack | 16:36 |
| dtantsur | mwhahaha: any problems with this approach? I'd go with it then | 16:36 |
| sai | mwhahaha: i dont find network-isolation.yaml in pike? Any idea where it went ? (https://github.com/openstack/tripleo-heat-templates/tree/master/environments) | 16:37 |
| mwhahaha | dtantsur: usually files & perms should be a package thing but the log files get weird | 16:37 |
| mwhahaha | dtantsur: so that's probably a cleaner targeted approach since it does also affect the udnercloud | 16:37 |
| mwhahaha | sai: it's generated, you still reference the same file going foward but it won't be in the folder | 16:37 |
| sai | mwhahaha: ughgot it | 16:38 |
| mwhahaha | sai: during the deploy you can still -e THT/environments/networking-isolation.yaml | 16:38 |
| sai | mwhahaha: makes sense, thanks! | 16:38 |
| mwhahaha | it'll resolve when the .j2 files are generated | 16:38 |
| * sai freaked out | 16:38 | |
| dtantsur | mwhahaha: is there a file ensure variant that says "only change if it exists"? similar to ansible's "file"? | 16:39 |
| mwhahaha | dtantsur: you could use exec to do that | 16:39 |
| mwhahaha | but we should probably just use the file resource | 16:39 |
| *** marios has quit IRC | 16:39 | |
| dtantsur | mwhahaha: just use ensure => "present" and let it create an empty file? | 16:40 |
| mwhahaha | yea | 16:40 |
| dtantsur | btw what does it do if I skip "ensure"? | 16:40 |
| mwhahaha | defaults to ensure present | 16:40 |
| dtantsur | okie, lemme give it a try | 16:41 |
| *** rhallisey has quit IRC | 16:41 | |
| *** itlinux has joined #tripleo | 16:42 | |
| *** dsariel__ has joined #tripleo | 16:42 | |
| *** pchavva has joined #tripleo | 16:42 | |
| *** achadha has joined #tripleo | 16:43 | |
| *** dsariel_ has quit IRC | 16:45 | |
| *** ebarrera has quit IRC | 16:45 | |
| *** jpena is now known as jpena|brb | 16:46 | |
| *** dsariel_ has joined #tripleo | 16:47 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-common master: Add Barbican worker and keystone listener images https://review.openstack.org/517076 | 16:47 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-common master: Add Barbican worker and keystone listener images https://review.openstack.org/517076 | 16:48 |
| *** dsariel__ has quit IRC | 16:49 | |
| sai | mwhahaha: also is /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml required if including docker-ha.yaml | 16:50 |
| openstackgerrit | Brad P. Crochet proposed openstack/python-tripleoclient master: Display Horizon URL at the end of a deployment https://review.openstack.org/518030 | 16:50 |
| mwhahaha | sai: i'm not sure. bandini, dciabrin do you need puppet-pacemaker with docker-ha? | 16:51 |
| bandini | sai: no it is not needed | 16:52 |
| sai | thanks mwhahaha bandini | 16:52 |
| *** dsariel__ has joined #tripleo | 16:52 | |
| sai | containers has added a whole new level of pain :( | 16:52 |
| *** dsariel_ has quit IRC | 16:54 | |
| *** newmember has quit IRC | 16:58 | |
| *** newmember has joined #tripleo | 16:58 | |
| *** radeks has joined #tripleo | 17:00 | |
| *** dsariel_ has joined #tripleo | 17:00 | |
| *** pcaruana has quit IRC | 17:01 | |
| *** adac has quit IRC | 17:01 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Allow SELinux shared context for containers state https://review.openstack.org/517383 | 17:01 |
| mandre | sai: you need to skip the generated files for the 'prepare' command | 17:01 |
| sai | mandre: ah, so no network-isolation.yaml to the prepare command | 17:01 |
| sai | makes sense | 17:01 |
| *** dsariel__ has quit IRC | 17:02 | |
| *** achadha_ has joined #tripleo | 17:02 | |
| mandre | sai: yeah, you can skip this one... the prepare command need only the environment files that enable services | 17:02 |
| *** achadha has quit IRC | 17:05 | |
| mwhahaha | sounds like should be ignoring non existant files in the prepare command | 17:06 |
| *** dsariel_ has quit IRC | 17:06 | |
| *** dsariel_ has joined #tripleo | 17:09 | |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 17:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 17:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 17:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 17:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 17:10 |
| *** dsariel__ has joined #tripleo | 17:10 | |
| *** tesseract has quit IRC | 17:10 | |
| *** yamahata has joined #tripleo | 17:11 | |
| *** salmankhan has quit IRC | 17:12 | |
| *** catintheroof has joined #tripleo | 17:12 | |
| *** pchavva has quit IRC | 17:12 | |
| *** salmankhan has joined #tripleo | 17:12 | |
| *** dsariel_ has quit IRC | 17:13 | |
| *** dsariel__ has quit IRC | 17:13 | |
| *** rmascena has joined #tripleo | 17:13 | |
| *** raildo has quit IRC | 17:14 | |
| sshnaidm | trown, I see slagle installs heat-agent in traas, not sure why..maybe it's required..? https://github.com/slagle/traas/blob/master/templates/traas.yaml#L102 | 17:17 |
| *** catintheroof has quit IRC | 17:17 | |
| *** moshele has joined #tripleo | 17:20 | |
| *** mcornea has quit IRC | 17:22 | |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Upgrade UC and OC using tripleo-upgrade role https://review.openstack.org/515643 | 17:22 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Allow SELinux shared context for containers state https://review.openstack.org/517383 | 17:23 |
| *** jpena|brb is now known as jpena | 17:24 | |
| trown | sshnaidm: hmm maybe... | 17:24 |
| *** brault has quit IRC | 17:24 | |
| trown | sshnaidm: maybe unbound | 17:25 |
| sshnaidm | trown, I installed unbound | 17:25 |
| *** hjensas has quit IRC | 17:26 | |
| *** fragatina has joined #tripleo | 17:27 | |
| *** moshele has quit IRC | 17:27 | |
| *** lucasagomes is now known as lucas-afk | 17:27 | |
| *** aufi has quit IRC | 17:27 | |
| jaosorior | mwhahaha: hey, if you have some time can you check this out https://review.openstack.org/#/c/517312/ ? | 17:27 |
| mwhahaha | k | 17:28 |
| *** yolanda has quit IRC | 17:28 | |
| trown | sshnaidm: a bit odd though, because the issue is happening before heat is involved at all ... I actually cant even figure out what is wrong with the tunnel config... it seems setup properly to me | 17:31 |
| *** yprokule has quit IRC | 17:32 | |
| *** jfrancoa has quit IRC | 17:34 | |
| trown | sshnaidm: ya no luck with heat-agent... I wonder if we should use public IPs instead of private | 17:34 |
| sshnaidm | trown, I don't have br-ctlplane though.. not sure it's created in this step | 17:34 |
| sshnaidm | trown, yeah, maybe | 17:34 |
| trown | br-ctlplane is made during undercloud-install | 17:35 |
| *** catintheroof has joined #tripleo | 17:37 | |
| openstackgerrit | Alfredo Moralejo proposed openstack/tripleo-quickstart-extras master: Install openstack-selinux with openvswitch https://review.openstack.org/518037 | 17:37 |
| openstackgerrit | Martin André proposed openstack/python-tripleoclient stable/pike: Deprecate --pull-source for container prepare command https://review.openstack.org/518040 | 17:40 |
| openstackgerrit | Martin André proposed openstack/python-tripleoclient stable/pike: Rename options to the prepare command for clarity https://review.openstack.org/518041 | 17:40 |
| openstackgerrit | Martin André proposed openstack/python-tripleoclient stable/pike: Add --environment-directory to the prepare command https://review.openstack.org/518042 | 17:40 |
| *** yamahata has quit IRC | 17:40 | |
| *** yamahata has joined #tripleo | 17:40 | |
| *** catintheroof has quit IRC | 17:41 | |
| jaosorior | mandre: added a comment ^^ https://review.openstack.org/518041 | 17:42 |
| mandre | jaosorior: if only I knew it earlier! this is a backport :-/ | 17:46 |
| dhill_ | I just tried deploying from trunk and I get this: | 17:46 |
| dhill_ | Nov 06 12:19:32 overcloud-controller-0 os-collect-config[3495]: "Error: Evaluation Error: Error while evaluating a Resource Statement, Class[Rabbitmq]:", | 17:46 |
| dhill_ | Nov 06 12:19:32 overcloud-controller-0 os-collect-config[3495]: " parameter 'port' expects an Integer value, got String", | 17:46 |
| dhill_ | Nov 06 12:19:32 overcloud-controller-0 os-collect-config[3495]: " parameter 'ssl_port' expects an Integer value, got String", | 17:46 |
| dhill_ | Nov 06 12:19:32 overcloud-controller-0 os-collect-config[3495]: " parameter 'file_limit' expects a value of type Integer[-1, default] or Enum['infinity', 'unlimited'], got String at /etc/puppet/modules/tripleo/ | 17:46 |
| dhill_ | Nov 06 12:19:32 overcloud-controller-0 os-collect-config[3495]: "Notice: hiera(): Cannot load backend module_data: cannot load such file -- hiera/backend/module_data_backend", | 17:46 |
| dhill_ | Nov 06 12:19:32 overcloud-controller-0 os-collect-config[3495]: "Notice: Scope(Class[Tripleo::Firewall::Post]): At this stage, all network traffic is blocked." | 17:46 |
| dhill_ | Nov 06 12:19:32 overcloud-controller-0 os-collect-config[3495]: ], | 17:46 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: fix tripleo-ui non-secure setup https://review.openstack.org/518043 | 17:46 |
| dhill_ | Nov 06 12:19:32 overcloud-controller-0 os-collect-config[3495]: "failed_when_result": false | 17:46 |
| dhill_ | Nov 06 12:19:32 overcloud-controller-0 os-collect-config[3495]: } | 17:46 |
| dhill_ | so rabbitmq won't start and the overcloud deployment will eventually fail | 17:46 |
| dhill_ | because the computes are never able to connect to rabbitmq | 17:46 |
| dhill_ | but the heat-stack never received an error from stack deployment on the controllers ... that's wrong | 17:47 |
| jaosorior | mandre: aawww shit | 17:47 |
| jaosorior | mandre: ok, no biggie | 17:47 |
| weshay | jrist, https://review.openstack.org/518043 I can log in now | 17:47 |
| jrist | thx | 17:47 |
| weshay | jrist, but hit Maximum call stack size exceeded | 17:47 |
| weshay | TripleO | 17:47 |
| weshay | admin | 17:47 |
| weshay | Language | 17:47 |
| weshay | 17:47 | |
| weshay | Logout | 17:47 |
| weshay | Plans | 17:47 |
| weshay | Nodes | 17:47 |
| weshay | oops | 17:47 |
| jrist | whoa | 17:47 |
| jrist | oh yeah the infinite stacks | 17:47 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: DNM: test net https://review.openstack.org/518045 | 17:48 |
| *** dmarlin has joined #tripleo | 17:48 | |
| jaosorior | dhill_: are you using the latest t-h-t? the ssl_port issue shouldn't be there anymore. | 17:48 |
| dhill_ | jaosorior, I should be | 17:49 |
| jrist | weshay: known bug | 17:49 |
| dhill_ | jaosorior, delorean-openstack-tripleo-heat-templates-d25d88fb42c444bda718e61a9cd2042c6e5a397c | 17:49 |
| *** jpich has quit IRC | 17:50 | |
| *** yamahata has quit IRC | 17:53 | |
| *** yamahata has joined #tripleo | 17:53 | |
| mandre | jaosorior: I'll keep it in mind for next time, thanks for the comment as always | 17:54 |
| *** catintheroof has joined #tripleo | 17:56 | |
| jaosorior | mandre: no biggie, I'll +2 once it passes | 17:56 |
| honza | EmilienM: how are params passed to puppet-tripleo when run from 'openstack undercloud install'? Especially interested in manifest/ui.pp vars. | 17:58 |
| *** bogdando has quit IRC | 18:01 | |
| *** derekh has quit IRC | 18:01 | |
| *** rmascena has quit IRC | 18:04 | |
| *** raildo has joined #tripleo | 18:04 | |
| *** trown is now known as trown|lunch | 18:06 | |
| honza | hmmmm, i think it's in instack-undercloud | 18:06 |
| honza | puppet-stack-config | 18:06 |
| sai | mwhahaha: the steps during overcloud deploy, step 1 through step 5, what do they denote? Does each step signify whats going on during the deploy? | 18:08 |
| mwhahaha | sai: there's a docs around this, let me see if i can dig them up | 18:08 |
| mwhahaha | sai: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/README.rst#deployment-steps | 18:09 |
| openstackgerrit | Luigi Toscano proposed openstack/tripleo-quickstart master: Fix/enable the Tempest tests for Sahara and Manila https://review.openstack.org/509554 | 18:09 |
| sai | mwhahaha: thank u | 18:10 |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 18:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 18:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 18:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 18:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 18:10 |
| *** dsneddon has joined #tripleo | 18:10 | |
| weshay | sshnaidm, you are crackin me up.. lol :) | 18:11 |
| *** dhill_ has quit IRC | 18:12 | |
| *** salmankhan has quit IRC | 18:13 | |
| sshnaidm | weshay, ?? | 18:15 |
| weshay | mwhahaha, the cloud provider vexxhost-ca is timing out on our jobs quite a bit | 18:15 |
| *** dtantsur is now known as dtantsur|afk | 18:15 | |
| weshay | 35% of the timeouts, https://bugs.launchpad.net/tripleo/+bug/1730471 <-- just for doc | 18:16 |
| openstack | Launchpad bug 1730471 in tripleo "track tripleo job timeout, exit_value=143" [High,Incomplete] | 18:16 |
| weshay | sshnaidm, your email | 18:16 |
| * mwhahaha blames mnaser | 18:16 | |
| mnaser | hi | 18:16 |
| mnaser | dang | 18:16 |
| *** rhallisey has joined #tripleo | 18:16 | |
| mnaser | let me see what i can do about that :\ | 18:16 |
| *** jpena is now known as jpena|off | 18:17 | |
| bnemec | weshay: Oh man, I completely missed that typo. :-) | 18:17 |
| *** colonwq has quit IRC | 18:17 | |
| weshay | lol | 18:17 |
| *** weshay changes topic to "CI status: Yellow, jobs timing out http://ow.ly/MTTI30gomCe | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest/" | 18:21 | |
| *** itlinux has quit IRC | 18:24 | |
| *** ckyriakidou has quit IRC | 18:25 | |
| *** moshele has joined #tripleo | 18:25 | |
| *** colonwq has joined #tripleo | 18:26 | |
| mwhahaha | looke like we're missing teh legacy-puppet-syntax-3 job on stable/newton | 18:26 |
| mwhahaha | legacy-puppet-syntax-3-centos-7 legacy-puppet-syntax-3-centos-7 : ERROR Unable to find playbook /var/lib/zuul/builds/87b687b9031544a2a104f242be3e0a5a/trusted/project_2/git.openstack.org/openstack-infra/openstack-zuul-jobs/playbooks/legacy/puppet-syntax-3-centos-7/run.yaml | 18:26 |
| * mwhahaha files an LP | 18:26 | |
| *** dhill_ has joined #tripleo | 18:28 | |
| mwhahaha | weshay, adarazs|rover, panda|ruck: fyi https://bugs.launchpad.net/tripleo/+bug/1730477 | 18:28 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 18:28 |
| Tengu | hello! | 18:35 |
| Tengu | what's missing on that review for the merge in master? https://review.openstack.org/#/c/517903/ | 18:35 |
| mwhahaha | Tengu: time | 18:40 |
| mwhahaha | it's currently in the gate | 18:40 |
| mwhahaha | http://zuulv3.openstack.org/ | 18:40 |
| Tengu | mwhahaha: ah, cool. | 18:41 |
| Tengu | so nothing more to do. | 18:41 |
| mwhahaha | correct | 18:41 |
| Tengu | would be nice to get the feedback on the review page. but I guess this isn't easy. | 18:41 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-pacemaker master: Use --config when we manipulate the CIB https://review.openstack.org/518093 | 18:41 |
| mwhahaha | Tengu: usually it'll clear the +1 and update the review to show it. but lately that's been delayed or goes missing | 18:42 |
| Tengu | ok, no problem | 18:42 |
| Tengu | just wanted to ensure it wasn't lost ;). | 18:42 |
| Tengu | btw… still strugling with another one :( | 18:43 |
| Tengu | mwhahaha: can you confirm me the error at the end of this log is NOT the cause of the "failed" test? http://logs.openstack.org/74/513274/13/check/legacy-tripleo-ci-centos-7-containers-multinode/c55ca73/logs/undercloud/home/zuul/vxlan_networking.sh.log.txt.gz | 18:44 |
| Tengu | if it is, there's an issue in the CI :/. | 18:45 |
| mwhahaha | Tengu: no that's not related to that job fail | 18:46 |
| mwhahaha | Tengu: http://logs.openstack.org/74/513274/13/check/legacy-tripleo-ci-centos-7-containers-multinode/c55ca73/job-output.txt.gz#_2017-11-04_14_57_46_765919 | 18:46 |
| Tengu | hmm. | 18:46 |
| mwhahaha | it failed because tempest | 18:46 |
| openstackgerrit | Toure Dunnon proposed openstack/python-tripleoclient master: Tripleoclient unified queue name. https://review.openstack.org/509902 | 18:46 |
| mwhahaha | i think there's an open issue for that around cinder packages/containers being out of sync | 18:46 |
| Tengu | mwhahaha: hmm ok. so not related to my code change then. | 18:46 |
| mwhahaha | probably not but let me look further | 18:47 |
| Tengu | thank you. I'm not realy efficient with that CI debugging, I Must confess. | 18:47 |
| mwhahaha | http://logs.openstack.org/74/513274/13/check/legacy-tripleo-ci-centos-7-containers-multinode/c55ca73/logs/subnode-2/var/log/cinder/volume.log.txt.gz#_2017-11-04_14_54_19_089 | 18:47 |
| mwhahaha | which is https://bugs.launchpad.net/tripleo/+bug/1730111 | 18:48 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 18:48 |
| Tengu | ah, ok. that's one of the issue I opened I thinkg. | 18:48 |
| Tengu | or the other one | 18:48 |
| mwhahaha | yea one of those two | 18:48 |
| Tengu | ook. | 18:48 |
| Tengu | so I'll stop stressing the CI with that one and will just rebase the code when needed. | 18:48 |
| mwhahaha | yea | 18:48 |
| Tengu | that's the last review I have for now ;). | 18:48 |
| *** yamahata has quit IRC | 18:49 | |
| *** itlinux has joined #tripleo | 18:50 | |
| *** yolanda has joined #tripleo | 18:54 | |
| *** trown|lunch is now known as trown | 18:56 | |
| *** amoralej is now known as amoralej|off | 18:58 | |
| *** thrash is now known as thrash|biab | 18:59 | |
| *** ecerquei_ has joined #tripleo | 18:59 | |
| *** ecerquei has quit IRC | 19:01 | |
| *** fzdarsky has joined #tripleo | 19:07 | |
| *** brault has joined #tripleo | 19:07 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 19:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 19:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 19:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 19:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 19:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 19:10 |
| *** brault has quit IRC | 19:12 | |
| *** fzdarsky has quit IRC | 19:13 | |
| tbarron | Do we have recipes/prio-art/guidance for building overcloud images where $component is based on git master (+ outstanding review) rather than an rpm? | 19:15 |
| tbarron | To be clear, $component is not THT or puppet, i.e. an openstack non-deployment tool component. | 19:16 |
| tbarron | I used to be able to git clone the repo for $component on the undercloud and virt-customize the overcloud-image. | 19:17 |
| tbarron | kinduva a pain but do-able | 19:17 |
| tbarron | is there an equiv with containerized images? | 19:17 |
| *** gfidente is now known as gfidente|afk | 19:18 | |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Remove hardcoded docker image names https://review.openstack.org/518094 | 19:23 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Remove hardcoded skydive container image https://review.openstack.org/518095 | 19:23 |
| *** mcornea has joined #tripleo | 19:38 | |
| *** adac has joined #tripleo | 19:39 | |
| Slower | tbarron: you can build your own containers.. | 19:40 |
| Slower | tbarron: if that's what you mean | 19:40 |
| *** rwsu has quit IRC | 19:40 | |
| *** gbarros has joined #tripleo | 19:41 | |
| Slower | tbarron: openstack overcloud container image build | 19:41 |
| Slower | tbarron: also see the overrides in tripleo-common | 19:41 |
| tbarron | Slower: I know the image build part, but maybe not the overrides, will look there. | 19:41 |
| *** tzumainn has joined #tripleo | 19:42 | |
| tbarron | Slower: I'm looking to build from the normal rpms *except* for the component I'm working on, which I want built from my local git repo. | 19:42 |
| Slower | tbarron: you could change components in here https://github.com/openstack/tripleo-common/blob/master/container-images/tripleo_kolla_template_overrides.j2 | 19:42 |
| Slower | tbarron: you could also just use kolla build | 19:42 |
| openstackgerrit | Or Idgar proposed openstack/tripleo-common master: Add ansible playbook for configuring octavia https://review.openstack.org/487450 | 19:43 |
| Slower | to build the specific container | 19:43 |
| *** brault has joined #tripleo | 19:43 | |
| Slower | tbarron: with kolla build you could specify a source build.. | 19:44 |
| *** brault has quit IRC | 19:44 | |
| Slower | or just jam whatever into that override.. your call | 19:45 |
| *** brault has joined #tripleo | 19:45 | |
| Slower | or add a new layer to the container | 19:45 |
| Slower | lots of options | 19:45 |
| tbarron | Slower: thanks for the help, I'll poke around. I think a new layer for just the source for my component on top of the regularly deployed overcloud container image is what I want. | 19:46 |
| tbarron | Slower: that would presumably be faster than building everything from source. | 19:46 |
| Slower | tbarron: if you're in a rush and it's a one-off you could just run the container by hand with bin/bash, modify it to your liking, then commit it and push it to a local repo | 19:46 |
| Slower | tbarron: you have to watch though because the commit will update the 'CMD' setting which we depend on | 19:47 |
| tbarron | Slower: that's a good point, kinda like the old virt-customize approach. | 19:47 |
| Slower | tbarron: this script yum updates a container and then commits/pushes it back | 19:47 |
| Slower | tbarron: https://github.com/imain/container-check/blob/master/container-check#L83 | 19:47 |
| Slower | tbarron: those are all the commands you need to add a new layer and have it work | 19:48 |
| *** chlong has joined #tripleo | 19:48 | |
| tbarron | Slower: awesome! ty. | 19:48 |
| *** d0ugal has quit IRC | 19:52 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-common master: Add ansible playbook for configuring octavia https://review.openstack.org/487450 | 19:52 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-common master: Octavia post deployment mistral wrapper https://review.openstack.org/515402 | 19:52 |
| *** iranzo has quit IRC | 19:56 | |
| *** chlong has quit IRC | 19:57 | |
| *** pcaruana has joined #tripleo | 19:57 | |
| *** etingof has quit IRC | 19:59 | |
| panda|ruck | Slower: do you know if there's the container meetup today ? | 20:02 |
| *** pcaruana has quit IRC | 20:03 | |
| openstackgerrit | Jon Schlueter proposed openstack/python-tripleoclient master: Remove defaults from namespace and prefix for container images https://review.openstack.org/518101 | 20:06 |
| *** chlong has joined #tripleo | 20:08 | |
| *** rfolco has quit IRC | 20:10 | |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 20:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 20:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 20:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 20:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 20:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 20:10 |
| *** gbarros has quit IRC | 20:10 | |
| *** ecerquei_ has quit IRC | 20:10 | |
| *** ecerquei has joined #tripleo | 20:12 | |
| openstackgerrit | Jon Schlueter proposed openstack/python-tripleoclient master: Use common defaults for image prepare CLI https://review.openstack.org/516139 | 20:12 |
| *** gfidente[m] has quit IRC | 20:15 | |
| *** d0ugal has joined #tripleo | 20:17 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Add /var/lib/mistral for log collection https://review.openstack.org/518102 | 20:17 |
| openstackgerrit | James Slagle proposed openstack/tripleo-quickstart master: fs010: use --config-download from tripleoclient https://review.openstack.org/514736 | 20:18 |
| *** atoth has quit IRC | 20:22 | |
| *** etingof has joined #tripleo | 20:25 | |
| *** morazi has joined #tripleo | 20:26 | |
| trown | sshnaidm: only thing I notice from your DNM merge patch is eth0 is /32 but your multinode template makes a /24 | 20:27 |
| trown | sshnaidm: but that is just from DHCP... | 20:28 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-common master: Octavia post deployment mistral wrapper https://review.openstack.org/515402 | 20:29 |
| *** thrash|biab is now known as thrash | 20:30 | |
| *** moshele has quit IRC | 20:35 | |
| *** akrivoka has quit IRC | 20:35 | |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-repos master: Update ceph and opstools repos https://review.openstack.org/517728 | 20:38 |
| *** itlinux has quit IRC | 20:39 | |
| openstackgerrit | Merged openstack/puppet-tripleo stable/pike: Release 7.4.4 (pike) https://review.openstack.org/517795 | 20:41 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-pacemaker master: Use --config when we manipulate the CIB https://review.openstack.org/518093 | 20:41 |
| *** brault has quit IRC | 20:44 | |
| openstackgerrit | Merged openstack/puppet-tripleo stable/ocata: Release 6.5.5 (ocata) https://review.openstack.org/517794 | 20:44 |
| *** brault has joined #tripleo | 20:45 | |
| *** liverpooler has quit IRC | 20:45 | |
| *** moshele has joined #tripleo | 20:46 | |
| *** brault has quit IRC | 20:47 | |
| *** toure is now known as toure_biab | 20:47 | |
| *** brault has joined #tripleo | 20:47 | |
| *** rwsu has joined #tripleo | 20:48 | |
| *** brault has quit IRC | 20:49 | |
| *** brault has joined #tripleo | 20:50 | |
| *** mcornea has quit IRC | 20:54 | |
| *** brault has quit IRC | 20:54 | |
| *** achadha has joined #tripleo | 20:56 | |
| *** achadha_ has quit IRC | 20:56 | |
| jrist | weshay: trying your patch now | 20:57 |
| jrist | weshay: or trying to try it. | 21:00 |
| jrist | weshay: how do I tell quickstart to use extras from repo? | 21:00 |
| *** achadha has quit IRC | 21:00 | |
| weshay | jrist, check out the change locally | 21:04 |
| jrist | and then... :) | 21:04 |
| weshay | then just change quickstart-extras-requirements.txt | 21:04 |
| weshay | it's a uri | 21:04 |
| weshay | so you can say | 21:04 |
| jrist | ah cool | 21:04 |
| weshay | file:///home/jrister/git/tripleo-quickstart-extras/#eggfoo | 21:04 |
| weshay | jrist, use --clean | 21:04 |
| weshay | and you got it | 21:04 |
| jrist | always | 21:04 |
| weshay | so crisp and so clean clean | 21:05 |
| jrist | eggfoo! | 21:05 |
| jrist | weshay: haha I'm glad I'm not the only one that does that | 21:05 |
| weshay | heh | 21:07 |
| jrist | thanks, 90s | 21:07 |
| jrist | er... 2000 | 21:08 |
| *** dsariel__ has joined #tripleo | 21:08 | |
| *** moshele has quit IRC | 21:09 | |
| jrist | weshay: got a meeting but I'll try all this in a minute | 21:09 |
| *** morazi has quit IRC | 21:10 | |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 21:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 21:10 |
| *** benone has quit IRC | 21:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 21:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 21:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 21:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 21:10 |
| *** atoth has joined #tripleo | 21:10 | |
| *** yamahata has joined #tripleo | 21:12 | |
| *** amodi has quit IRC | 21:12 | |
| *** moshele has joined #tripleo | 21:13 | |
| *** moshele has quit IRC | 21:16 | |
| *** aputtur_ has quit IRC | 21:26 | |
| *** radeks has quit IRC | 21:28 | |
| *** chlong has quit IRC | 21:29 | |
| jrist | weshay: does that requirements file have comments? # ? | 21:30 |
| weshay | jrist, no but that is a fine idea | 21:30 |
| jrist | it does | 21:30 |
| jrist | actually | 21:30 |
| mwhahaha | yea you can add comments to the requirements files | 21:32 |
| jrist | weshay: hmm | 21:33 |
| jrist | Collecting tripleo-quickstart-extras from git+https://git.openstack.org/openstack/tripleo-quickstart-extras/#egg=tripleo-quickstart-extras (from -r internal-requirements-new.txt (line 3)) | 21:33 |
| jrist | I'm thinking it didn't work | 21:33 |
| jrist | weshay: this is what it looks like: | 21:33 |
| jrist | file:///home/jrist/workspace/tripleo-quickstart-extras/#egg=tripleo-quickstart-extras | 21:34 |
| weshay | jrist, why don't you think it worked? | 21:34 |
| weshay | check ~/.quickstart/usr/local/share/ansible/roles/undercloud-deploy/tasks/tripleo-ui.yml | 21:35 |
| weshay | and see if the right code is ther | 21:35 |
| *** ansmith has quit IRC | 21:35 | |
| *** brault has joined #tripleo | 21:37 | |
| *** dprince has joined #tripleo | 21:40 | |
| *** ccamacho has quit IRC | 21:40 | |
| *** brault has quit IRC | 21:42 | |
| *** noslzzp has joined #tripleo | 21:43 | |
| *** benone has joined #tripleo | 21:44 | |
| jrist | weshay: because of this line Collecting tripleo-quickstart-extras from git+https://git.openstack.org/openstack/tripleo-quickstart-extras/#egg=tripleo-quickstart-extras (from -r internal-requirements-new.txt (line 3)) | 21:46 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: tempest plugins only need to be installed on convert for newton https://review.openstack.org/518114 | 21:46 |
| jrist | that's the default | 21:46 |
| weshay | jrist, oh.. you are testing w/ rhos | 21:47 |
| weshay | jrist, then ya.. update that requirements file man | 21:47 |
| jrist | oh duh | 21:47 |
| jrist | thanks for pointing that out weshay | 21:47 |
| weshay | LOLZ | 21:47 |
| weshay | ;) | 21:47 |
| jrist | weshay: worked. thanks | 21:48 |
| *** catintheroof has quit IRC | 21:51 | |
| *** trown is now known as trown|outtypewww | 21:54 | |
| *** adac has quit IRC | 21:55 | |
| *** moshele has joined #tripleo | 21:56 | |
| *** dprince has quit IRC | 21:56 | |
| *** gfidente[m] has joined #tripleo | 21:58 | |
| *** ecerquei has quit IRC | 21:58 | |
| *** dprince has joined #tripleo | 21:59 | |
| *** dprince has quit IRC | 21:59 | |
| *** gbarros has joined #tripleo | 21:59 | |
| openstackgerrit | Keith Schincke proposed openstack/puppet-tripleo master: Set ACLs on ceph client keyrings https://review.openstack.org/517347 | 21:59 |
| openstackgerrit | Keith Schincke proposed openstack/tripleo-heat-templates master: Set file ACLs for Ceph keyrings for non-containerized deployment https://review.openstack.org/509020 | 22:00 |
| *** achadha has joined #tripleo | 22:02 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Add /var/lib/mistral for log collection https://review.openstack.org/518102 | 22:02 |
| *** achadha_ has joined #tripleo | 22:03 | |
| *** achadha has quit IRC | 22:06 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 22:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 22:10 |
| *** colonwq has quit IRC | 22:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 22:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 22:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 22:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 22:10 |
| *** abishop has quit IRC | 22:11 | |
| *** colonwq has joined #tripleo | 22:12 | |
| *** jkilpatr has quit IRC | 22:13 | |
| *** radeks has joined #tripleo | 22:14 | |
| *** moshele has quit IRC | 22:15 | |
| *** owalsh has quit IRC | 22:16 | |
| *** owalsh has joined #tripleo | 22:16 | |
| *** akrivoka has joined #tripleo | 22:27 | |
| *** dparkes has joined #tripleo | 22:28 | |
| *** pradk has quit IRC | 22:31 | |
| *** ansmith has joined #tripleo | 22:31 | |
| *** chem has quit IRC | 22:33 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: create subunit test results for the undercloud sanity check https://review.openstack.org/517657 | 22:38 |
| *** ffiore has joined #tripleo | 22:39 | |
| *** radeks has quit IRC | 22:41 | |
| *** noslzzp has quit IRC | 22:41 | |
| openstackgerrit | wes hayutin proposed openstack/instack-undercloud master: DNM, TESTING ONLY https://review.openstack.org/518118 | 22:42 |
| *** dpeacock_ has joined #tripleo | 22:47 | |
| *** dpeacock has quit IRC | 22:47 | |
| *** dpeacock_ is now known as dpeacock | 22:47 | |
| *** gbarros has quit IRC | 22:48 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-common master: Octavia post deployment mistral wrapper https://review.openstack.org/515402 | 22:51 |
| *** jmelvin has quit IRC | 22:55 | |
| *** tzumainn has quit IRC | 22:56 | |
| *** radeks has joined #tripleo | 22:56 | |
| *** rwsu has quit IRC | 22:56 | |
| *** gfidente[m] has quit IRC | 23:00 | |
| *** salmankhan has joined #tripleo | 23:09 | |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-repos master: Update ceph and opstools repos https://review.openstack.org/517728 | 23:09 |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 23:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 23:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 23:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 23:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 23:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 23:10 |
| *** bobh has quit IRC | 23:10 | |
| *** gfidente[m] has joined #tripleo | 23:12 | |
| *** akrivoka has quit IRC | 23:14 | |
| *** salmankhan has quit IRC | 23:14 | |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart-extras master: Collect logs: add ansible steps log file to logstash index https://review.openstack.org/518125 | 23:27 |
| *** nguyentrihai has quit IRC | 23:33 | |
| *** nguyentrihai has joined #tripleo | 23:33 | |
| *** radeks has quit IRC | 23:33 | |
| *** gbarros has joined #tripleo | 23:34 | |
| *** ianw has quit IRC | 23:35 | |
| weshay | bah timeout https://review.openstack.org/#/c/467074/ | 23:35 |
| weshay | mwhahaha, lots o timeouts man | 23:35 |
| *** gfidente[m] has quit IRC | 23:35 | |
| *** tosky has quit IRC | 23:36 | |
| mwhahaha | weshay: probably means something is broken | 23:37 |
| *** ianw has joined #tripleo | 23:37 | |
| weshay | mwhahaha, well said | 23:37 |
| mwhahaha | I shall be refered to by my new name, general obvious | 23:38 |
| mwhahaha | you can be my captain | 23:38 |
| weshay | lol | 23:38 |
| weshay | indeed | 23:38 |
| weshay | gate just reset on http://logs.openstack.org/74/467074/7/gate/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq/451d2a8/ | 23:39 |
| *** bandini has quit IRC | 23:39 | |
| weshay | panda|ruck, ^ fyi | 23:39 |
| weshay | at least openstack health rss feeds are working well | 23:40 |
| *** bandini has joined #tripleo | 23:40 | |
| *** achadha has joined #tripleo | 23:41 | |
| mwhahaha | so it's timingout while running tempest but that's probably because some aspect of the process is running longer | 23:41 |
| mwhahaha | as we only left ~8 mins to run tempest http://logs.openstack.org/74/467074/7/gate/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq/451d2a8/job-output.txt.gz#_2017-11-06_23_20_03_073188 | 23:41 |
| mwhahaha | mirror problems or something? | 23:41 |
| *** gfidente[m] has joined #tripleo | 23:42 | |
| *** gfidente[m] has quit IRC | 23:43 | |
| mwhahaha | configuring ironic pxe_ssh takes 27 mins? | 23:44 |
| *** achadha_ has quit IRC | 23:44 | |
| mwhahaha | or am i reading the logs wrong | 23:44 |
| mwhahaha | weshay: http://logs.openstack.org/74/467074/7/gate/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq/451d2a8/job-output.txt.gz#_2017-11-06_21_43_05_899849 | 23:44 |
| weshay | mwhahaha, volumeboot pattern failed again | 23:45 |
| weshay | I think is the real cause | 23:45 |
| mwhahaha | meh | 23:45 |
| *** achadha has quit IRC | 23:45 | |
| weshay | ssh timed out | 23:45 |
| weshay | http://logs.openstack.org/74/467074/7/gate/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq/451d2a8/logs/undercloud/home/zuul/tempest_output.log.txt.gz#_2017-11-06_23_29_16 | 23:45 |
| *** gfidente|afk has quit IRC | 23:46 | |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-repos master: Update ceph and opstools repos https://review.openstack.org/517728 | 23:46 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-repos master: WIP Implement Proxy Support https://review.openstack.org/518127 | 23:46 |
| weshay | hrm.. is password: none correct there? | 23:47 |
| weshay | arxcruz, ^ | 23:47 |
| mwhahaha | yea | 23:47 |
| weshay | k.. because I see 2017-11-06 23:29:16 | 2017-11-06 23:29:02.750 25698 ERROR tempest.lib.common.ssh AuthenticationException: Authentication failed. | 23:47 |
| mwhahaha | i think it's always been password none, but maybe not | 23:47 |
| *** dmarlin has quit IRC | 23:48 | |
| mwhahaha | it uses pub key | 23:49 |
| mwhahaha | 2017-11-06 23:23:48,040 25698 INFO [tempest.lib.common.ssh] Creating ssh connection to '192.168.24.113:22' as 'cirros' with public key authentication | 23:50 |
| *** fzdarsky has joined #tripleo | 23:50 | |
| mwhahaha | though auth failed | 23:50 |
| mwhahaha | 2017-11-06 23:28:37,957 25698 WARNING [tempest.lib.common.ssh] Failed to establish authenticated ssh connection to cirros@192.168.24.113 (Authentication failed.). Number attempts: 23. Retry after 24 seconds. | 23:50 |
| mwhahaha | hrm | 23:51 |
| * mwhahaha wanders off and will look later | 23:52 | |
| *** gfidente[m] has joined #tripleo | 23:54 | |
| arxcruz | mwhahaha: weshay is these runs have the mtu fix ? | 23:54 |
| weshay | arxcruz, they should that merged while ago I think | 23:55 |
| weshay | not sure of it though | 23:55 |
| *** raildo has quit IRC | 23:55 | |
| weshay | Slower, we want to update {{ undercloud_rpm_dependencies }} | 23:56 |
| weshay | in the config | 23:56 |
| weshay | Slower, it's in the release configs | 23:57 |
| weshay | in tq | 23:58 |
| weshay | Slower, undercloud_rpm_dependencies: >- | 23:58 |
| weshay | python-tripleoclient | 23:58 |
| weshay | ceph-ansible | 23:58 |
| weshay | https://review.openstack.org/#/c/517445/3/config/release/tripleo-ci/master.yml | 23:58 |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart-extras master: DNM, update the undercloud install for containers https://review.openstack.org/517444 | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!