| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK https://review.openstack.org/430277 | 00:00 |
|---|---|---|
| *** jmelvin has quit IRC | 00:01 | |
| *** Goneri has joined #tripleo | 00:11 | |
| *** pmannidi has joined #tripleo | 00:13 | |
| *** Goneri has quit IRC | 00:23 | |
| *** dsneddon_afk is now known as dsneddon | 00:25 | |
| *** jkilpatr has quit IRC | 00:31 | |
| *** jkilpatr has joined #tripleo | 00:32 | |
| *** limao has joined #tripleo | 00:51 | |
| *** leifmadsen has quit IRC | 00:55 | |
| *** pmannidi has quit IRC | 00:57 | |
| *** leifmadsen has joined #tripleo | 00:57 | |
| *** mrunge has quit IRC | 01:06 | |
| *** kjw3 has quit IRC | 01:11 | |
| *** panda|pto has quit IRC | 01:12 | |
| *** pmannidi has joined #tripleo | 01:13 | |
| *** jkilpatr has quit IRC | 01:13 | |
| *** panda|pto has joined #tripleo | 01:15 | |
| *** abishop_ has quit IRC | 01:18 | |
| *** pmannidi has quit IRC | 01:21 | |
| *** jkilpatr has joined #tripleo | 01:25 | |
| *** limao has quit IRC | 01:36 | |
| *** limao has joined #tripleo | 01:36 | |
| *** pmannidi has joined #tripleo | 01:37 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add migration SSH tunneling support https://review.openstack.org/450842 | 01:39 |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Ignore libvirt start if running chrooted https://review.openstack.org/447000 | 01:39 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Updated from global requirements https://review.openstack.org/456010 | 01:40 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Use infra's docker registry mirror for featureset jobs https://review.openstack.org/455221 | 01:40 |
| *** japestinho has joined #tripleo | 01:40 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Updated from global requirements https://review.openstack.org/455909 | 01:52 |
| *** mrunge has joined #tripleo | 01:53 | |
| openstackgerrit | Rabi Mishra proposed openstack/instack-undercloud master: Make 'reauthentication_auth_method=trusts' the default https://review.openstack.org/457837 | 02:04 |
| *** fragatin_ has joined #tripleo | 02:11 | |
| *** fragatina has quit IRC | 02:14 | |
| *** fragatin_ has quit IRC | 02:15 | |
| *** fragatina has joined #tripleo | 02:17 | |
| *** yamahata has quit IRC | 02:17 | |
| *** fragatina has quit IRC | 02:21 | |
| *** darc_ has quit IRC | 02:23 | |
| *** darc_ has joined #tripleo | 02:23 | |
| *** darc_ has quit IRC | 02:25 | |
| *** darc_ has joined #tripleo | 02:25 | |
| *** yolanda has quit IRC | 02:26 | |
| *** yolanda has joined #tripleo | 02:26 | |
| *** lvlds has joined #tripleo | 02:32 | |
| diana_clarke | lvlds: Welcome! | 02:34 |
| lvlds | Hi, thanks! | 02:35 |
| lvlds | Maybe tomorrow the script will finish installing everything. | 02:37 |
| diana_clarke | lvlds: Heh, and I'll let you know if my quickstart exists successfully this time, but for now I'm off to bed. Cheers ;) | 02:38 |
| lvlds | Today I was able to log in the Horizon dashboard, but I did not understand what TripleO UI does, could make it run | 02:38 |
| lvlds | I am on this TripleO Deep Dive August 11, 2016 - Networking - https://www.youtube.com/watch?v=zYNq2uT9pfM | 02:39 |
| lvlds | Rest well. Cheers ;) | 02:40 |
| diana_clarke | lvlds: Thanks! I'll watch that tomorrow. I really like Dan Prince's teaching style (hands on, command line, etc). | 02:42 |
| *** tzumainn has quit IRC | 02:59 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/newton: Automatically backup and restore Swift rings from the undercloud https://review.openstack.org/435285 | 03:11 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Create bigswitch agent profile https://review.openstack.org/448234 | 03:11 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Use tripleo profile for bigswitch agent https://review.openstack.org/448236 | 03:11 |
| *** darc_ has quit IRC | 03:13 | |
| *** links has joined #tripleo | 03:19 | |
| *** jerrygb has joined #tripleo | 03:21 | |
| *** psahoo has joined #tripleo | 03:37 | |
| *** lifeless has quit IRC | 03:38 | |
| *** gkadam has joined #tripleo | 03:43 | |
| *** gkadam is now known as Guest79394 | 03:43 | |
| *** Guest79394 has quit IRC | 03:44 | |
| *** gkadam_ has joined #tripleo | 03:44 | |
| *** gkadam_ is now known as gkadam | 03:45 | |
| *** lifeless has joined #tripleo | 03:46 | |
| *** ykarel has joined #tripleo | 03:54 | |
| *** chlong has quit IRC | 03:58 | |
| *** limao has quit IRC | 04:11 | |
| *** limao has joined #tripleo | 04:11 | |
| *** limao has quit IRC | 04:16 | |
| *** kjw3 has joined #tripleo | 04:17 | |
| *** arxcruz has quit IRC | 04:18 | |
| *** arxcruz has joined #tripleo | 04:18 | |
| *** fragatina has joined #tripleo | 04:26 | |
| *** fragatina has quit IRC | 04:29 | |
| *** fragatina has joined #tripleo | 04:30 | |
| *** links has quit IRC | 04:36 | |
| *** sudipto has joined #tripleo | 04:39 | |
| *** skramaja has joined #tripleo | 04:46 | |
| *** limao has joined #tripleo | 04:47 | |
| *** links has joined #tripleo | 04:52 | |
| *** ratailor has joined #tripleo | 04:53 | |
| *** bkopilov has joined #tripleo | 04:54 | |
| *** bkopilov has quit IRC | 04:59 | |
| *** bkopilov has joined #tripleo | 05:00 | |
| *** mdnadeem has joined #tripleo | 05:00 | |
| *** bkopilov has quit IRC | 05:02 | |
| *** bkopilov has joined #tripleo | 05:04 | |
| *** udesale has joined #tripleo | 05:04 | |
| *** padkrish has joined #tripleo | 05:07 | |
| *** padkrish has quit IRC | 05:07 | |
| openstackgerrit | Rabi Mishra proposed openstack/instack-undercloud master: Make 'reauthentication_auth_method=trusts' the default https://review.openstack.org/457837 | 05:13 |
| *** yamahata has joined #tripleo | 05:15 | |
| *** eggmaster has quit IRC | 05:16 | |
| *** janki has joined #tripleo | 05:22 | |
| *** links has quit IRC | 05:25 | |
| *** jerrygb has quit IRC | 05:25 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Use underscore for Aodh and Gnocchi's container names https://review.openstack.org/456465 | 05:26 |
| *** iranzo has joined #tripleo | 05:34 | |
| *** iranzo has joined #tripleo | 05:34 | |
| *** masco has joined #tripleo | 05:35 | |
| *** nmathew has joined #tripleo | 05:36 | |
| *** links has joined #tripleo | 05:37 | |
| *** cmyster has quit IRC | 05:40 | |
| *** aufi has joined #tripleo | 05:43 | |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates master: WIP: Derive Parameter environment via plan-environment https://review.openstack.org/457874 | 05:45 |
| *** prateek has joined #tripleo | 05:48 | |
| *** karthiks has joined #tripleo | 05:52 | |
| skramaja | jtomasek: ^ i am working with Steven to comeup with a format, which will be useful to provide parameters for workflow without adding the parameters to heat environment. | 05:54 |
| skramaja | jtomasek: i am wondering how it will be used from UI presective, need your inputs in this regard. note this is not the final apporach. i am doing a PoC to understand how it will look a like, format may change after discussion. | 05:55 |
| *** rcernin has joined #tripleo | 06:08 | |
| ratailor | Could anybody pls tell me, how can I test changes in stable branch. I have deployed mitaka and now I want to apply some changes and test whether those changes are working or not. | 06:11 |
| *** pgadiya has joined #tripleo | 06:14 | |
| *** milan has joined #tripleo | 06:22 | |
| *** florianf has joined #tripleo | 06:23 | |
| *** lmiccini has joined #tripleo | 06:23 | |
| *** numans has quit IRC | 06:25 | |
| *** flepied has quit IRC | 06:29 | |
| *** janki has quit IRC | 06:30 | |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-specs master: Deriving TripleO Parameters https://review.openstack.org/423304 | 06:30 |
| *** pcaruana has joined #tripleo | 06:33 | |
| *** jprovazn has joined #tripleo | 06:35 | |
| *** agurenko has joined #tripleo | 06:35 | |
| *** cmyster has joined #tripleo | 06:36 | |
| *** cmyster has quit IRC | 06:36 | |
| *** cmyster has joined #tripleo | 06:36 | |
| *** saibarspeis has joined #tripleo | 06:38 | |
| *** paramite has joined #tripleo | 06:45 | |
| *** numans has joined #tripleo | 06:47 | |
| *** nmathew- has joined #tripleo | 06:47 | |
| *** dsariel has quit IRC | 06:48 | |
| *** janki has joined #tripleo | 06:49 | |
| *** nmathew has quit IRC | 06:51 | |
| *** jaosorior has joined #tripleo | 06:53 | |
| *** garyk has joined #tripleo | 06:54 | |
| *** numans has quit IRC | 06:56 | |
| *** gbarros has joined #tripleo | 06:58 | |
| *** numans has joined #tripleo | 07:00 | |
| *** tesseract has joined #tripleo | 07:01 | |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-specs master: Deriving TripleO Parameters https://review.openstack.org/423304 | 07:02 |
| *** yamahata has quit IRC | 07:03 | |
| *** flepied has joined #tripleo | 07:05 | |
| *** dparkes has joined #tripleo | 07:05 | |
| *** athomas has joined #tripleo | 07:09 | |
| *** limao has quit IRC | 07:10 | |
| *** limao has joined #tripleo | 07:12 | |
| *** cylopez has joined #tripleo | 07:14 | |
| *** shardy has joined #tripleo | 07:14 | |
| *** ffiore has joined #tripleo | 07:16 | |
| *** ffiore has quit IRC | 07:17 | |
| *** ffiore has joined #tripleo | 07:17 | |
| *** jlinkes has joined #tripleo | 07:18 | |
| jtomasek | skramaja: thanks, I'll have a look | 07:20 |
| skramaja | shardy: https://review.openstack.org/#/c/457874/ - i tried incorporate workflow parameters in plan-environment, i feel this is a good start, can you take a look and let me know if this is ok? | 07:28 |
| *** dbecker has joined #tripleo | 07:29 | |
| skramaja | shardy: later the 'data' section in the plan-environment can be converted to user-environment, but it needs some additional work parsing in the tripleoclient. we can add that support later and easily migrate to it with the current sample. | 07:30 |
| shardy | skramaja: thanks, I added some ideas | 07:34 |
| *** jpena|off is now known as jpena | 07:37 | |
| skramaja | shardy: this is one sample of the env file which is needed -https://review.openstack.org/#/c/457874/1/environments/derive-params/derive_parameters.yaml, but the list of parameters may increase. adding all directly in plan-environment will bloat the file. thats the reason for isolating to a seprate env file. | 07:37 |
| shardy | skramaja: sure, but we could just use a j2 include or something perhaps? | 07:38 |
| shardy | or allow multiple plan-environment files that can be merged? | 07:38 |
| skramaja | shardy: nice. | 07:38 |
| shardy | skramaja: just throwing some ideas around, I think the direction is fine, I'm just not sure the interface is exactly optimal in plan-environment yet | 07:39 |
| skramaja | shardy: i like both, i will start with 2nd one. it is more easy to understand. | 07:39 |
| skramaja | for the users. | 07:39 |
| shardy | skramaja: yeah I guess so, it's similar to the heat environments, although the original idea of plan-environment was to have one definition of the plan | 07:40 |
| shardy | so part of the plan creation workflow would have to be to write one file we use to create the deployment | 07:40 |
| skramaja | shardy: agreed. i am also wondering how these workflow parameters will be used by ui. any idea around it. i have asked jtomasek to have look too. | 07:42 |
| jtomasek | skramaja, shardy: so the problem we're trying to solve with this is how to provide inputs for derive parameters workflows? | 07:44 |
| skramaja | yes jtomasek | 07:44 |
| skramaja | jtomasek: with those parameters, derive params workflow will ouput the heat parameteres, to be used for deployment. | 07:45 |
| *** ratailor is now known as ratailor|Lunch | 07:45 | |
| openstackgerrit | Martin André proposed openstack-infra/tripleo-ci master: Add undercloud-containers job to cistatus https://review.openstack.org/446957 | 07:45 |
| jtomasek | skramaja, shardy: from UI point of view, we'd just want to list workflow inputs with its defaults and let user edit them in the form | 07:45 |
| jtomasek | skramaja, shardy: which I believe Mistral should be able to do now | 07:45 |
| jtomasek | skramaja, shardy: it seems to me that those new 'special' environments are just a way for CLI to provide those input defaults | 07:46 |
| shardy | jtomasek: Ok, I guess we'll have to think about that, becaue mistral doesn't provide a parameter/input schema like heat does, only key/value pairs I think? | 07:46 |
| shardy | jtomasek: note these aren't special environments, it's a new data structure that's part of the plan-environment | 07:46 |
| jtomasek | shardy: if we need certain nested structure, the input then needs to be json? | 07:47 |
| jtomasek | (in the end anyway) | 07:47 |
| shardy | jtomasek: we started out considering putting these into the service templates, but that's potentially confusing because the values aren't ever passed directly to heat | 07:47 |
| jtomasek | shardy: I know, it is files such as roles-data.yaml | 07:47 |
| shardy | jtomasek: sure, it's json but we don't know the types - mistral allows specifying defaults but I'm not sure how we preview them | 07:48 |
| openstackgerrit | Luca Lorenzetto proposed openstack/tripleo-docs master: Fixed page title https://review.openstack.org/457905 | 07:48 |
| *** bogdando has joined #tripleo | 07:48 | |
| shardy | jtomasek: I guess you'd just have a way to show the workflow_parameters from the enabled plan-environment, and assume it was all text? | 07:48 |
| jtomasek | d0ugal: do you know if Mistral API is able to list certain workflow inputs? | 07:48 |
| jtomasek | d0ugal: (list of inputs + default values) | 07:49 |
| shardy | jtomasek: yes it looks like it: | 07:49 |
| shardy | http://paste.openstack.org/show/607114/ | 07:49 |
| openstackgerrit | Simon Leinen proposed openstack/diskimage-builder master: Turn off strict_id mode for Ec2 datasource https://review.openstack.org/457015 | 07:49 |
| *** jpich has joined #tripleo | 07:49 | |
| shardy | it's just key/value pairs but the data is exposed | 07:50 |
| skramaja | shardy: jtomasek A crude way is, add a mistral action to provide the required documentation on the workflow parameters. | 07:50 |
| skramaja | *mistral workflow | 07:50 |
| skramaja | which is specific to derive parmeters. | 07:50 |
| shardy | skramaja: so you pass a list of enabled environments, and get back a list of workflow parameters, and the defaults? | 07:51 |
| skramaja | shardy: yes. list of plan-environments to be specific. | 07:51 |
| jtomasek | skramaja, shardy: I quite like the Mistral approach more because we could avoid adding additional files. Then we'd just need to identify a way for CLI to provide those inputs somehow | 07:51 |
| shardy | skramaja: I guess I'm wondering if we can just put the defaults directly into the workflows? | 07:52 |
| bogdando | jistr, shardy, mandre, weshay: hello folks, PTAL https://review.openstack.org/#/c/450792/ it has passed ovb containers CI | 07:52 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: TLS-everywhere: Enable for TLS libvirt live migration https://review.openstack.org/450726 | 07:52 |
| shardy | then e.g mistral workflow-get <workflow> shows the defaults | 07:52 |
| shardy | skramaja: there are also tags for workflows | 07:52 |
| shardy | can we just tag the workflow with the environment or service name? | 07:52 |
| shardy | then there could be a workflow that transforms a list of enabled environments or services into a list of workflows and workflow parameters (and the defaults)? | 07:53 |
| *** amoralej|off is now known as amoralej | 07:54 | |
| cylopez | Hello folk, How I can use environment to push entree in /etc/hosts ? | 07:54 |
| cylopez | in newton ? | 07:54 |
| *** yolanda has quit IRC | 07:54 | |
| *** jerrygb has joined #tripleo | 07:55 | |
| skramaja | shardy: we will not know the service name, so it is not an option. the list of enabled environments per role is decided based on the service enabled on the role. | 07:55 |
| jtomasek | shardy, skramaja: btw. it would be ideal if derive parameters workflows did not require any input. We're providing parameters to generate different parameters here... | 07:55 |
| bogdando | flaper87: ^ ^ please also take a look https://review.openstack.org/#/c/450792/ it follows up on the merged undercloud containers and allows customized cases | 07:55 |
| *** mhenkel_ has joined #tripleo | 07:55 | |
| therve | Hearing about PTAL gives me some PTSD: https://pypi.python.org/pypi/zope.tal | 07:56 |
| shardy | jtomasek: yeah, I think the idea is we specify sane defaults, but invariably where a calculation is involved, someone will want to tweak the values | 07:56 |
| skramaja | jtomasek: shardy derive params is nothing but a set of formulas, we need to provide customization to the formuls which are mandatory based on the environent. | 07:56 |
| shardy | skramaja: I guess the question is, how often will folks need to change the formulas? | 07:56 |
| shardy | skramaja: e.g would it be acceptable to defer this interface change, and say they have to upload a modified workflow yaml with different defaults, directly to mistral? | 07:57 |
| shardy | then the per-deploy interface doesn't change at all | 07:57 |
| skramaja | shardy: lesser probablity to change formauls, because it is writted based onperformnce team output. but there are variable in it. | 07:57 |
| jtomasek | confusion for the user is that he provides some configuration which produces another configuration and then he may come to a problem of not knowing which 'layer' of configuration to tweak afterwards | 07:57 |
| skramaja | shardy: for example, single queue or multi quue for dpdk is operators choice. | 07:58 |
| skramaja | shardy: based on what they choose we need to derive parametters. | 07:58 |
| jtomasek | ok | 07:58 |
| shardy | jtomasek: I agree, which is why I wanted to avoid putting these mistral parameters into the heat templates | 07:58 |
| skramaja | jtomasek: the purpose of tje derive params, is to ease the deployments | 07:59 |
| jtomasek | shardy: +1 | 07:59 |
| shardy | my preference is we just allow per-workflow parameters via the plan_environment, but leave all the defaults in the workflows themselves | 07:59 |
| skramaja | jtomasek: so the opeartor does not worry about the details in depth. | 07:59 |
| shardy | then we have a way to override defaults, but it doesn't change the interfaces much, just one new key in plan_environment | 08:00 |
| *** zzzeek has quit IRC | 08:00 | |
| shardy | we have to figure out how to get the data from plan-environment into each workflow though | 08:00 |
| jtomasek | shardy: I think that is something to implement in tripleo-client | 08:00 |
| *** jerrygb has quit IRC | 08:00 | |
| jtomasek | as it is CLI specific | 08:00 |
| shardy | jtomasek: it's not CLI specific, it's specific to the workflows | 08:01 |
| *** zzzeek has joined #tripleo | 08:01 | |
| *** hjensas has joined #tripleo | 08:01 | |
| shardy | jtomasek: if you write some workflow parameters to plan-environment, then run the workflow, how does it know to use the new values? | 08:01 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/ocata: Migrate Swift ring handling from tripleo-heat-templates to puppet-tripleo https://review.openstack.org/455074 | 08:01 |
| shardy | each workflow will have to read it explitly from the plan-environment I guess | 08:01 |
| openstackgerrit | Merged openstack/tripleo-common master: Add skip_deploy_identifier https://review.openstack.org/452314 | 08:02 |
| skramaja | shardy: jtomasek my intentation was to read it in the workflow | 08:02 |
| *** athomas has quit IRC | 08:02 | |
| shardy | skramaja: Ok, that works | 08:02 |
| skramaja | jtomasek: shardy read the plan-environment and get the workflow parameters and apply the formulas. | 08:02 |
| jtomasek | shardy, skramaja: hm, so we're not using Mistral workflow 'input' feature in that case | 08:03 |
| *** snecklifter has quit IRC | 08:04 | |
| shardy | jtomasek: I think perhaps we'd want a wrapper workflow that reads plan-environment, then calls all the calculate parameters workflows | 08:04 |
| skramaja | shardy: jtomasek we will add a workflow which returns the parametere and defaults. | 08:04 |
| shardy | then the input feature is used, but it can be overridden by the parent workflow | 08:04 |
| skramaja | shardy: jtomasek i prefer not to use the input feature, but we will conclude on specific details on workflow reviews. as of now on high leve, we are good. | 08:05 |
| d0ugal | jtomasek: good question, I'll check. | 08:05 |
| jtomasek | from UI POw we'd not need to store anything in plan-environment. I'd read defaults from the workflow via mistral api and let user provide changes as input to the workflow | 08:06 |
| jtomasek | If I am not missing something | 08:06 |
| jtomasek | to read last used inputs I'd just fetch the latest execution maybe | 08:06 |
| d0ugal | jtomasek: it looks like the inputs are in the response for getting a workflow? http://www.discode.co/2Z | 08:06 |
| jtomasek | d0ugal: yeah, right. thanks | 08:07 |
| d0ugal | jtomasek: but that isn't very useful, as in that case we have a list, boolean and string. | 08:07 |
| d0ugal | jtomasek: so automatically creating a UI for that isn't trivial | 08:07 |
| *** lucas-afk is now known as lucasagomes | 08:07 | |
| *** mcornea has joined #tripleo | 08:07 | |
| jtomasek | d0ugal: is it like that when using API too? (not json?) | 08:08 |
| skramaja | jtomasek: ohh. provding back to the workfow is a point to consider, can we not update the plan-environment? | 08:08 |
| skramaja | jtomasek: incase of preview and deriving again, it is need to restore the same values. so we have to store it some where.. | 08:08 |
| d0ugal | jtomasek: it is just a string it seems - not json | 08:08 |
| openstackgerrit | Madhuri Kumari proposed openstack/diskimage-builder master: Fix the wrong sentence in baremetal README https://review.openstack.org/457912 | 08:08 |
| d0ugal | jtomasek: but I think the biggest issue is that nothing tells you what the type of each input is | 08:09 |
| jtomasek | skramaja: that would be stored in workflow execution | 08:09 |
| jtomasek | d0ugal: ack, but in case of having this stored in yaml, it is the same IIUC | 08:09 |
| *** leanderthal|afk has quit IRC | 08:10 | |
| *** karimb has joined #tripleo | 08:10 | |
| skramaja | jtomasek: storing the customized values in the plan environment is ideal, we can refer it anytime we need. this is what CLI is doing, CLI will update the plan-environment. I prefer UI also does the same. | 08:10 |
| jtomasek | skramaja: ok, I am not opposed to that | 08:11 |
| bogdando | folks, let's please merge https://review.openstack.org/#/c/456583/ | 08:11 |
| d0ugal | jtomasek: I don't follow :) but I am probably just missing some context. | 08:11 |
| bogdando | shardy: ^ ^it fixes a subtle issue with explicit puppet tags | 08:12 |
| *** akrivoka has joined #tripleo | 08:12 | |
| bogdando | folks are used to put anything non empty in order to make validation check passing. While it must be working with the full list of default tags | 08:12 |
| shardy | bogdando: lgtm | 08:15 |
| *** dsariel has joined #tripleo | 08:17 | |
| jtomasek | skramaja: btw. updating the plan-environment needs to be done via mistral action (used by CLI and GUI) not done directly by CLI | 08:18 |
| skramaja | jtomasek: agreed. | 08:18 |
| jtomasek | skramaja, shardy: one more question | 08:19 |
| *** hewbrocca_afk is now known as hewbrocca | 08:19 | |
| jtomasek | skramaja, shardy: nm, I'll ask in review... | 08:20 |
| *** leanderthal|afk has joined #tripleo | 08:22 | |
| jaosorior | owalsh: can you check this out https://review.openstack.org/#/c/450726/ ? | 08:25 |
| *** anshul has joined #tripleo | 08:25 | |
| *** derekh has joined #tripleo | 08:25 | |
| jaosorior | owalsh: just tried it out in my deployment | 08:25 |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-specs master: Deriving TripleO Parameters https://review.openstack.org/423304 | 08:27 |
| *** gfidente has joined #tripleo | 08:29 | |
| *** gfidente has joined #tripleo | 08:29 | |
| *** yolanda has joined #tripleo | 08:30 | |
| *** zoli|gone is now known as zoliXXL | 08:32 | |
| *** numans has quit IRC | 08:33 | |
| openstackgerrit | Merged openstack/tripleo-validations master: Updated from global requirements https://review.openstack.org/456752 | 08:40 |
| *** janki has quit IRC | 08:41 | |
| *** numans has joined #tripleo | 08:44 | |
| *** chem has joined #tripleo | 08:47 | |
| *** janki has joined #tripleo | 08:49 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-common master: Generate RabbitCookie https://review.openstack.org/457927 | 08:51 |
| shardy | jaosorior: Hey, if you have a moment can you help me understand where haproxy_stats_certificate is supposed to be set? | 08:53 |
| shardy | https://github.com/openstack/puppet-tripleo/blob/master/manifests/haproxy.pp#L145 | 08:53 |
| shardy | I'm trying to do a non-ssl minimal deployment and it ends up undef, which then blows up because we conditionally define haproxy_stats_bind_certificate | 08:54 |
| *** salmankhan has joined #tripleo | 08:54 | |
| shardy | grep doesn't show it set anywhere in t-h-t, so I'm confused | 08:54 |
| jaosorior | shardy: It's not set AFAIK (though it could now that I think about it) | 08:55 |
| jaosorior | shardy: this TODO was left there https://github.com/openstack/puppet-tripleo/blob/master/manifests/haproxy.pp#L732 | 08:55 |
| shardy | jaosorior: ah, I see - the puppet output is actually only a warning | 08:55 |
| shardy | so something else must be broken | 08:55 |
| shardy | jaosorior: thanks | 08:55 |
| *** janki has quit IRC | 08:56 | |
| *** salmankhan has quit IRC | 08:57 | |
| *** janki has joined #tripleo | 08:57 | |
| jaosorior | shardy: if you want, just paste the logs and I can take a look | 08:57 |
| *** salmankhan has joined #tripleo | 08:58 | |
| shardy | jaosorior: thanks, I think it's unrelated to the ssl stuff so I'll keep digging, thanks though :) | 08:58 |
| *** yolanda_ has joined #tripleo | 08:59 | |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-common master: move mistral base action dependency to mistral_lib https://review.openstack.org/454632 | 09:00 |
| *** pgadiya has quit IRC | 09:01 | |
| *** yolanda_ has quit IRC | 09:01 | |
| *** yolanda_ has joined #tripleo | 09:01 | |
| *** yolanda has quit IRC | 09:03 | |
| *** ratailor|Lunch is now known as ratailor | 09:05 | |
| openstackgerrit | Bartosz Stopa proposed openstack/tripleo-puppet-elements master: Add neutron-linuxbridge package https://review.openstack.org/457934 | 09:06 |
| openstackgerrit | Bartosz Stopa proposed openstack/puppet-tripleo master: Add linuxbridge agent profile https://review.openstack.org/457936 | 09:06 |
| openstackgerrit | Bartosz Stopa proposed openstack/tripleo-heat-templates master: Add support for linuxbridge agent https://review.openstack.org/457938 | 09:08 |
| *** stendulker has joined #tripleo | 09:09 | |
| *** eck` is now known as eck`gone | 09:12 | |
| *** ckyriakidou has joined #tripleo | 09:15 | |
| *** hjensas has quit IRC | 09:16 | |
| *** pgadiya has joined #tripleo | 09:16 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: TLS-everywhere: Enable for TLS libvirt live migration https://review.openstack.org/450726 | 09:20 |
| jaosorior | marios: thanks for the review, added the release note ^^ | 09:20 |
| openstackgerrit | Martin André proposed openstack-infra/tripleo-ci master: [WIP] add containerized deployment on multinode https://review.openstack.org/454152 | 09:20 |
| *** fragatin_ has joined #tripleo | 09:20 | |
| marios | jaosorior: thanks | 09:21 |
| *** fragatina has quit IRC | 09:22 | |
| *** numans has quit IRC | 09:24 | |
| jaosorior | marios: thanks for the review, dude | 09:26 |
| *** zoliXXL is now known as zoli|lunch | 09:28 | |
| *** udesale__ has joined #tripleo | 09:30 | |
| *** psahoo_ has joined #tripleo | 09:31 | |
| *** psahoo has quit IRC | 09:31 | |
| *** psahoo_ has quit IRC | 09:32 | |
| *** psahoo has joined #tripleo | 09:32 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates master: WIP: N->O upgrade, fix wrong parameters to nova placement. https://review.openstack.org/457965 | 09:32 |
| bogdando | folks, I have a question for init steps, how this ever works with /var/lib/mysql:ro? https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/database/mysql.yaml#L118-L126 | 09:32 |
| bogdando | shardy, jistr: ^^ ? | 09:32 |
| openstackgerrit | Stephen Finucane proposed openstack/python-tripleoclient master: Explicitly set 'builders' option https://review.openstack.org/457966 | 09:33 |
| *** udesale has quit IRC | 09:33 | |
| *** ykarel has quit IRC | 09:33 | |
| bogdando | I'm fixing containerized rabbitmq, and wanted to move policy/user tags there as well | 09:33 |
| bogdando | but stuck with mount mode for /var/lib/rabbitmq | 09:33 |
| *** psahoo has quit IRC | 09:34 | |
| bogdando | I'll make it rw... | 09:36 |
| bogdando | but I'm really confused how an ephemeral init mysql manages to create database, users and grants with RO mode... | 09:36 |
| *** ykarel has joined #tripleo | 09:36 | |
| jistr | bogdando: i think the one you pointed at just connects to an mysql which is already running in a different container (or on a different server) | 09:37 |
| jistr | puppet_tags: 'mysql_database,mysql_grant,mysql_user' | 09:37 |
| jistr | ^ that suggests it just performs SQL operations | 09:38 |
| bogdando | jistr: ...on a read only mounted /var/lib/mysql | 09:38 |
| bogdando | that is so odd | 09:38 |
| jistr | bogdando: no, remotely, via a tcp connection | 09:38 |
| jistr | i mean that would be my expectation :) | 09:38 |
| jistr | whether that actually happens, might be another thing :) | 09:39 |
| bogdando | My understanding it runs an ephemeral container, does the init and stops it | 09:40 |
| *** prateek_ has joined #tripleo | 09:40 | |
| bogdando | connecting to nothing else | 09:40 |
| chem | marios: https://bugs.launchpad.net/tripleo/+bug/1684058, I'm not sure we should restart the nova-compute at all | 09:40 |
| openstack | Launchpad bug 1684058 in tripleo "N -> O upgrade: wrong nova placement parameters." [Critical,Confirmed] | 09:40 |
| *** tosky has joined #tripleo | 09:40 | |
| bogdando | jistr: we shold check if that works at all | 09:40 |
| *** psahoo has joined #tripleo | 09:40 | |
| chem | marios: it's still newton at that moment and don't need the new parameter or do I miss something ? | 09:41 |
| chem | mcornea: ^ | 09:41 |
| *** prateek_ has quit IRC | 09:41 | |
| *** prateek_ has joined #tripleo | 09:41 | |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations master: Add host list by service to inventory https://review.openstack.org/457972 | 09:41 |
| jistr | bogdando: yes exactly, but i think that one doesn't run mysql process, IIUC it runs puppet | 09:42 |
| *** prateek has quit IRC | 09:42 | |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates master: WIP: Providing parameters specific to a workflow via plan-environment https://review.openstack.org/457874 | 09:42 |
| bogdando | jistr: oh well, you're right. But I'm still wondering how it persists results | 09:42 |
| bogdando | /etc is mounted ro as well :o | 09:42 |
| marios | chem: well its meant to be for picking up that placement config i think owalsh was helping us with that at the time.. owalsh we are talking about https://github.com/openstack/tripleo-heat-templates/blob/299b9f532377a3a0c16ba9cb4fe92c637fc38eeb/puppet/major_upgrade_steps.j2.yaml#L63 which happens on nova-compute nodes before the controller ansible upgrade | 09:43 |
| bogdando | magic | 09:43 |
| jistr | bogdando: i think it connects to another container/host via tcp and runs SQL queries | 09:43 |
| jistr | and the other container persists the results | 09:43 |
| bogdando | jistr: hm, but if there are none up and ready?.. | 09:43 |
| marios | chem: i think it was 'in order to allow nova compute to talk to the upgraded nova and other services on controllers after we upgrade _those_ we should set these things on nova node first | 09:43 |
| shardy | jistr, bogdando: Yes that's my understanding too, and I hit an issue with the socket yesterday which proves it I think | 09:44 |
| jistr | bogdando: this one should be running https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/database/mysql.yaml#L109-L116 | 09:44 |
| bogdando | shardy, jistr: nice! so we have a bug | 09:44 |
| shardy | bogdando: the container and the init are done in step3 | 09:44 |
| shardy | bogdando: perhaps, I was repeatedly running the undercloud install locally and may have hit something specific to my environment | 09:44 |
| bogdando | we should be using step+1 for init perhaps | 09:44 |
| shardy | bogdando: well we do the init tasks after starting the container due to the ordering in docker-steps.j2 | 09:45 |
| chem | marios: oki, but only when the package is upgraded has for nova compute newton I think placement configuration is irrelevent | 09:45 |
| chem | marios: /has/as/ | 09:45 |
| bogdando | shardy: ah, I see. thanks | 09:45 |
| jistr | bogdando, shardy: i actually checked the steps as it seemed weird to me too (possible race) but the puppet step should be done after normal step | 09:45 |
| jistr | so from that point of view it should be ok | 09:45 |
| shardy | bogdando: so they container and the init can be in the same step, unless there is some chance they will race, e.g the container won't be ready | 09:45 |
| jistr | yea that could be an issue | 09:45 |
| chem | marios: (I didn't check really ... but I *think* they are irrelevent) | 09:45 |
| jistr | if mysql takes too long before accepting connections | 09:46 |
| shardy | the issue I hit yesterday did look a bit like that, as the socket connection was refused | 09:46 |
| shardy | but I've not been able to reliably reproduce it since | 09:46 |
| chem | owalsh: hi, if you have some time to enlighten us here :) | 09:46 |
| bogdando | shardy: this may be reproduced with a stub command for a container | 09:46 |
| bogdando | like sleep 100 | 09:46 |
| bogdando | then run mysqld_safe | 09:47 |
| bogdando | let's fill a bug to investigate | 09:47 |
| bogdando | data races are not nice | 09:47 |
| shardy | bogdando: yeah perhaps, now my undercloud deploy times out on step3 so still investigating that | 09:47 |
| shardy | definitely running cleanup.sh && run.sh repeatedly is not completely solid yet | 09:48 |
| bogdando | shardy: I'll try to repro with sleep command as well | 09:48 |
| shardy | it worked a few times, then I started having problems | 09:48 |
| yolanda_ | hi, i'm hitting a 401 error on os-collect-config, on the compute node, when trying an upgrade | 09:49 |
| yolanda_ | what can be causing that? it fails on the aodh upgrade | 09:49 |
| marios | chem: right you mean for newton configuring placement is irrelevant | 09:49 |
| shardy | actually the mariadb container is restarting, whats the best way to see why? | 09:49 |
| bogdando | shardy, jistr: could you please fill a bug, or I can do as well? | 09:49 |
| chem | marios: matbu_ bandini, hi shouldn't we start this only on the bootstrap node https://github.com/openstack/tripleo-heat-templates/blob/299b9f532377a3a0c16ba9cb4fe92c637fc38eeb/puppet/services/pacemaker.yaml#L146 | 09:50 |
| marios | chem: i need to check the module, but it might not be setting --all | 09:50 |
| bogdando | shardy: I do like this: docker stop foo; docker run --env KOLLA_BOOTSTRAP=True --env KOLLA_CONFIG_STRATEGY=COPY_ALWAYS -it --rm --name rmq -v /var/lib/foo/:/var/lib/foo -v /var/lib/kolla/config_files/foo.json:/var/lib/kolla/config_files/config.json:ro -v /var/lib/config-data/foo/etc/foo/:/etc/rabbitmq/:ro -v /etc/hosts:/etc/hosts:ro -v /etc/localtime:/etc/localtime:ro tripleoupstream/centos-binary-foo:latest /usr/lib/rabbitmq/bin/foo | 09:50 |
| marios | chem: matbu would know :D | 09:51 |
| bogdando | KOLLA_* depends... | 09:51 |
| bogdando | and volumes depends on that we have for it in t-h-t | 09:51 |
| marios | matbu_: are you still using https://github.com/redhat-openstack/ansible-pacemaker (looks like it) | 09:52 |
| openstackgerrit | Stephen Finucane proposed openstack/python-tripleoclient master: Use Sphinx 1.5 warning-is-error https://review.openstack.org/457981 | 09:52 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Fix RabbitMQ service deployment https://review.openstack.org/457984 | 09:55 |
| matbu_ | marios: yep | 09:55 |
| matbu_ | marios: well not me, but the upgrade does :) | 09:56 |
| openstackgerrit | Luke Hinds proposed openstack/tripleo-heat-templates master: [WIP] Implements management of `/etc/login.defs` https://review.openstack.org/457985 | 09:56 |
| marios | matbu_: chem yeah so afaics it is pcs cluster stop/start but not --all so if we want it to happen on bootstrap only we'd need to change that for starters | 09:57 |
| * matbu_ should update his nick, missing ping | 09:57 | |
| *** matbu_ is now known as matbu | 09:57 | |
| marios | matbu_: chem i mean https://github.com/redhat-openstack/ansible-pacemaker/blob/master/modules/pacemaker_cluster.py#L115 | 09:57 |
| marios | matbu: ^ :D | 09:57 |
| matbu | marios: you mean we should use the pcs cluster start --all ? | 09:58 |
| jistr | yolanda_: hi, i remember seeing some 401s in os-collect-config when /etc/os-collect-config.conf got somehow garbled | 09:58 |
| bogdando | shardy, jistr: I'll file the bug. And here is rabbit fixes (related to mysql races as well) https://review.openstack.org/#/c/457984/ | 09:58 |
| jistr | yolanda_: i think some of that was related to problems with nova metadata, something else also to restoring from backups, but i only recall it very vaguely | 09:59 |
| openstackgerrit | Luke Hinds proposed openstack/puppet-tripleo master: [WIP] Introduces puppet module for `/etc/login.defs` https://review.openstack.org/457989 | 09:59 |
| chem | marios: matbu hum ... didn't know about the --all option. That mean the the "pcs cluster stop" only stop resource on the local node | 09:59 |
| jistr | yolanda_: i'd try restarting nova services on the undercloud, and then maybe os-collect-config on the overcloud node that has problems | 10:00 |
| yolanda_ | ok let me try | 10:00 |
| openstackgerrit | Luke Hinds proposed openstack/tripleo-heat-templates master: [WIP] Implements management of `/etc/login.defs` https://review.openstack.org/457985 | 10:01 |
| marios | chem: yeah that is my understanding anyway | 10:01 |
| *** karimb has quit IRC | 10:02 | |
| * jistr lunch, bbl | 10:02 | |
| matbu | marios: chem we could just add an option | 10:03 |
| matbu | to the module | 10:03 |
| chem | matbu: marios I would not change this if we're not sure it's needed :) | 10:03 |
| matbu | actually, we should use the node parameter, to set the all | 10:03 |
| matbu | chem: yep of course | 10:03 |
| matbu | btw my upgrade testing is broken since yesterday, an issue with puppet, | 10:04 |
| matbu | chem: marios did you hit something ? | 10:04 |
| openstackgerrit | Merged openstack/tripleo-ui stable/ocata: Imported Translations from Zanata https://review.openstack.org/456118 | 10:04 |
| chem | matbu: well there is this https://bugzilla.redhat.com/show_bug.cgi?id=1441635 which is still unexplained | 10:05 |
| openstack | bugzilla.redhat.com bug 1441635 in rhosp-director "OSP10 -> OSP11 upgrade: nova instance live migration gets stuck with MIGRATING status before running compute node upgrade" [Urgent,Assigned] - Assigned to sathlang | 10:05 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Drop extra UNDERCLOUD_SSL=0 from undercloud block https://review.openstack.org/457254 | 10:05 |
| *** janki has quit IRC | 10:05 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Rework DOCKER_PUPPET_CONFIG validate tool https://review.openstack.org/456583 | 10:05 |
| matbu | chem: 100% reproducible ? | 10:06 |
| bogdando | shardy, jistr: https://bugs.launchpad.net/tripleo/+bug/1684075 | 10:07 |
| openstack | Launchpad bug 1684075 in tripleo "Data races for init containers (docker_puppet_tasks) vs docker_config steps" [Undecided,New] | 10:07 |
| chem | matbu: nope, and only with ipv6 it seems | 10:07 |
| yolanda_ | jistr, same error after restarts | 10:08 |
| *** limao has quit IRC | 10:09 | |
| *** limao has joined #tripleo | 10:09 | |
| *** karimb has joined #tripleo | 10:10 | |
| shardy | bogdando: thanks - I wonder if we need some validation step, I know weshay has started looking at that | 10:10 |
| shardy | it'd be nice to start the container, prove it's listenting etc, then do the init | 10:11 |
| matbu | jistr: yolanda_ it's during the aodh migration, do you have some debug info output from heat ? | 10:13 |
| *** limao has quit IRC | 10:14 | |
| yolanda_ | hi matbu , where can i get it? i can extract the files if you point me | 10:15 |
| yolanda_ | in the undercloud, i just see heat iterating through it | 10:15 |
| yolanda_ | and on the computes i just saw this 401 | 10:16 |
| *** janki has joined #tripleo | 10:16 | |
| matbu | yolanda_: i used to debug with this file: https://github.com/matbu/openstack-notes/blob/master/tools/heat-debug.py | 10:22 |
| matbu | yolanda_: source stackrc before | 10:22 |
| yolanda_ | on undercloud? | 10:22 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates stable/ocata: Use comma_delimited_list for token flush cron time settings https://review.openstack.org/458003 | 10:22 |
| *** cmyster has quit IRC | 10:22 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates stable/ocata: Run token flush cron job hourly by default https://review.openstack.org/458005 | 10:23 |
| matbu | yolanda_: yes | 10:23 |
| yolanda_ | ok trying | 10:23 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Containerize etcd service https://review.openstack.org/447627 | 10:25 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Fix RabbitMQ service deployment https://review.openstack.org/457984 | 10:26 |
| yolanda_ | i don't have any deployment in failed state, just stuck on update | 10:26 |
| yolanda_ | because compute is just looping on the 401 error | 10:26 |
| yolanda_ | matbu ^ | 10:27 |
| *** ykarel has quit IRC | 10:31 | |
| yolanda_ | matbu, that's the stacks that are in progress, just computes | 10:32 |
| yolanda_ | http://paste.openstack.org/show/607130/ | 10:32 |
| yolanda_ | caused by that 401 | 10:32 |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart-extras master: Update tempest tag to 16.0.0 for ocata https://review.openstack.org/458013 | 10:32 |
| yolanda_ | matbu, and in computes, in the /var/log/messages, i just see | 10:35 |
| yolanda_ | http://paste.openstack.org/show/607131/ | 10:35 |
| *** udesale__ has quit IRC | 10:36 | |
| *** janki has quit IRC | 10:37 | |
| *** janki has joined #tripleo | 10:37 | |
| *** prateek has joined #tripleo | 10:38 | |
| *** prateek_ has quit IRC | 10:38 | |
| shardy | bogdando: thanks for the command - seems like we could use a good way to surface that for each service without having to look at the templates, and also to bind the per-service logs into the normal locations on the undercloud | 10:41 |
| shardy | it's kind of annoying there's nothing logged anywhere when a service breaks | 10:41 |
| *** janki has quit IRC | 10:41 | |
| *** jkilpatr has quit IRC | 10:42 | |
| bogdando | shardy: wrt logs, here is updated WIP that I'm testing atm (looks good so far!) https://review.openstack.org/#/c/442603/ | 10:42 |
| bogdando | memcached is only tbd | 10:42 |
| shardy | bogdando: nice, let me try it :) | 10:43 |
| yolanda_ | matbu, i checked the metadata urls in heat, and it matches with the one in os-collect-config on the compute | 10:43 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/ocata: Modify pci_passthrough hiera value as string https://review.openstack.org/451168 | 10:45 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Ensure we configure ssl.conf https://review.openstack.org/456712 | 10:45 |
| *** suuuper has joined #tripleo | 10:46 | |
| bogdando | shardy: for the captured docker commands, the most elegant solution I can think of is to make heat agent docker hook to log that it runs | 10:50 |
| bogdando | so devs could just look and reply it | 10:50 |
| *** janki has joined #tripleo | 10:50 | |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates master: Add role specific information to the service template https://review.openstack.org/437956 | 10:51 |
| *** janki has quit IRC | 10:53 | |
| *** bogdando has quit IRC | 10:54 | |
| openstackgerrit | Merged openstack/puppet-tripleo stable/ocata: Enable creation of keystone domain when ldap backends are created https://review.openstack.org/454952 | 10:54 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Remove update abort https://review.openstack.org/449241 | 10:54 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Introduce common resources for docker templates https://review.openstack.org/456629 | 10:54 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Introduce common CAs to be mounted to the containers https://review.openstack.org/456630 | 10:54 |
| yolanda_ | if looks as a swift problem | 10:55 |
| jaosorior | can someone review this https://review.openstack.org/#/c/457582/ ? | 10:56 |
| *** Vijayendra has quit IRC | 10:56 | |
| *** prateek_ has joined #tripleo | 10:57 | |
| *** pkovar has joined #tripleo | 10:58 | |
| *** zoli|lunch is now known as zoliXXL | 10:58 | |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo master: Allow to configure haproxy daemon's status https://review.openstack.org/458025 | 10:59 |
| *** prateek has quit IRC | 11:00 | |
| *** jkilpatr has joined #tripleo | 11:00 | |
| *** bogdando has joined #tripleo | 11:01 | |
| *** jpich has quit IRC | 11:01 | |
| shardy | jistr, bogdando: hey were you discussing a port conflict earlier? | 11:02 |
| shardy | the reason mysql won't start is the TCP port is already in use | 11:02 |
| *** dmk0202 has joined #tripleo | 11:02 | |
| shardy | it looks like a conflict with haproxy on the host? | 11:02 |
| *** ffiore_ has joined #tripleo | 11:03 | |
| *** sudipto has quit IRC | 11:03 | |
| *** ffiore has quit IRC | 11:03 | |
| *** sudipto has joined #tripleo | 11:03 | |
| *** sudipto has quit IRC | 11:03 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: containers: TLS in the internal network for telemetry services https://review.openstack.org/458026 | 11:04 |
| *** cmyster has joined #tripleo | 11:06 | |
| *** cmyster has quit IRC | 11:06 | |
| *** cmyster has joined #tripleo | 11:06 | |
| *** stendulker has quit IRC | 11:06 | |
| *** yprokule has joined #tripleo | 11:08 | |
| jistr | yolanda_: not sure if it may have a common root cause with this https://bugs.launchpad.net/tripleo/+bug/1681576 | 11:10 |
| openstack | Launchpad bug 1681576 in tripleo "overcloud deploy giving "401 Unauthorized" when accessing swift from a mistral workflow" [Undecided,Confirmed] | 11:10 |
| jistr | if yes, then folks were able to work around that ^ problem by restarting various things, in the worst case the whole undercloud node | 11:11 |
| openstackgerrit | Radomir Dopieralski proposed openstack/tripleo-common master: Add horizon container https://review.openstack.org/458027 | 11:12 |
| jaosorior | yolanda_: how did you deploy? | 11:12 |
| jaosorior | jistr: that 401 bug in zaqar/swift should have been fixed by https://review.openstack.org/#/c/452098/ | 11:13 |
| EmilienM | hello | 11:13 |
| yolanda_ | jaosorior, i did a deploy with quickstart, then launched the upgrade script | 11:14 |
| openstackgerrit | Joe Talerico proposed openstack/tripleo-heat-templates master: Adding Tuned Service https://review.openstack.org/454123 | 11:15 |
| *** ansmith has quit IRC | 11:15 | |
| jaosorior | yolanda_: so, the computes are getting a 401 when accessing swift? | 11:17 |
| jaosorior | yolanda_: what credentials are they using? | 11:17 |
| jaosorior | EmilienM: sup dude | 11:17 |
| *** janki has joined #tripleo | 11:17 | |
| yolanda_ | jaosorior, those are credentials on os-collect-config.conf | 11:18 |
| yolanda_ | http://paste.openstack.org/show/607139/ | 11:18 |
| yolanda_ | that endpoint is swift | 11:19 |
| yolanda_ | and in /var/log/mesages on the computes i just get | 11:19 |
| yolanda_ | Apr 19 07:18:44 localhost os-collect-config: 2017-04-19 11:18:44.077 32612 WARNING os_collect_config.request [-] 401 Client Error: Unauthorized | 11:19 |
| yolanda_ | Apr 19 07:18:44 localhost os-collect-config: 2017-04-19 11:18:44.078 32612 WARNING os-collect-config [-] Source [request] Unavailable. | 11:19 |
| yolanda_ | Apr 19 07:18:44 localhost os-collect-config: 2017-04-19 11:18:44.078 32612 WARNING os_collect_config.local [-] /var/lib/os-collect-config/local-data not found. Skipping | 11:19 |
| yolanda_ | it's a cycle of that | 11:19 |
| jaosorior | wow, that's strange | 11:19 |
| jaosorior | yolanda_: is that environment accessible from anywhere? | 11:20 |
| *** dmk0202 has quit IRC | 11:21 | |
| *** hjensas has joined #tripleo | 11:21 | |
| *** dmk0202 has joined #tripleo | 11:21 | |
| yolanda_ | no, it's under vpn | 11:21 |
| yolanda_ | on a customer | 11:21 |
| *** liverpooler has quit IRC | 11:21 | |
| EmilienM | jaosorior: yo | 11:22 |
| *** numans has joined #tripleo | 11:25 | |
| openstackgerrit | Radomir Dopieralski proposed openstack/tripleo-heat-templates master: WIP: Containerize Horizon https://review.openstack.org/450303 | 11:27 |
| *** abishop_ has joined #tripleo | 11:30 | |
| *** abishop_ has quit IRC | 11:32 | |
| *** abishop has joined #tripleo | 11:34 | |
| *** ykarel has joined #tripleo | 11:36 | |
| *** pgadiya has quit IRC | 11:37 | |
| openstackgerrit | Lukas Bezdicka proposed openstack/puppet-tripleo stable/ocata: Ensure we configure ssl.conf https://review.openstack.org/458033 | 11:41 |
| *** thrash|g0ne is now known as thrash | 11:42 | |
| *** nmathew- has quit IRC | 11:48 | |
| *** pgadiya has joined #tripleo | 11:52 | |
| *** numans has quit IRC | 11:52 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo master: Move ceilometer upgrade re-run out of collector https://review.openstack.org/458036 | 11:53 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Fix RabbitMQ service deployment https://review.openstack.org/457984 | 11:53 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Deprecate ceilometer collector https://review.openstack.org/450885 | 11:54 |
| *** numans has joined #tripleo | 11:54 | |
| *** karimb has quit IRC | 11:55 | |
| *** karimb has joined #tripleo | 11:56 | |
| *** karimb has quit IRC | 11:57 | |
| *** lucasagomes is now known as lucas-hungry | 11:59 | |
| *** ffiore_ has quit IRC | 11:59 | |
| *** yolanda_ has quit IRC | 12:04 | |
| *** yolanda has joined #tripleo | 12:04 | |
| *** yolanda has quit IRC | 12:04 | |
| *** yolanda has joined #tripleo | 12:05 | |
| *** panda|pto has quit IRC | 12:05 | |
| yolanda | shardy, around? you seem the one that is able to help with the os-collect-config issues | 12:06 |
| yolanda | i'm getting a 401 on upgrade, on the compute node | 12:06 |
| jistr | mandre: fyi i've reported the puppet bugs i see in container job logs | 12:07 |
| *** jpich has joined #tripleo | 12:07 | |
| jistr | https://bugs.launchpad.net/tripleo/+bug/1684104 | 12:07 |
| openstack | Launchpad bug 1684104 in tripleo "container puppet -- fails to curl rabbitmqadmin from rabbitmq" [High,Triaged] | 12:07 |
| jistr | https://bugs.launchpad.net/tripleo/+bug/1684107 | 12:07 |
| openstack | Launchpad bug 1684107 in tripleo "container puppet -- could not find Service[libvirt] for relationship" [High,Triaged] | 12:07 |
| jistr | i've hit the RabbitMQ one locally too, but not the libvirt one | 12:07 |
| *** sudipto has joined #tripleo | 12:09 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo master: Move ceilometer upgrade re-run out of collector https://review.openstack.org/458036 | 12:09 |
| shardy | With the undercloud deploy, are you guys running cleanup.sh before every deploy? | 12:09 |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient master: Add --skip-deploy-identifier https://review.openstack.org/452223 | 12:09 |
| shardy | I expected to be able to run it multiple times, but it looks like that breaks, mysql init fails, and I wonder if it's because we create a new password every deploy | 12:10 |
| *** jpich has quit IRC | 12:12 | |
| bogdando | EmilienM: I would abandon https://review.openstack.org/#/c/457927/1/tripleo_common/constants.py if I understood the use case when one should use tripleo-common, like EtcdInitialClusterToken does, and when - t-h-t | 12:14 |
| bogdando | t-h-t's overcloud.j2.yaml | 12:14 |
| bogdando | EmilienM: and would that work for undercloud deployments, if it is not generated in the tripleo-common but in the overcloud.j2? | 12:15 |
| *** amoralej is now known as amoralej|lunchy | 12:15 | |
| *** amoralej|lunchy is now known as amoralej|lunch | 12:15 | |
| shardy | bogdando: it's a historical thing - initially we generated some passwords and other random things in overcloud.yaml | 12:16 |
| shardy | bogdando: but that didn't work that well, because you can never move the heat resource (e.g to a service template for example) | 12:16 |
| shardy | without re-creating the secret | 12:16 |
| shardy | bogdando: so more recently, we've instead generated passwords in tripleo-common and just passed the values into the heat stack | 12:17 |
| bogdando | EmilienM: shardy: okay, got it! So the fix would be to remove rabbitsalt/cookie from the overcloud.j2 and just use that I have for the tripleo-common, but limited to 20 chars? | 12:18 |
| openstackgerrit | Merged openstack/tripleo-docs master: Fixed page title https://review.openstack.org/457905 | 12:18 |
| bogdando | or would that break backward compat with overcloud.j2<=pike? | 12:18 |
| *** morazi has joined #tripleo | 12:18 | |
| shardy | bogdando: we can't remove any existing secrets from overcloud.j2.yaml, but we shouldn't add any new ones | 12:18 |
| bogdando | got it, thanks! | 12:18 |
| mandre | jistr: ack, I haven't seen it yet | 12:19 |
| jistr | mandre: btw you mentioned trying to reproduce the nova container libvirt error earlier -- did you see that error somewhere else than just the upgrades CI? | 12:21 |
| jistr | i mean https://bugs.launchpad.net/tripleo/+bug/1684107 | 12:21 |
| openstack | Launchpad bug 1684107 in tripleo "container puppet -- could not find Service[libvirt] for relationship" [High,Triaged] | 12:21 |
| jistr | unless you meant a different libvirt error... | 12:21 |
| *** karthiks has quit IRC | 12:22 | |
| EmilienM | bogdando: yeah, constants should be good | 12:23 |
| EmilienM | but like shardy, for future variables | 12:23 |
| mandre | jistr: in the multinode containerized overcloud, https://review.openstack.org/#/c/454152/ | 12:24 |
| bogdando | mandre, jistr: so as we noop providers, we shall dereference any services for containerized puppet steps | 12:24 |
| mandre | jistr: http://logs.openstack.org/52/454152/7/check/gate-tripleo-ci-centos-7-nonha-multinode-oooq/f56c84e/logs/undercloud/home/jenkins/failed_deployment_list.log.txt.gz | 12:24 |
| bogdando | and noop them as well, right? | 12:24 |
| *** jayg|g0n3 is now known as jayg | 12:24 | |
| bogdando | a little change to the noop provider is required | 12:25 |
| bogdando | btw, there are many things like that for the neutron | 12:25 |
| mandre | bogdando: probably yes, it doesn't make sense to use the service resource provider inside containers | 12:25 |
| *** liverpooler has joined #tripleo | 12:25 | |
| bogdando | and that is why I had submitted the Saga of the Atomic Classes :) http://lists.openstack.org/pipermail/openstack-dev/2015-August/073268.html (I hit the same issue with services referenced in puppets) | 12:26 |
| bogdando | I was* | 12:27 |
| bogdando | mandre, jistr: ^^ | 12:27 |
| bogdando | it looks very similat to me, the case | 12:27 |
| bogdando | my solution was just to put stubs for referenced Services | 12:28 |
| *** zoliXXL is now known as zoli|afk | 12:30 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates master: N->O upgrade, fix wrong parameters to nova placement. https://review.openstack.org/457965 | 12:30 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates master: N->O upgrade, fix wrong parameters to nova placement. https://review.openstack.org/457965 | 12:30 |
| *** leanderthal|afk is now known as leanderthal | 12:31 | |
| *** leanderthal is now known as leanderthal|afk | 12:31 | |
| EmilienM | marios, chem: do you know why we use crudini in this file? https://review.openstack.org/#/c/457965/3/puppet/major_upgrade_steps.j2.yaml - instead of using the upgrade_tasks (ansible) in the composable services? | 12:33 |
| EmilienM | I'm confused why we configue things in this file, which is supposed to be generic. | 12:33 |
| EmilienM | shardy ^ any thoughts? | 12:33 |
| shardy | bogdando: actually what we discussed above is why undercloud deploy isn't idempotent - every time we create the disposable heat stack we create a new MysqlRootPassword resource I think | 12:33 |
| chem | EmilienM: it's not happening at the same time | 12:33 |
| EmilienM | chem: can't we do it at step1? | 12:34 |
| *** fultonj has joined #tripleo | 12:34 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Fix containerized RabbitMQ service deployment https://review.openstack.org/458051 | 12:34 |
| *** karimb has joined #tripleo | 12:35 | |
| *** fultonj has quit IRC | 12:35 | |
| chem | EmilienM: well no, because compute nove are (by default) excluded from the upgrade_task process | 12:35 |
| *** fultonj has joined #tripleo | 12:35 | |
| shardy | EmilienM: but yeah you are right - we should generate a playbook from upgrade_tasks and write that to the compute nodes | 12:35 |
| shardy | instead of that crudini stuff | 12:35 |
| EmilienM | yeah, it's very confusing | 12:36 |
| shardy | IIRC that was a quick cut/paste hack | 12:36 |
| shardy | from the old script based approach | 12:36 |
| EmilienM | I'm ok with hacks, as long as we document it in the code as TODO and fixme asap | 12:36 |
| EmilienM | but I don't see that | 12:36 |
| shardy | EmilienM: yeah lets raise a bug so we can track improving it | 12:36 |
| shardy | I remember it was discussed during the review FWIW | 12:36 |
| EmilienM | shardy: I'll do it | 12:36 |
| *** trown|outtypewww is now known as trown | 12:37 | |
| marios | EmilienM: yeah it happens on a compute node where we aren't running the ansible steps at least for n..o | 12:37 |
| chem | EmilienM: check the disable_upgrade_deployment parameter | 12:37 |
| shardy | What I think we should do is make UpgradeBatchConfig a single config, with a playbook of tasks ordered by step | 12:38 |
| yolanda | EmilienM, on OSP8 upgrades, i'm having problem with one hiera value: ceilometer::dispatcher::gnocchi::url: ://: | 12:38 |
| shardy | then we can just write that config to the nodes instead | 12:38 |
| yolanda | see that final value | 12:38 |
| yolanda | is that familiar to you? | 12:38 |
| shardy | EmilienM, marios, chem: It's similar to what I'm doing in https://review.openstack.org/#/q/status:open+project:openstack/tripleo-heat-templates+branch:master+topic:docker_ansible4 | 12:38 |
| chem | shardy: yep, and not being exclusive of the normal upgrade_tasks | 12:39 |
| shardy | I'll see if I can do a patch for the upgrade steps unless anyone else gets to it first | 12:39 |
| shardy | chem: yeah | 12:39 |
| EmilienM | shardy, chem, marios: https://bugs.launchpad.net/tripleo/+bug/1684110 - bug created, can someone take it? | 12:39 |
| openstack | Launchpad bug 1684110 in tripleo "upgrades: remove hacks in puppet/major_upgrade_steps.j2.yaml" [Medium,Triaged] | 12:39 |
| shardy | the one change I think we may need is to switch from tags to a step variable | 12:39 |
| shardy | or, make SoftwareDeployments capable of passing a different tag/option | 12:39 |
| shardy | which currently isn't possible | 12:39 |
| cschwede | Hi! Can I annoy someone to have a look at this t-h-t backport: https://review.openstack.org/#/c/455669/ ? Just needs one more +2/A | 12:40 |
| EmilienM | yolanda: I would check with pradk (not connected it seems) - can you file a bug maybe and share your environment? | 12:40 |
| *** panda|pto has joined #tripleo | 12:40 | |
| EmilienM | cschwede: done | 12:40 |
| cschwede | EmilienM: thx a lot! | 12:41 |
| yolanda | ok | 12:41 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs https://review.openstack.org/359215 | 12:41 |
| chem | EmilienM: taken, will pass it on somebody else if blocker bug keeps coming in | 12:41 |
| *** salmankhan has quit IRC | 12:42 | |
| *** jpich has joined #tripleo | 12:45 | |
| *** ratailor has quit IRC | 12:46 | |
| EmilienM | chem: thx | 12:46 |
| *** mburned is now known as mburned_out | 12:46 | |
| *** ramishra has quit IRC | 12:48 | |
| *** Goneri has joined #tripleo | 12:48 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: [DO_NOT_MERGE] Deploy Ceph using ceph-ansible via external workflow https://review.openstack.org/458058 | 12:49 |
| thrash | trown: what generates this? https://trunk.rdoproject.org/centos7/delorean-deps.repo | 12:50 |
| thrash | trown: it's pointing to the buildlogs.centos.org url | 12:51 |
| thrash | trown: which means if you're following docs, it don't work. :D | 12:51 |
| *** jpena is now known as jpena|lunch | 12:52 | |
| EmilienM | thrash: in CI or locally? | 12:52 |
| thrash | EmilienM: locally | 12:52 |
| thrash | EmilienM: I'm guessing something in CI works around it. | 12:52 |
| EmilienM | thrash: something like here https://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/master.yml | 12:52 |
| thrash | EmilienM: probably. Yeah. :) | 12:53 |
| EmilienM | oh wait | 12:53 |
| *** jerrygb has joined #tripleo | 12:53 | |
| EmilienM | thrash: no, sorry, https://trunk.rdoproject.org/centos7/delorean-deps.repo is not generated by quickstart | 12:53 |
| EmilienM | the file is generated by something else, I would ask jpena|lunch or amoralej|lunch | 12:54 |
| EmilienM | or dmsimard ^ | 12:54 |
| thrash | EmilienM: ack. Thanks. | 12:54 |
| *** ramishra has joined #tripleo | 12:55 | |
| *** artom has quit IRC | 12:55 | |
| *** tzumainn has joined #tripleo | 12:56 | |
| *** artom has joined #tripleo | 12:56 | |
| *** psahoo has quit IRC | 12:56 | |
| *** nmathew- has joined #tripleo | 12:57 | |
| *** nmathew- has quit IRC | 12:57 | |
| thrash | EmilienM: ahh... My problem may be that the docs are still using https://buildlogs.centos.org/centos/7/cloud/x86_64/rdo-trunk-master-tripleo/delorean.repo for the main repo | 12:57 |
| thrash | EmilienM: let me try that first. | 12:58 |
| *** morazi has quit IRC | 12:58 | |
| *** ansmith has joined #tripleo | 13:00 | |
| *** noslzzp has quit IRC | 13:00 | |
| *** ansmith has quit IRC | 13:00 | |
| *** ansmith has joined #tripleo | 13:01 | |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart master: Add supplemental node provisioner https://review.openstack.org/451523 | 13:01 |
| *** mburned_out is now known as mburned | 13:01 | |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Add missing tag to nova-placement docker image https://review.openstack.org/458064 | 13:02 |
| *** jmelvin has joined #tripleo | 13:02 | |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart master: Update quickstart-extras to deploy FreeIPA https://review.openstack.org/453223 | 13:03 |
| florianf | EmilienM: If I want to propose a new tripleo-validations release (ocata), is it as simple as adding a new release section here? https://github.com/openstack/releases/blob/master/deliverables/ocata/tripleo-validations.yaml | 13:03 |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-docs master: Fix dlrn link to point to trunk.rdoproject.org https://review.openstack.org/458065 | 13:04 |
| thrash | EmilienM: ^^^^ | 13:04 |
| thrash | That works. :) | 13:04 |
| *** mburned is now known as mburned_out | 13:04 | |
| EmilienM | florianf: yes ! | 13:05 |
| florianf | EmilienM: Excellent! Thanks. :) | 13:05 |
| EmilienM | florianf: add me as reviewer and I'll +1 | 13:05 |
| EmilienM | thrash: nice shot | 13:05 |
| florianf | EmilienM: great, thx | 13:05 |
| *** lvlds has quit IRC | 13:05 | |
| *** pgadiya has quit IRC | 13:06 | |
| *** lucas-hungry is now known as lucasagomes | 13:07 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Secure RabbitMQ Erlang Cookie https://review.openstack.org/457984 | 13:11 |
| openstackgerrit | Martin André proposed openstack/tripleo-common master: Add cinder container images https://review.openstack.org/458068 | 13:11 |
| *** salmankhan has joined #tripleo | 13:13 | |
| openstackgerrit | Martin André proposed openstack/tripleo-common master: Add cinder container images https://review.openstack.org/458068 | 13:14 |
| *** artom has quit IRC | 13:14 | |
| *** artom has joined #tripleo | 13:14 | |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Docker services for Cinder Api and Scheduler https://review.openstack.org/457011 | 13:15 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-common master: Generate RabbitCookie https://review.openstack.org/457927 | 13:15 |
| *** gkadam has quit IRC | 13:17 | |
| florianf | EmilienM: https://review.openstack.org/#/c/458071/ thanks for reviewing... | 13:18 |
| *** lblanchard has joined #tripleo | 13:19 | |
| *** yolanda has quit IRC | 13:20 | |
| EmilienM | florianf: done | 13:22 |
| florianf | EmilienM: thanks! | 13:22 |
| bogdando | EmilienM, shardy, jaosorior: folks, how could we protect existing secrets for the https://review.openstack.org/#/c/457927/ ? IIUC, the same applies to that we have for EtcdInitialCLusterToken. If it was deployed before upgrade, new token will be generated by the tripleo-common magic as well? | 13:22 |
| *** nyechiel has joined #tripleo | 13:22 | |
| bogdando | I do not understand how the thing works and why (if?) there are idempotency issues impacting upgrades wrt secrets? | 13:23 |
| jaosorior | Hey guys, could someone check this out https://review.openstack.org/#/c/457582/ ? | 13:23 |
| *** hjensas has quit IRC | 13:24 | |
| jaosorior | bogdando: protect them where? | 13:24 |
| shardy | bogdando: The code that generates the passwords looks at the existing environment and only generates those passwords that are missing: | 13:24 |
| shardy | https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/undercloud_deploy.py#L213 | 13:25 |
| bogdando | shardy, jaosorior: so, there is no issue with upgrades, right? | 13:25 |
| shardy | bogdando: for the undercloud deploy we just have to make sure all secrets, including those in overcloud.j2.yaml, are in the tripleo-undercloud-passwords.yaml file | 13:26 |
| shardy | for overcloud deploy, those values are stored in heat and in the plan | 13:26 |
| *** lblanchard has quit IRC | 13:26 | |
| bogdando | shardy: um, when running upgrade for the overcloud, in ocata say, there is no such a secret, so... the cookie will get changed :/ | 13:26 |
| bogdando | this looks like a common issue with secrets and upgrades to me | 13:28 |
| shardy | bogdando: https://github.com/openstack/tripleo-heat-templates/blob/stable/ocata/overcloud.j2.yaml#L486 | 13:28 |
| shardy | bogdando: there is a secret in ocata, so I'd like to clarify why you say it's not set | 13:28 |
| bogdando | and it should be fixed in tripleo client perhaps, with some keyword to start looking for existing passwords | 13:28 |
| bogdando | this may impact other things like changing mysql binary logs size on a fly, which breaks things, right? | 13:29 |
| bogdando | shardy: I'm trying to figure out the upgrade impact from the jaosorior's comment | 13:29 |
| bogdando | and apply an induction :) | 13:29 |
| shardy | bogdando: sorry, on which review? | 13:30 |
| * shardy can only follow one topic at a time ;) | 13:30 | |
| bogdando | shardy: https://review.openstack.org/#/c/457927 | 13:30 |
| *** kjw3 has quit IRC | 13:30 | |
| shardy | bogdando: yes, the problem you're trying to fix there is specific to the undercloud deploy I think | 13:31 |
| shardy | bogdando: it's the same issue I saw with the mysql root password | 13:31 |
| shardy | the issue is we don't put the values from DefaultPasswords into tripleo-undercloud-passwords.yaml | 13:31 |
| shardy | before deleting the temporary heat stack | 13:31 |
| *** rlandy has joined #tripleo | 13:31 | |
| shardy | so they get recreated on the next run of undercloud deploy | 13:31 |
| shardy | unless I'm missing something | 13:32 |
| *** ffiore_ has joined #tripleo | 13:32 | |
| bogdando | shardy, jaosorior: IIUC, the comment applies to overcloud | 13:32 |
| bogdando | "this is unsafe on update/upgrade because it will change the RabbitCookie for the running cluster; I am not against the change but we need to make sure it works as intended on update/upgrade" | 13:32 |
| shardy | bogdando: No, your patches will break overcloud upgrades, but currently they work because we persist the heat stack | 13:32 |
| shardy | bogdando: AFAICT it will change the cookie, and I think we should not do that | 13:32 |
| *** yolanda has joined #tripleo | 13:33 | |
| trown | thrash: I am confused why buildlogs would not work for deps | 13:33 |
| shardy | we should just fix the undercloud deploy to save the secrets it's missing | 13:33 |
| jaosorior | bogdando: that comment wasn't mine :) | 13:33 |
| thrash | trown: turned out is was the delorean repo itself. | 13:33 |
| trown | thrash: the issue with buildlogs is that they stopped syncing our images and dlrn content | 13:33 |
| thrash | https://review.openstack.org/458065 | 13:33 |
| bogdando | shardy: okay, now let's try to iduct to a generic case, for all secrets. It seems a major issue, right? | 13:33 |
| thrash | trown: ^^^ | 13:33 |
| trown | thrash: but the deps should be fine | 13:33 |
| *** eck`gone is now known as eck` | 13:33 | |
| trown | thrash: ah cool | 13:33 |
| *** nyechiel has quit IRC | 13:33 | |
| bogdando | induct, you know, from the specific case to common things, that's that I mean | 13:33 |
| shardy | bogdando: it is a major issue, but only for the undercloud deploy, for a small and well known subset of secrets | 13:33 |
| shardy | bogdando: give me a few minutes and I'll push a patch | 13:34 |
| bogdando | shardy: thanks a lot! I just do not understand the topic right | 13:34 |
| shardy | bogdando: np, it'll probably be easier to discuss with code, and I agree it needs to be fixed | 13:34 |
| bogdando | jaosorior: oh, I'm sorry | 13:35 |
| bogdando | gfidente: ^^ :) | 13:35 |
| jaosorior | EmilienM, jistr: Hey guys, could someone check this out https://review.openstack.org/#/c/457582/ ? | 13:36 |
| bogdando | gfidente: could you please elaborate if the comment applies to overcloud and overcloud? https://review.openstack.org/#/c/457927/ | 13:36 |
| bogdando | and undercloud* | 13:36 |
| yolanda | EmilienM, is that clear enough? https://bugs.launchpad.net/tripleo/+bug/1684125 | 13:37 |
| openstack | Launchpad bug 1684125 in tripleo "L->M upgrade fails with invalid gnocchi_internal_url error" [Undecided,New] | 13:37 |
| yolanda | environment is just the default from quickstart with vms | 13:37 |
| shardy | bogdando: I just said, it's specific to the undercloud, and yes gfidente is right, the patch will break overclouds | 13:38 |
| shardy | bogdando: I think you're only testing with undercloud deploy, right? | 13:38 |
| bogdando | shardy: yes | 13:38 |
| skramaja | flaper87 shardy: do you know if anyone has started to look in to os-net-config containerization? we are starting with the analysis from DPDK prespective. | 13:39 |
| bogdando | shardy: how could we fix overcloud upgrades? | 13:39 |
| bogdando | as I said, we have the same issue for etcd initial token, for example | 13:39 |
| shardy | bogdando: overcloud upgrades aren't broken, we just need to add the secrets to tripleo-common | 13:40 |
| shardy | for any new serviecs | 13:40 |
| shardy | services | 13:40 |
| shardy | skramaja: No, for now we're using net=host which implies running os-net-config on the host | 13:41 |
| shardy | so AFAIK that's not been under investigation yet | 13:41 |
| *** lblanchard has joined #tripleo | 13:41 | |
| bogdando | um, shardy that's exactly that both patches do: etcd token: https://review.openstack.org/#/c/446194/ and rabbit cookie https://review.openstack.org/#/c/457927/. If the latter breaks overcloud upgrades, so does the former | 13:41 |
| bogdando | and that's why I'm trying to figure out the scope of the issue | 13:42 |
| openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: WIP: restrict nova migration ssh tunnel https://review.openstack.org/458077 | 13:42 |
| bogdando | shardy: are there more secrets, like mysql root passwords you've mentioned, affected overcloud as well? | 13:42 |
| *** amoralej|lunch is now known as amoralej | 13:43 | |
| shardy | bogdando: EtcdInitialClusterToken is in tripleo-undercloud-passwords.yaml so I'm not sure why that is broken, lets fix one problem at a time | 13:43 |
| bogdando | makes sense | 13:43 |
| shardy | bogdando: No, the mysql root password is the same problem as the RabbitCookie I think | 13:43 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/default_passwords.yaml | 13:44 |
| shardy | bogdando: AFAICS the problem is only those secrets ^^ | 13:44 |
| shardy | for overcloud deploy, we save them, for undercloud deploy we don't | 13:44 |
| bogdando | shardy: right, it's getting clear now, thanks | 13:44 |
| shardy | it's a tripleoclient bug, so lets fix that then see why the etcd thing doesn't work | 13:45 |
| skramaja | shardy: i am wondering when openvswitch is containarized, how ifcfg scripts will invoke ovs-vsctl commads as these commands will be available only inside the containers. when i spoke the team who is working on ovs containarization, i got to know that they are not looking in to it. may be i am missing somthing here. | 13:45 |
| *** saibarspeis has quit IRC | 13:45 | |
| *** ykarel has quit IRC | 13:46 | |
| numans | beagles, Hi, can you please add this to review queue - https://review.openstack.org/#/c/372274/ | 13:46 |
| shardy | skramaja: ovs isn't currently running in a container, and I don't think that has been investigated yet | 13:46 |
| numans | beagles, its the patch to support ha for ovn | 13:47 |
| shardy | skramaja: dprince, flaper87 or Slower_ may know more though | 13:47 |
| skramaja | shardy: neutron team has started to look in to it.. | 13:47 |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates master: Containerize HAProxy for the non-ha case https://review.openstack.org/458079 | 13:47 |
| skramaja | shardy: neutron team have started the analysis on ovs, but wondering without os-net-config adaption, it will be supported. | 13:48 |
| amoralej | thrash, EmilienM, delorean-deps is generated by puppet-dlrn https://github.com/rdo-infra/puppet-dlrn/blob/master/files/centos-master-uc-dlrn-deps.repo | 13:48 |
| thrash | amoralej: False alarm. I was confused about where certain packages should come from. It was actually the delorean.repo link that was wrong. | 13:49 |
| skramaja | shardy thanks for pointing out ppl, i will reach out | 13:49 |
| amoralej | ok | 13:50 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Pass RabbitMQ Erlang Cookie for containerized service https://review.openstack.org/457984 | 13:52 |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: WIP: restrict nova migration ssh tunnel https://review.openstack.org/458082 | 13:54 |
| *** colonwq has joined #tripleo | 13:55 | |
| jistr | mandre: check out my reply @ https://bugs.launchpad.net/tripleo/+bug/1684104 | 13:56 |
| openstack | Launchpad bug 1684104 in tripleo "container puppet -- fails to curl rabbitmqadmin from rabbitmq" [High,Triaged] | 13:56 |
| *** jpena|lunch is now known as jpena | 13:56 | |
| jistr | i'm really perplexed how come we hit different bugs with rabbitmq... | 13:56 |
| hewbrocca | #erlang #filmat11 | 13:56 |
| jistr | this one looks like it shouldn't be upgrade specific, and also i'd expect it to be 100% reproducible | 13:56 |
| *** mburned_out is now known as mburned | 13:57 | |
| jistr | if the puppet container, which should generate configs for rabbitmq container, tries to connect to rabbitmq, which wasn't started yet because it has to wait on the configs to be generated by puppet... | 13:58 |
| hamzy | I'm seeing a rabbit problem when installing newton: "unable to connect to epmd (port 4369) on overcloud-controller-0: timeout (timed out)" | 13:58 |
| flaper87 | skramaja: that's actually and open task that we should start looking into | 13:58 |
| flaper87 | skramaja: we used to have openvswitch containerized and then we didn't but it's something that we want to do | 13:58 |
| flaper87 | skramaja: would you be interested in working on it? | 13:58 |
| hamzy | does anyone know where rabbit is configured to use overcloud-controller-0.localdomain or overcloud-controller-0.internalapi.localdomain? | 13:59 |
| *** dprince has joined #tripleo | 13:59 | |
| *** mburned is now known as mburned_out | 14:00 | |
| *** prateek_ has quit IRC | 14:00 | |
| skramaja | flaper87: i am looking in to DPDK containarizations, for which there is WIP kolla image patches. we are just started to look in to it to containarize DPDK as standalone. i got to know that neutron team has started looking in to ovs (just started). | 14:01 |
| *** mburned_out is now known as mburned | 14:01 | |
| *** rlandy is now known as rlandy|brb | 14:01 | |
| flaper87 | skramaja: gotcha, cool | 14:01 |
| skramaja | flaper87: i will start a mail thread to dicuss on it further. | 14:01 |
| *** links has quit IRC | 14:02 | |
| flaper87 | please do, also, please send to openstack-dev | 14:02 |
| *** chlong has joined #tripleo | 14:02 | |
| skramaja | ok | 14:02 |
| mandre | jistr: ack, I'm trying to reproduce locally | 14:03 |
| *** masco has quit IRC | 14:04 | |
| *** noslzzp has joined #tripleo | 14:04 | |
| *** garyk has quit IRC | 14:06 | |
| *** janki has quit IRC | 14:06 | |
| *** salmankhan1 has joined #tripleo | 14:06 | |
| *** salmankhan has quit IRC | 14:07 | |
| *** salmankhan1 is now known as salmankhan | 14:07 | |
| *** yolanda has quit IRC | 14:08 | |
| *** yolanda has joined #tripleo | 14:08 | |
| *** eck` is now known as eck`gone | 14:09 | |
| *** agurenko has quit IRC | 14:10 | |
| jistr | mandre: i'm also trying to reproduce it on a fresh deployment now. I have an idea how to fix it... basically we're trying to "download" the rabbitmqadmin script from rabbitmq, but even if we succeeded, it's worthless as the script would just go away together with the ephemeral puppet container. So we might just as well disable it. | 14:11 |
| *** skramaja has quit IRC | 14:12 | |
| *** eck`gone is now known as eck` | 14:13 | |
| *** salmankhan has quit IRC | 14:13 | |
| jprovazn | hi, any idea why overcloud validation is looking for node with profile None if the corresponding flavor has profile "ceph"? http://paste.openstack.org/show/607162/ | 14:13 |
| *** salmankhan has joined #tripleo | 14:14 | |
| mandre | jistr: right, I don't understand why we're just seing this issue now | 14:15 |
| *** zoli|afk is now known as zoli|wfh | 14:16 | |
| *** morazi has joined #tripleo | 14:16 | |
| *** flepied has quit IRC | 14:16 | |
| *** salmankhan1 has joined #tripleo | 14:16 | |
| *** salmankhan has quit IRC | 14:18 | |
| *** salmankhan1 is now known as salmankhan | 14:18 | |
| *** pradk has joined #tripleo | 14:18 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Fix containerized RabbitMQ service deployment https://review.openstack.org/458051 | 14:20 |
| *** salmankhan1 has joined #tripleo | 14:26 | |
| *** salmankhan has quit IRC | 14:27 | |
| *** salmankhan1 is now known as salmankhan | 14:27 | |
| *** rlandy|brb is now known as rlandy | 14:28 | |
| *** Goneri has quit IRC | 14:30 | |
| *** dsariel has quit IRC | 14:31 | |
| yolanda | EmilienM, i filed the bug but got a "won't fix" https://bugs.launchpad.net/bugs/1684125 | 14:32 |
| openstack | Launchpad bug 1684125 in tripleo "L->M upgrade fails with invalid gnocchi_internal_url error" [Undecided,Won't fix] | 14:32 |
| yolanda | although l is eol, osp8 is not. Shall i i filed it on bugzilla instead? | 14:32 |
| EmilienM | yolanda: I saw your message, I was in a meeting until now | 14:32 |
| EmilienM | yolanda: yes, libery / mitaka are eol now. I think we should use bugzilla on this one | 14:32 |
| openstackgerrit | Luke Hinds proposed openstack/tripleo-heat-templates master: [WIP] Implements management of `/etc/login.defs` https://review.openstack.org/457985 | 14:37 |
| *** kjw3 has joined #tripleo | 14:37 | |
| *** yolanda has quit IRC | 14:38 | |
| *** yolanda has joined #tripleo | 14:39 | |
| *** anshul has quit IRC | 14:44 | |
| *** hjensas has joined #tripleo | 14:47 | |
| flaper87 | gfidente: d0ugal how would you feel about using yaml here? https://github.com/d0ugal/mistral-ansible-actions/blob/master/mistral_ansible_actions.py#L53 I mean, since it can take yaml and we have yaml everywhere else in tripleo, I think it'd make more sense | 14:49 |
| bogdando | mandre, shardy: jistr: I have an idea how to fix the rabbitmqadmin chicken and egg bug, but first I have to know if we could have different config_settings for init container vs role_data's one? | 14:56 |
| *** flepied has joined #tripleo | 14:58 | |
| bogdando | the idea is to run init container with 'admin_enable: true' and the puppet_config - with a false | 14:58 |
| bogdando | but this may not work, IIRC, the management plugin must be set up for each node as well | 14:59 |
| shardy | bogdando: you could do a map_replace or something in the service template perhaps | 14:59 |
| shardy | bogdando: or map_merge with a admin_enable: false map | 14:59 |
| *** mdnadeem has quit IRC | 15:00 | |
| jistr | bogdando: yes i think we could. FWIW i have a patch ready that sets rabbitmq::admin_enable to false, i can just submit it. Not sure if that was your idea? | 15:00 |
| *** karthiks has joined #tripleo | 15:00 | |
| bogdando | shardy: so can I refer to the config_settings from puppet_config ? | 15:00 |
| bogdando | jistr: yes, a very similar to that | 15:00 |
| bogdando | jistr: could you share? I need that for my local testing of logs | 15:00 |
| bogdando | cuz I have the same blocker now) | 15:01 |
| *** karthiks has quit IRC | 15:01 | |
| *** rcernin has quit IRC | 15:03 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates master: Containers: disable downloading rabbitmqadmin https://review.openstack.org/458119 | 15:04 |
| jistr | bogdando: ^^ i'll set it to WIP as we need to test it first | 15:04 |
| d0ugal | flaper87: that would be fine with me | 15:05 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates master: Containers: disable downloading rabbitmqadmin https://review.openstack.org/458119 | 15:05 |
| jistr | bogdando: btw any chance you hit that bug locally too? https://bugs.launchpad.net/tripleo/+bug/1684104 | 15:06 |
| openstack | Launchpad bug 1684104 in tripleo "container puppet -- fails to curl rabbitmqadmin from rabbitmq" [High,Triaged] | 15:06 |
| bogdando | jistr: exactly that one | 15:06 |
| jistr | bogdando: i only hit it on upgrade envs so far, and mandre didn't hit it on his fresh deployment. So i'm wondering what's happening. | 15:06 |
| bogdando | I have it at fresh undercloud | 15:06 |
| bogdando | but doing reapply deploys on top | 15:07 |
| openstackgerrit | Feng Pan proposed openstack-infra/tripleo-ci master: DO NOT MERGE: Testing TLS everywhere for etcd https://review.openstack.org/455309 | 15:07 |
| d0ugal | flaper87: I'm on PTO for the rest of the week and Monday - so added you and gfidente as collaborators if you want to merge anything while I'm out. | 15:07 |
| bogdando | jistr: but I have it with https://review.openstack.org/#/c/458051/ and https://review.openstack.org/#/c/457984/ as well | 15:07 |
| bogdando | jistr: well, here is my dev version https://github.com/bogdando/tripleo-heat-templates/blob/156bc3ec78e53dc77e26e0c1dfd5309328e44517/docker/services/rabbitmq.yaml#L31 of it. The passwords thing was too complicated to me, so I applied that w/a... | 15:09 |
| shardy | bogdando: another workarounds is to add RabbitCookie to tripleo-undercloud-passwords.yaml | 15:11 |
| bogdando | so this removed all of the obstacles for me to get into the https://bugs.launchpad.net/tripleo/+bug/1684104 :) | 15:11 |
| openstack | Launchpad bug 1684104 in tripleo "container puppet -- fails to curl rabbitmqadmin from rabbitmq" [High,Triaged] | 15:11 |
| shardy | which is basically what the patch I'm working on does | 15:11 |
| jistr | bogdando: i just wonder, how come the rabbitmq deployment used to work, and we started hitting all these errors all of a sudden... what changed... | 15:11 |
| bogdando | shardy: thanks! nice to know, I did that I understood :) | 15:11 |
| bogdando | jistr: yeah, that's always the question. How on earth the thing have been passing CI? :D | 15:12 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/ocata: Decouple Swift ringbuilding logic https://review.openstack.org/455669 | 15:12 |
| *** itzikb has joined #tripleo | 15:12 | |
| bogdando | jistr: nothing changes it seems | 15:12 |
| flaper87 | d0ugal: thanks :) | 15:13 |
| bogdando | has changed* | 15:13 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Add --skip-deploy-identifier https://review.openstack.org/452223 | 15:13 |
| openstackgerrit | Rob Crittenden proposed openstack/tripleo-quickstart-extras master: Increase default novajoin read timeout to 20 seconds https://review.openstack.org/458130 | 15:15 |
| bogdando | jistr: does ovb containers deploy overcloud with pacemaker? | 15:15 |
| bogdando | cuz if it does, then the rabbit tht code is untested | 15:15 |
| bogdando | (for containers) | 15:16 |
| jistr | dprince ^ might know more about that... i've been looking at multinode pretty much exclusively | 15:16 |
| bogdando | so this can be an answer to why CI passes with broken docker/service/rabbitmq.yaml | 15:17 |
| jistr | it should run w/o pacemaker https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/quickstart/ovb-containers-oooq.yml#L7 | 15:17 |
| jistr | also i think mandre deployed w/o pacemaker when testing, and didn't hit this bug | 15:18 |
| jistr | strange days | 15:18 |
| bogdando | indeed | 15:18 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements https://review.openstack.org/456030 | 15:18 |
| gfidente | d0ugal w00t thanks | 15:19 |
| *** masco has joined #tripleo | 15:22 | |
| *** iranzo has quit IRC | 15:22 | |
| flaper87 | gfidente: did you see my question above ? | 15:23 |
| dprince | bogdando: no pacemaker for the OVB containers job yet | 15:23 |
| dprince | bogdando: but soon, hopefully | 15:23 |
| dprince | bogdando: the containers HA integration work is ongoing now https://etherpad.openstack.org/p/tripleo-containers-ha | 15:24 |
| openstackgerrit | Rob Crittenden proposed openstack/tripleo-quickstart-extras master: Increase default novajoin read timeout to 20 seconds https://review.openstack.org/458130 | 15:24 |
| bogdando | dprince: thanks! | 15:24 |
| marios | chem: we good to land this owalsh ? https://review.openstack.org/#/c/457965/3 | 15:25 |
| marios | chem: maybe we can get the stable/ocata done during the US day ;) | 15:26 |
| marios | chem: we will force gfidente to stay up all night he's totally fine with that | 15:26 |
| gfidente | flaper87 taking yaml is fine with me but ansible itself only takes strings or json as extra-vars | 15:27 |
| *** pcaruana has quit IRC | 15:28 | |
| jaosorior | Hey guys, could someone check this commit out https://review.openstack.org/#/c/450726/ ? | 15:28 |
| *** yolanda has quit IRC | 15:28 | |
| flaper87 | gfidente: it also takes yaml | 15:28 |
| gfidente | flaper87 extra-vars ? | 15:28 |
| flaper87 | gfidente: http://docs.ansible.com/ansible/playbooks_variables.html#passing-variables-on-the-command-line | 15:29 |
| flaper87 | gfidente: "Also as of Ansible 1.3, extra vars can be formatted as YAML, either on the command line or in a file as above." | 15:29 |
| gfidente | as of ansible 1.3 ! | 15:29 |
| gfidente | ahahah +1 :) | 15:29 |
| *** yolanda has joined #tripleo | 15:29 | |
| mandre | jistr, bogdando: I have the best undercloud, it's amazing | 15:32 |
| mandre | your underclouds are not even underclouds, they're fake | 15:32 |
| bogdando | mandre: \o/ | 15:32 |
| marios | the one true undercloud? | 15:32 |
| *** fzdarsky has joined #tripleo | 15:33 | |
| * marios grabs his coat | 15:33 | |
| mandre | marios: there can be only one :) | 15:33 |
| *** atheurer has joined #tripleo | 15:35 | |
| jistr | mandre: lol | 15:35 |
| gfidente | marios you tell me what I need to monitor and which command to give when it's red | 15:35 |
| jistr | mandre: i know one undercloud and it's truly the best one, there are no underclouds like that | 15:36 |
| gfidente | we'll see if from the best undercloud the best overcloud will be deployed | 15:37 |
| jistr | that's certain, nobody questions that. I know many people and nobody questions that. | 15:38 |
| * jistr stops :D | 15:39 | |
| jaosorior | bandini, bnemec: are we using the haproxy stats socket for anything? | 15:39 |
| bandini | jaosorior: I was asked to add the unix domain one because qe wanted to use it to collect infos/stats | 15:40 |
| bnemec | jaosorior: We aren't, but users do. | 15:40 |
| bandini | not entirely sure how much they ended up using it though | 15:40 |
| jaosorior | bandini, bnemec: thanks | 15:41 |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-quickstart-extras master: Add new role for tripleo-validations https://review.openstack.org/403576 | 15:43 |
| *** itzikb has quit IRC | 15:44 | |
| *** mhenkel_ has quit IRC | 15:47 | |
| *** dparkes has quit IRC | 15:48 | |
| *** shardy has quit IRC | 15:48 | |
| *** cylopez has quit IRC | 15:49 | |
| *** shardy has joined #tripleo | 15:49 | |
| *** aufi has quit IRC | 15:52 | |
| *** shardy has quit IRC | 15:53 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Mount individual hostpath logs on /var/log https://review.openstack.org/442603 | 15:53 |
| *** yamahata has joined #tripleo | 15:54 | |
| *** mhenkel_ has joined #tripleo | 15:55 | |
| rook | larsks: https://review.openstack.org/#/c/456760/7 <-- enable FluentD logging from oslo.log | 15:56 |
| rook | I have this working in a ocata deployment (that I just blew away :D ) | 15:57 |
| *** jlinkes has quit IRC | 15:59 | |
| *** tesseract has quit IRC | 15:59 | |
| *** atheurer has quit IRC | 16:01 | |
| *** dparkes has joined #tripleo | 16:03 | |
| pradk | can i get some reviews on https://review.openstack.org/#/c/456385/ | 16:03 |
| *** lmiccini has quit IRC | 16:03 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates master: Add docker templates to configure Ironic inspector https://review.openstack.org/457822 | 16:06 |
| *** lucasagomes is now known as lucas-afk | 16:08 | |
| *** dmk0202 has quit IRC | 16:09 | |
| *** atheurer has joined #tripleo | 16:12 | |
| *** jcoufal has joined #tripleo | 16:14 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Containerize Redis service https://review.openstack.org/442151 | 16:17 |
| openstackgerrit | Merged openstack/instack-undercloud master: Migrate plans from Mistral to Swift https://review.openstack.org/456889 | 16:18 |
| openstackgerrit | Michael Bayer proposed openstack/tripleo-quickstart-extras master: Disambiguate between URL and host for devmode targets https://review.openstack.org/458153 | 16:20 |
| cschwede | EmilienM: re https://review.openstack.org/#/c/456382/ - could you please +A that one? CI is all green on it | 16:20 |
| chem | EmilienM: should we open a launchpad bug or is it irrelevant http://logs.openstack.org/82/458082/1/check/gate-tripleo-ci-centos-7-multinode-upgrades-nv/7922cf5/console.html#_2017-04-19_16_13_18_003645 ? | 16:21 |
| chem | EmilienM: hum ... the question is more, are we expecting gate-tripleo-ci-centos-7-multinode-upgrades-nv to work ? | 16:22 |
| *** Goneri has joined #tripleo | 16:22 | |
| mandre | bogdando: my undercloud is so superior to the other underclouds it can recover from https://bugs.launchpad.net/tripleo/+bug/1684104 | 16:23 |
| openstack | Launchpad bug 1684104 in tripleo "container puppet -- fails to curl rabbitmqadmin from rabbitmq" [High,Triaged] | 16:23 |
| mandre | bogdando: joking apart, I can see the error in my deploy logs but my deployment went through | 16:24 |
| mandre | bogdando: your real issue must be something else | 16:24 |
| bogdando | mandre: hard to say why I have had it | 16:24 |
| bogdando | but w/a works | 16:24 |
| openstackgerrit | Pradeep Kilambi proposed openstack/instack-undercloud master: Disable Telemetry services on undercloud by default https://review.openstack.org/458155 | 16:25 |
| bogdando | mandre: it may be that racy bug | 16:25 |
| bogdando | a tricky thing to reproduce | 16:25 |
| mandre | bogdando: I see the error in my puppet logs, but it looks to me it's a non critical error and the deployment goes on with its life | 16:26 |
| mandre | bogdando: you must have another error that caused your deployment to fail | 16:27 |
| bogdando | mandre: if it applies to the case jistr submitted as a bug, we could set it low then | 16:28 |
| jistr | mandre, bogdando: yea it looks like mandre is right even considering upgrades CI. The error which was fatal was probably during configuring nova placement service. | 16:28 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Configure tempest if it's going to run https://review.openstack.org/458158 | 16:28 |
| *** hewbrocca is now known as hewbrocca_afk | 16:29 | |
| jistr | mandre, bogdando: look for "ERROR configuring" in the logs, that should list puppet containers which actually had fatal errors | 16:29 |
| *** dprince has quit IRC | 16:30 | |
| jistr | maybe that's the libvirt error | 16:31 |
| *** jpich has quit IRC | 16:31 | |
| jistr | but it's quite hard to discern what puppet errors relate to what container, unless i'm missing something | 16:31 |
| *** jcoufal_ has joined #tripleo | 16:31 | |
| jistr | yea most probably it's the libvirt bug | 16:33 |
| *** karimb has quit IRC | 16:34 | |
| *** jcoufal has quit IRC | 16:35 | |
| *** suuuper has quit IRC | 16:35 | |
| * jistr off for today, will continue the hunt tomorrow | 16:35 | |
| *** yamahata has quit IRC | 16:36 | |
| *** cylopez has joined #tripleo | 16:37 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Mount individual hostpath logs on /var/log https://review.openstack.org/442603 | 16:39 |
| trozet | mwhahaha: hi | 16:43 |
| mwhahaha | trozet: hi2u fine sir | 16:43 |
| trozet | mwhahaha: :) is there some problem with the upgrades scenario 002 job? it seems to keep failing for https://review.openstack.org/#/c/457541/ | 16:44 |
| mwhahaha | trozet: i think they are timing out | 16:44 |
| trozet | mwhahaha: from the log ./toci_instack_osinfra.sh:1): '[' 143 '!=' 0 ']' | 16:44 |
| trozet | mwhahaha: so should I just keep rechecking, or wait for some fix? | 16:45 |
| *** milan has quit IRC | 16:45 | |
| mwhahaha | trozet: yea the cut off for that i think is 2:30 | 16:45 |
| mwhahaha | trozet: i'm not aware of any current issues so recheck | 16:45 |
| trozet | mwhahaha: ok will do, thanks | 16:45 |
| openstackgerrit | Merged openstack/instack-undercloud master: Set different virtual_router_id for keepalived https://review.openstack.org/457573 | 16:46 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Use custom network-isolation-absolute.yaml in ipv4 net-iso jobs https://review.openstack.org/422297 | 16:46 |
| *** salmankhan has quit IRC | 16:46 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: allow easy access to undercloud's horizon dashboard https://review.openstack.org/444464 | 16:46 |
| *** Goneri has quit IRC | 16:49 | |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: SSHD Service extensions https://review.openstack.org/444622 | 16:53 |
| bogdando | https://review.openstack.org/#/c/442603 works for my undercloud tests | 16:55 |
| *** numans has quit IRC | 16:57 | |
| *** numans has joined #tripleo | 16:58 | |
| *** Goneri has joined #tripleo | 16:58 | |
| *** bogdando has quit IRC | 16:59 | |
| *** newmember has joined #tripleo | 16:59 | |
| *** florianf has quit IRC | 17:01 | |
| *** jcoufal has joined #tripleo | 17:02 | |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: SSHD Service extensions https://review.openstack.org/444622 | 17:03 |
| EmilienM | chem: back from lunch | 17:03 |
| EmilienM | chem: gate-tripleo-ci-centos-7-multinode-upgrades-nv is not supposed to work on master | 17:03 |
| EmilienM | chem: upgrades are broken currently, because of neutron packaging | 17:03 |
| *** ffiore_ has quit IRC | 17:03 | |
| EmilienM | chem: I filed a bug for that: https://bugs.launchpad.net/bugs/1683835 | 17:03 |
| openstack | Launchpad bug 1683835 in tripleo "[ocata to pike upgrades] Neutron db-sync fails: AttributeError: 'module' object has no attribute 'ServicePluginBase'" [High,Triaged] | 17:03 |
| EmilienM | chem: that will be fixed by https://review.rdoproject.org/r/#/q/I01e97ae3a6db40fefdbb85f0e9b4a1a9d3c39600,n,z | 17:04 |
| *** jcoufal__ has joined #tripleo | 17:04 | |
| EmilienM | chem: so no, don't open a bug, it sounds like we cought the same thing | 17:04 |
| EmilienM | cschwede: approved | 17:04 |
| cschwede | EmilienM: thanks a lot :) | 17:05 |
| *** derekh has quit IRC | 17:05 | |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: WIP: restrict nova migration ssh tunnel https://review.openstack.org/458082 | 17:06 |
| *** jcoufal_ has quit IRC | 17:06 | |
| *** newmember has quit IRC | 17:07 | |
| *** newmember has joined #tripleo | 17:07 | |
| *** trown is now known as trown|lunch | 17:08 | |
| *** jcoufal has quit IRC | 17:08 | |
| *** jlinkes has joined #tripleo | 17:08 | |
| bnemec | trozet: The multinode-upgrades jobs are extremely unstable right now, at least according to the output from https://review.openstack.org/#/c/453807/ | 17:09 |
| bnemec | Unfortunately they're voting on the ocata branch. :-( | 17:09 |
| *** chlong has quit IRC | 17:09 | |
| bnemec | I only see a handful of passes in the entire cistatus page with those jobs included, and very few to no jobs passing all of them at once. | 17:10 |
| bnemec | Hence https://review.openstack.org/#/c/453912/ | 17:10 |
| *** dmacpher is now known as dmacpher-afk | 17:10 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Add storage-mgmt network to compute nodes https://review.openstack.org/419049 | 17:12 |
| bnemec | \o/ At last | 17:13 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Fix unqualified Ansible boolean variables https://review.openstack.org/453674 | 17:19 |
| *** toure has quit IRC | 17:21 | |
| yolanda | EmilienM, hitting another one... https://bugzilla.redhat.com/show_bug.cgi?id=1443674 | 17:21 |
| openstack | bugzilla.redhat.com bug 1443674 in openstack-tripleo-heat-templates "TripleO update fails from OSP8 to OSP9" [Unspecified,New] - Assigned to emacchi | 17:21 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Update OVB containers job to use infra docker proxy https://review.openstack.org/454893 | 17:22 |
| trozet | bnemec: oh ok, so you are going to fix it with 453912? | 17:22 |
| rook | Anyone know if there is a way to have Services log normally + have oslo.log work? | 17:23 |
| rook | It seems that it is a all or nothing | 17:23 |
| *** karimb has joined #tripleo | 17:24 | |
| EmilienM | thrash: hey, https://review.openstack.org/#/c/454710/11/mistral/api/wsgi.py is not backward compatible, it breaks deployments | 17:24 |
| *** toure has joined #tripleo | 17:24 | |
| EmilienM | thrash: proof: https://review.openstack.org/#/c/455425/ | 17:24 |
| *** amoralej is now known as amoralej|off | 17:25 | |
| EmilienM | thrash: we're now having a chicken/egg problem where this patch needs a promotion but to have promotion we need this patch | 17:25 |
| thrash | EmilienM: ack. | 17:25 |
| *** chlong has joined #tripleo | 17:25 | |
| EmilienM | thrash: and it usually means, something is not backward compatible | 17:25 |
| cschwede | anyone aware of a timeout of 150 minutes or so on the gate-tripleo-ci-centos-7-scenario001/2-multinode-upgrades jobs? | 17:25 |
| thrash | EmilienM: It should be backwards-compatible... | 17:25 |
| EmilienM | cschwede: upgrade jobs tend to timeout sometimes, I haven't spent time on investigating it... | 17:26 |
| cschwede | EmilienM: seems to be always the case if they are close to 2:40 - they pass if shorter, but abort on different stages if they reach that time. https://review.openstack.org/#/c/456264/1 | 17:26 |
| leifmadsen | Running openstack overcloud deploy, and getting a mistral failure where it seems like the templates are not copied to the directory where mistral wants to read them? | 17:26 |
| EmilienM | thrash: why do we need https://review.openstack.org/#/c/455425/ then? | 17:26 |
| leifmadsen | Deploying templates in the directory /tmp/tripleoclient-R2zMO_/tripleo-heat-templates | 17:26 |
| leifmadsen | that directory never ends up existing, and mistral just bails since it can't find any contents | 17:27 |
| EmilienM | thrash: iiuc, it menas that /usr/lib/python2.7/site-packages/mistral/api/wsgi.py can't be used anymore | 17:27 |
| thrash | EmilienM: it's probably not *completely* necessary. | 17:27 |
| thrash | EmilienM: It can be. | 17:27 |
| thrash | EmilienM: It passed here: http://logs.openstack.org/10/454710/11/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/619faee/ | 17:28 |
| thrash | EmilienM: https://review.openstack.org/#/c/454710/ | 17:28 |
| thrash | EmilienM: which is where it was introduced.. | 17:28 |
| bnemec | trozet: Well, it's not exactly a fix. It's just removing the blocker to merging ocata patches. | 17:28 |
| bnemec | IMHO we shouldn't have jobs voting on stable branches that aren't voting on master, but that's another discussion. | 17:29 |
| trozet | bnemec: well it will fix my -1 :) | 17:29 |
| EmilienM | bnemec: upgrade jobs on master can't pass, for reasons you probably know | 17:29 |
| EmilienM | but having them voting on stable help us to maintain upgrades working | 17:30 |
| thrash | EmilienM: tbh... The puppet changes: https://review.openstack.org/#/c/455425/ are not completely necessary. | 17:30 |
| EmilienM | bnemec: we all know people ignore non voting jobs, it's not new | 17:30 |
| *** yamahata has joined #tripleo | 17:30 | |
| EmilienM | thrash: do we deploy mistral in wsgi in gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet ? | 17:30 |
| thrash | EmilienM: Yes | 17:31 |
| EmilienM | thrash: nope | 17:31 |
| thrash | I used scenario003 as my testbed... | 17:31 |
| thrash | EmilienM: really? | 17:31 |
| EmilienM | let me show you | 17:31 |
| EmilienM | http://logs.openstack.org/25/455425/5/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/950f9ad/logs/undercloud/etc/httpd/conf.d/ | 17:31 |
| EmilienM | no mistral | 17:31 |
| thrash | EmilienM: unless I'm mixing up the puppet-openstack-integration scenario003 and that one. | 17:32 |
| EmilienM | http://logs.openstack.org/25/455425/5/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/950f9ad/logs/subnode-2/etc/httpd/conf.d/ | 17:32 |
| EmilienM | no mistral either | 17:32 |
| EmilienM | thrash: yes, p-o-i uses wsgi | 17:32 |
| *** chem is now known as chem_gone | 17:33 | |
| chem_gone | EmilienM: ack | 17:33 |
| thrash | EmilienM: I'm confused now... https://review.openstack.org/#/c/455425/5 shows gate-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet as passing... | 17:35 |
| EmilienM | thrash: I'm going to workaround now and disable wsgi ni poi | 17:35 |
| EmilienM | thrash: yes it's passing because, again, it's not deploying wsgi | 17:35 |
| thrash | EmilienM: I thought that was the job you were talking about. :) | 17:36 |
| *** jcoufal__ has quit IRC | 17:36 | |
| *** karimb has quit IRC | 17:36 | |
| *** jcoufal has joined #tripleo | 17:37 | |
| *** jmelvin has quit IRC | 17:38 | |
| EmilienM | thrash: https://review.openstack.org/458177 | 17:38 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Run ovb jobs in transition mode https://review.openstack.org/454134 | 17:39 |
| EmilienM | thrash: and the revert with Depends-on: https://review.openstack.org/458179 | 17:40 |
| EmilienM | thrash: it would be great to enable wsgi in tripleo, can we take action on that? | 17:40 |
| thrash | EmilienM: that's where all this was going. :D | 17:40 |
| * EmilienM hugs thrash | 17:41 | |
| thrash | EilienM: is http://logs.openstack.org/25/455425/5/check/gate-puppet-mistral-puppet-beaker-rspec-centos-7/585ad80/console.html#_2017-04-19_12_09_58_339344 related? | 17:41 |
| *** sudipto has quit IRC | 17:43 | |
| *** sudipto has joined #tripleo | 17:43 | |
| *** sudipto has quit IRC | 17:44 | |
| EmilienM | thrash: same problem :) | 17:44 |
| thrash | EmilienM: ack | 17:44 |
| *** gfidente is now known as gfidente|afk | 17:45 | |
| *** tosky has quit IRC | 17:45 | |
| *** rbowen has joined #tripleo | 17:46 | |
| *** jcoufal_ has joined #tripleo | 17:49 | |
| *** jpena is now known as jpena|off | 17:49 | |
| *** jcoufal has quit IRC | 17:51 | |
| *** jaosorior is now known as jaosorior_away | 17:54 | |
| *** jlinkes has quit IRC | 17:56 | |
| *** jkilpatr has quit IRC | 17:57 | |
| *** jkilpatr has joined #tripleo | 18:00 | |
| *** fragatin_ has quit IRC | 18:02 | |
| *** zoli|wfh is now known as zoli|gone | 18:02 | |
| *** gbarros has quit IRC | 18:04 | |
| *** fragatina has joined #tripleo | 18:05 | |
| *** mhenkel_ has quit IRC | 18:05 | |
| *** masco has quit IRC | 18:07 | |
| *** pkovar has quit IRC | 18:07 | |
| *** akrivoka has quit IRC | 18:07 | |
| *** trown|lunch is now known as trown | 18:07 | |
| *** jmelvin has joined #tripleo | 18:09 | |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart master: Bump ansible to 2.3 https://review.openstack.org/458193 | 18:10 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Always rebuild docs venv for tripleo-docs https://review.openstack.org/446724 | 18:14 |
| *** ckyriakidou has quit IRC | 18:15 | |
| openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: Refactor SSHD config to allow both SSHD options and banner/motd to be set https://review.openstack.org/457585 | 18:16 |
| weshay | EmilienM, we probably want chem_gone and matbu to review https://review.openstack.org/#/c/453912 | 18:16 |
| weshay | ya? | 18:16 |
| *** mcornea has quit IRC | 18:17 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Move OVB settings to tripleo-quickstart-extras https://review.openstack.org/448805 | 18:18 |
| EmilienM | weshay: probably? | 18:20 |
| *** chlong has quit IRC | 18:20 | |
| *** mhenkel_ has joined #tripleo | 18:21 | |
| *** mhenkel_ has quit IRC | 18:21 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Parameterize OVB stack flavors https://review.openstack.org/445163 | 18:21 |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: WIP: restrict nova migration ssh tunnel https://review.openstack.org/458082 | 18:23 |
| weshay | jrist, yo.. | 18:25 |
| weshay | jrist, I need some ui help | 18:25 |
| openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: WIP: restrict nova migration ssh tunnel https://review.openstack.org/458077 | 18:25 |
| *** jlinkes has joined #tripleo | 18:28 | |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart master: Stop using CentOS buildlogs CDN https://review.openstack.org/456225 | 18:28 |
| *** dparkes has quit IRC | 18:32 | |
| *** fragatina has quit IRC | 18:34 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Remove root_template and root_environment from capabilities-map.yaml https://review.openstack.org/441052 | 18:38 |
| *** eggmaster has joined #tripleo | 18:39 | |
| *** chlong has joined #tripleo | 18:45 | |
| *** mhenkel_ has joined #tripleo | 18:51 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/tripleo-ci master: Replace "ara generate" by "ara generate html", move report to ara_oooq https://review.openstack.org/448108 | 18:54 |
| *** mhenkel_ has quit IRC | 18:55 | |
| thrash | EmilienM: What job made you think that the mistral side broke promotion? | 18:56 |
| EmilienM | thrash: puppet | 18:56 |
| thrash | EmilienM: The tests I'm doing on my side are working... So I'm really confused.. | 18:56 |
| EmilienM | thrash: it's pretty clear that people who used to deployed in wsgi are now broken | 18:56 |
| EmilienM | p-o-i is deploying Mistral with WSGI | 18:57 |
| thrash | EmilienM: right. | 18:57 |
| EmilienM | let me show the failure | 18:57 |
| EmilienM | gimme a sec | 18:57 |
| * EmilienM clicks on https://ci.centos.org/view/rdo/view/weirdo-pipelines/view/weirdo-promote-puppet/ | 18:57 | |
| thrash | EmilienM: that's a new place for me. :) | 18:58 |
| *** chandankumar is now known as chkumar|sleeping | 18:58 | |
| EmilienM | https://logs.rdoproject.org/weirdo-generic-puppet-openstack-scenario003/1456/weirdo-project/logs/apache/mistral_wsgi_error_ssl.txt.gz | 18:59 |
| EmilienM | Target WSGI script '/var/www/cgi-bin/mistral/app' does not contain WSGI application 'application' | 18:59 |
| *** jlinkes has quit IRC | 18:59 | |
| EmilienM | so puppet-mistral users who deploy Mistral in WSGI will hit that error ^ | 18:59 |
| EmilienM | thrash: ^ | 19:04 |
| thrash | EmilienM: ack | 19:05 |
| thrash | EmilienM: still trying to figure out *why* | 19:05 |
| thrash | EmilienM: it really doesn't make any sense... | 19:05 |
| thrash | EmilienM: I'll keep looking at it.. | 19:07 |
| *** yprokule has quit IRC | 19:08 | |
| *** jcoufal_ has quit IRC | 19:09 | |
| *** jcoufal has joined #tripleo | 19:10 | |
| *** thrash is now known as thrash|biab | 19:14 | |
| *** fragatina has joined #tripleo | 19:15 | |
| *** mhenkel_ has joined #tripleo | 19:22 | |
| *** jcoufal has quit IRC | 19:23 | |
| *** jcoufal has joined #tripleo | 19:25 | |
| *** mhenkel_ has quit IRC | 19:27 | |
| *** jkilpatr has quit IRC | 19:27 | |
| *** jkilpatr has joined #tripleo | 19:30 | |
| *** jkilpatr has quit IRC | 19:30 | |
| *** jkilpatr has joined #tripleo | 19:30 | |
| jrist | weshay: talking | 19:35 |
| jrist | weshay: was at the gym | 19:35 |
| jrist | sup? | 19:35 |
| weshay | jrist, trying to access the tripleo-ui from ocata and master | 19:36 |
| weshay | is the right port w/o ssl 3000? | 19:36 |
| jrist | ok | 19:36 |
| jrist | w/o no | 19:36 |
| jrist | wait | 19:36 |
| jrist | yes | 19:36 |
| jrist | sorry | 19:36 |
| jrist | w/ssl = ip w/o = 3000 but you might have to configure a tunnel/tripleo_ui_config.js | 19:36 |
| weshay | jrist, does it work w/ elinks? | 19:37 |
| jrist | highly doubt it. but maybe :) | 19:37 |
| jrist | let me see! | 19:37 |
| jrist | installing elinks | 19:37 |
| weshay | [stack@undercloud ~]$ netstat -an | grep 3000 | 19:38 |
| weshay | tcp 0 0 192.168.24.1:3000 0.0.0.0:* LISTEN | 19:38 |
| weshay | tcp 0 0 192.168.24.1:3000 192.168.24.1:40040 SYN_RECV | 19:38 |
| weshay | tcp 0 0 192.168.24.3:3000 0.0.0.0:* LISTEN | 19:38 |
| jrist | weshay: yeah, I don't think it works with elinks :) | 19:38 |
| jrist | but it loads | 19:38 |
| weshay | I'm just trying to see if it's running | 19:39 |
| jrist | yeah | 19:39 |
| mwhahaha | did the docker packaging get updated (and break udnercloud installs)? http://logs.openstack.org/66/457866/2/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-puppet/558cdb5/logs/undercloud/home/jenkins/undercloud_install.log.txt.gz#_2017-04-19_19_13_23 | 19:39 |
| jrist | you should see tripleo top right weshay | 19:39 |
| jrist | with elinks | 19:39 |
| * weshay working on https://review.openstack.org/#/c/444464/ | 19:39 | |
| weshay | hrm | 19:39 |
| jrist | undercloud has horizon? :) | 19:39 |
| weshay | wha? | 19:40 |
| jrist | I have never used the horizon on the undercloud | 19:41 |
| weshay | oh, my review | 19:41 |
| weshay | ya ya | 19:41 |
| weshay | I need to update that | 19:41 |
| jrist | anyway I'm confused | 19:42 |
| weshay | for the install.. all I need is.. enable_ui = true in the undercloud.conf | 19:42 |
| weshay | and run the undercloud-install? | 19:42 |
| jrist | theoretically yes | 19:42 |
| weshay | I'm so confused.. | 19:42 |
| weshay | didn't work on master | 19:42 |
| weshay | trying ocata now | 19:43 |
| jrist | so are you trying to get the tripleo_ui or horizon? | 19:44 |
| weshay | tripleo-ui | 19:44 |
| weshay | haven't tried this in years | 19:44 |
| weshay | lolz | 19:44 |
| weshay | sorry | 19:44 |
| jrist | weshay: ok so are you trying a tunnel? | 19:45 |
| jrist | you might have to edit tripleo_ui_config.js | 19:45 |
| jrist | and you might need a tunnel | 19:45 |
| jrist | if you use SSL you won't need either | 19:45 |
| jrist | is this on baremetal? | 19:45 |
| weshay | virthost | 19:45 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Revert "SSH known_hosts config" https://review.openstack.org/458218 | 19:46 |
| jrist | yeah so you'll need a tunnel | 19:46 |
| weshay | cp config/tripleo_ui_config.js.sample dist/tripleo_ui_config.js ? | 19:46 |
| owalsh | bnemec: what's the problem? ^^^ | 19:46 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Revert "Run token flush cron job hourly by default" https://review.openstack.org/458219 | 19:46 |
| jrist | weshay: I think just in that dir now | 19:47 |
| jrist | dtrainor: can you confirm/help? :) | 19:47 |
| bnemec | owalsh: I'm shotgun reverting tht patches until I find the one that caused a 10 minute regression in stack updates. | 19:47 |
| *** dmk0202 has joined #tripleo | 19:47 | |
| dtrainor | 1s | 19:48 |
| bnemec | Oh wait, that's dumb. | 19:48 |
| bnemec | I just need to look at which patch that merged yesterday had a 500+ second ControllerServiceChain update time in it updates job. | 19:48 |
| bnemec | (if any) | 19:48 |
| dtrainor | hey, weshay. the undercloud runs out of apache which only listens on $controller_host | 19:49 |
| mwhahaha | bnemec: do we collect that information anywhere so we can track it? | 19:49 |
| dtrainor | for non-ssl that is, port 3000 | 19:49 |
| bnemec | mwhahaha: It's in postci | 19:49 |
| *** jcoufal has quit IRC | 19:50 | |
| openstackgerrit | Dan Prince proposed openstack/puppet-tripleo master: Switch to overlay driver for storage https://review.openstack.org/451916 | 19:50 |
| dtrainor | when using ssl, haproxy listens on $undercloud_public_host:443 and forwards the connection to the backend httpd host that answers the request on 3000 | 19:50 |
| mwhahaha | bnemec: we should collect that and publish it elsewhere (i hate trying to read postci) | 19:50 |
| weshay | dtrainor, ? | 19:50 |
| *** jcoufal has joined #tripleo | 19:50 | |
| weshay | on $controller_host | 19:50 |
| bnemec | mwhahaha: We used to publish it to graphite, but the resources that have random identifiers caused obscene numbers of metrics to be created. | 19:51 |
| dtrainor | /etc/puppet/hieradata/puppet-stack-config.yaml:controller_host: 192.168.24.1 #local-ipv4 | 19:51 |
| bnemec | We could maybe whitelist some of the resources for graphite though. | 19:51 |
| dtrainor | or i guess you can think of it as the undercloud's *:bind_host | 19:51 |
| bnemec | ControllerDeployment_Step3, ControllerServiceChain, etc. | 19:51 |
| *** lblanchard has quit IRC | 19:51 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates master: Add docker templates to configure Ironic inspector https://review.openstack.org/457822 | 19:52 |
| mwhahaha | bnemec: maybe something that would report per change would be nice. graphite would be good for trend tracking but not sure it would be beneficial when reviewing | 19:52 |
| bnemec | mwhahaha: That's what's in postci. Are you looking for that piece of the log to be split out into a separate file? | 19:53 |
| mwhahaha | bnemec: yea because postci is a giant file of sadness | 19:53 |
| *** mhenkel_ has joined #tripleo | 19:53 | |
| dtrainor | weshay, am I just confusing you more? are you just trying to use lynx to access the UI? the UI won't work, it's js heavy, but you'd at least be able to make the connection to the ui. where are you trying to run lynx from? if you tell me where, then i can give you an example of where it will and will not be able to connect to the ui. | 19:54 |
| dtrainor | or elinks, sorry, whichever - same result | 19:54 |
| weshay | dtrainor, atm.. I'm just using a txt based browser to see if anything returns | 19:54 |
| weshay | but also creating tunnels | 19:54 |
| dtrainor | ok - where are you running that browser from and what does the tunnel look like? that way i can give you a more concise answer | 19:55 |
| weshay | stack@undercloud ~]$ curl localhost:3000 | 19:55 |
| weshay | curl: (7) Failed connect to localhost:3000; Connection refused | 19:55 |
| weshay | [stack@undercloud ~]$ netstat -an | grep 3000 | 19:56 |
| weshay | tcp 0 0 192.168.24.1:3000 0.0.0.0:* LISTEN | 19:56 |
| dtrainor | does localhost resolve to something weird? | 19:56 |
| dtrainor | 192.168.24.1:3000 is the apache service listening for non-ssl connections, it will listen on that address and that port regardless of SSL being used or not | 19:57 |
| *** jprovazn has quit IRC | 19:57 | |
| *** mhenkel_ has quit IRC | 19:58 | |
| weshay | dtrainor, that was it :) http://paste.openstack.org/show/607212/ | 19:58 |
| *** dtrainor has quit IRC | 19:59 | |
| *** dtrainor has joined #tripleo | 19:59 | |
| dtrainor | sorry, my daily laptop lock-up occurred early today | 20:00 |
| dtrainor | <dtrainor> 192.168.24.1:3000 is the apache service listening for non-ssl connections, it will listen on that address and that port regardless of SSL being used or not | 20:00 |
| jrist | weshay: hooray | 20:00 |
| dtrainor | what did i miss? | 20:01 |
| bnemec | Mmm, this might be promotion related: https://www.diffchecker.com/9SDDKxaR | 20:02 |
| bnemec | Two jobs that ran close together, one has the problem and the other doesn't. | 20:02 |
| bnemec | Most obvious difference is that all of the openstack packages were changed. | 20:02 |
| dtrainor | needle in a stack of needles | 20:03 |
| dtrainor | weshay, did you find what you were looking for? sorry, i had gotten disconnected. | 20:03 |
| *** idlemind has joined #tripleo | 20:04 | |
| weshay | dtrainor, ya. think so thanks | 20:04 |
| dtrainor | excellent - please feel free to let me know if you have any other questions, happy to help | 20:04 |
| weshay | thanks man | 20:04 |
| dtrainor | my pleasure | 20:05 |
| *** thrash|biab is now known as thrash | 20:05 | |
| owalsh | mwhahaha: looks like a bad docker package alright, /etc/docker doesn't exist | 20:05 |
| bnemec | mwhahaha: Oh, heh. The deploy times are already being written to a separate log file: http://logs.openstack.org/22/444622/8/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/33c8924/logs/undercloud/var/log/heat-deploy-times.txt.gz | 20:08 |
| mwhahaha | bnemec: ooo handy | 20:08 |
| *** jcoufal has quit IRC | 20:09 | |
| *** mburned is now known as mburned_out | 20:09 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Move OVB settings to tripleo-quickstart-extras https://review.openstack.org/448805 | 20:09 |
| *** mhenkel_ has joined #tripleo | 20:10 | |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud update time" [Critical,Triaged] | 20:10 |
| *** mburned_out is now known as mburned | 20:10 | |
| openstackgerrit | Leif Madsen proposed openstack/tripleo-quickstart master: Work around libvirt fails w/o environment https://review.openstack.org/458223 | 20:12 |
| *** mhenkel_ has quit IRC | 20:14 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-specs master: Deriving TripleO Parameters https://review.openstack.org/423304 | 20:16 |
| pradk | EmilienM, you have any suggestions for this https://review.openstack.org/#/c/458036/ .. we could pass in bootstrap node param to this class and only run if thats true? | 20:17 |
| pradk | if we dont include collector by default, not sure where else we could run | 20:17 |
| *** dsariel has joined #tripleo | 20:18 | |
| mwhahaha | pradk: where does the ceilometer db creation stuff get run? | 20:18 |
| pradk | mwhahaha, collector | 20:18 |
| *** gbarros has joined #tripleo | 20:18 | |
| pradk | mwhahaha, there is no db for ceilo with collector deprecated | 20:18 |
| mwhahaha | pradk: oh right | 20:19 |
| *** mburned has left #tripleo | 20:19 | |
| *** mburned has joined #tripleo | 20:19 | |
| mwhahaha | pradk: yea so other than doing a bootstrap node, you don't really have much choice | 20:19 |
| pradk | yea i thought since this is not doing any db sync really, we dont need that bootstrap check | 20:20 |
| pradk | but if we need to only run this once, thats only way i guess | 20:20 |
| *** karimb has joined #tripleo | 20:23 | |
| *** liverpooler has quit IRC | 20:31 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo master: Move ceilometer upgrade re-run out of collector https://review.openstack.org/458036 | 20:39 |
| *** mhenkel_ has joined #tripleo | 20:41 | |
| openstackgerrit | Michael Bayer proposed openstack/tripleo-quickstart-extras master: Disambiguate between URL and host for devmode targets https://review.openstack.org/458153 | 20:44 |
| *** mhenkel_ has quit IRC | 20:46 | |
| *** paramite has quit IRC | 20:50 | |
| flaper87 | has anyone tried to deploy an undercloud recently with oooq? I'm getting a "timeout priviledges escalation error" on this task: "provision/remote : Ensure XDG_RUNTIME_DIR is set" | 20:50 |
| flaper87 | at least it's consistent | 20:50 |
| mwhahaha | sounds like a sudo timeout issue? | 20:51 |
| flaper87 | mwhahaha: it is, I just don't understand why I'm getting that now | 20:51 |
| flaper87 | the rest of the tasks seem to pass just fine | 20:51 |
| mwhahaha | i did run into problems with that being set actually (not sure if it should still be doing that) but thats another story | 20:51 |
| *** trown is now known as trown|outtypewww | 20:53 | |
| *** mburned is now known as mburned_out | 20:54 | |
| *** mburned_out is now known as mburned | 20:54 | |
| *** gbarros has quit IRC | 21:01 | |
| *** cylopez has quit IRC | 21:04 | |
| *** dprince has joined #tripleo | 21:05 | |
| openstackgerrit | Rajini Ram proposed openstack/puppet-tripleo master: Dell SC: Add secondary DSM support https://review.openstack.org/455386 | 21:08 |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 21:10 |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud update time" [Critical,Triaged] | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstackgerrit | Rajini Ram proposed openstack/puppet-tripleo master: Dell SC: Add secondary DSM support https://review.openstack.org/455386 | 21:10 |
| EmilienM | pradk: i'll look asap | 21:11 |
| openstackgerrit | Rajini Ram proposed openstack/puppet-tripleo master: Dell SC: Add secondary DSM support https://review.openstack.org/455386 | 21:11 |
| *** mhenkel_ has joined #tripleo | 21:13 | |
| *** ansmith has quit IRC | 21:13 | |
| owalsh | dprince: hi, FYI undercloud install is failing in CI with Augeas[docker-daemon.json]: Could not evaluate: Saving failed | 21:13 |
| *** mburned is now known as mburned_out | 21:14 | |
| dprince | owalsh: oh, got a link to the failure? | 21:14 |
| dprince | owalsh: I will have a look | 21:14 |
| owalsh | dprince: http://logs.openstack.org/85/457585/4/check/gate-tripleo-ci-centos-7-nonha-multinode-oooq/78b7b53/logs/undercloud/home/jenkins/undercloud_install.log.txt.gz#_2017-04-19_20_48_50 | 21:14 |
| owalsh | dprince: FYI there are new docker package in the centos repo today, probably related | 21:14 |
| dprince | owalsh: ack, good to know | 21:15 |
| owalsh | dprince: and I don't see /etc/docker, but maybe it's not collected by the CI scripts | 21:15 |
| dprince | owalsh: yeah, its not collected I think | 21:16 |
| *** mhenkel_ has quit IRC | 21:17 | |
| owalsh | dprince: have it here (passed) http://logs.openstack.org/85/457585/3/check/gate-tripleo-ci-centos-7-undercloud-oooq/064da0e/logs/undercloud/etc/ | 21:18 |
| owalsh | but not here (failed) http://logs.openstack.org/85/457585/4/check/gate-tripleo-ci-centos-7-undercloud-oooq/7234f72/logs/undercloud/etc/ | 21:18 |
| *** masterjcool has quit IRC | 21:19 | |
| dprince | owalsh: the new docker-daemon.json file is part of how we configure docker to consume infra mirrors | 21:19 |
| dprince | owalsh: haven't tried the new docker release yet though | 21:20 |
| *** mburned_out is now known as mburned | 21:23 | |
| dprince | owalsh: sure enough. The latest docker package removed that file: http://paste.openstack.org/show/607221/ | 21:25 |
| *** eck` is now known as eck`gone | 21:26 | |
| *** ramishra has quit IRC | 21:27 | |
| *** ramishra has joined #tripleo | 21:29 | |
| dprince | owalsh: https://bugs.launchpad.net/tripleo/+bug/1684297 | 21:29 |
| openstack | Launchpad bug 1684297 in tripleo "Augeas[docker-daemon.json]: Could not evaluate. docker package removed /etc/docker/daemon.json" [Critical,In progress] | 21:29 |
| owalsh | dprince: /etc/docker/daemon.json doesn't existing in docker-common-1.12.6-16.el7.centos.x86_64.rpm | 21:30 |
| owalsh | dprince: it's in docker-common-1.12.6-11.el7.centos.x86_64.rpm | 21:31 |
| *** masterjcool has joined #tripleo | 21:31 | |
| dprince | owalsh: it isn't in docker-common-1.12.6-16.el7.centos.x86_64 (the latest) I think | 21:32 |
| dprince | owalsh: fwiw, we only install 'docker'. So the docker-common package is a dependency | 21:32 |
| owalsh | dprince: yea, it's not in the latest pkg, but was in the previous | 21:33 |
| *** atheurer has quit IRC | 21:34 | |
| *** hjensas has quit IRC | 21:37 | |
| * owalsh had missed the paste with the changelog | 21:37 | |
| *** atheurer has joined #tripleo | 21:38 | |
| *** tobias-fiberdata has quit IRC | 21:39 | |
| EmilienM | stevebaker: merged | 21:39 |
| *** jerrygb has quit IRC | 21:47 | |
| stevebaker | EmilienM: bah, wha? | 21:48 |
| *** cwolferh has quit IRC | 21:50 | |
| EmilienM | stevebaker: the tripleo-common-distgit patch | 21:51 |
| stevebaker | EmilienM: oh, cool. do you think this could merge too? https://review.openstack.org/#/c/453428/ | 21:52 |
| EmilienM | stevebaker: approved | 21:53 |
| stevebaker | \o/ | 21:53 |
| stevebaker | EmilienM: I'm proposing the changes to import paunch into tripleo btw | 21:54 |
| EmilienM | cool | 21:54 |
| *** jmelvin has quit IRC | 21:55 | |
| stevebaker | EmilienM: also, shall I propose a tripleo-common release, and on what git ref? http://lists.openstack.org/pipermail/openstack-dev/2017-April/115678.html | 21:57 |
| openstackgerrit | Dan Prince proposed openstack/puppet-tripleo master: Ensure /etc/docker/daemon.json https://review.openstack.org/458253 | 21:57 |
| dprince | owalsh: ^^^ | 21:57 |
| dprince | EmilienM: ^^^ | 21:57 |
| *** abishop has quit IRC | 21:57 | |
| EmilienM | dprince: ack, I'll get it tonight | 21:57 |
| EmilienM | stevebaker: yeah, sure. Sounds like it was agreed with Doug, right? Which tag you mean? | 21:58 |
| owalsh | dprince: ack | 21:58 |
| openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: Refactor SSHD config to allow both SSHD options and banner/motd to be set https://review.openstack.org/457585 | 21:59 |
| openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: WIP: restrict nova migration ssh tunnel https://review.openstack.org/458077 | 22:02 |
| EmilienM | dprince: wait | 22:03 |
| EmilienM | dprince: to create this file, we need /etc/docker, right? So we need a dependency on the packge I think | 22:03 |
| dprince | EmilienM: The file no longer exists in the package | 22:04 |
| mwhahaha | dprince: no to create the directory | 22:04 |
| dprince | EmilienM: see the LP bug which has the changelog | 22:04 |
| dprince | EmilienM: oh, yes! | 22:04 |
| dprince | EmilienM: I can add that dep in there, sure | 22:04 |
| EmilienM | good | 22:04 |
| mwhahaha | but i think since we package is already in that class, i'm not sure the requirement is exactly an necessity | 22:04 |
| EmilienM | mwhahaha: because puppet4 and ordering? | 22:05 |
| mwhahaha | since i think it would do it in order (but i can never remember the ordering rules) | 22:05 |
| mwhahaha | yea if it's in the file it's top down unless otherwise specified | 22:05 |
| EmilienM | yeah tbh, I still use require stuff | 22:05 |
| mwhahaha | but it doesn't hert | 22:05 |
| mwhahaha | or hurt | 22:05 |
| *** Goneri has quit IRC | 22:05 | |
| openstackgerrit | Dan Prince proposed openstack/puppet-tripleo master: Ensure /etc/docker/daemon.json https://review.openstack.org/458253 | 22:06 |
| stevebaker | EmilienM: we just need to decide whether to release 7.0.0 from master or 7.0.0.0b1 (I would prefer master) | 22:06 |
| *** chlong has quit IRC | 22:07 | |
| mwhahaha | dprince: alignment | 22:07 |
| EmilienM | stevebaker: go for master... | 22:08 |
| EmilienM | stevebaker: but just for you | 22:08 |
| stevebaker | :D | 22:08 |
| dprince | EmilienM, mwhahaha: I added the dependency on the Package | 22:09 |
| mwhahaha | dprince: yea i commented, arrow alignment | 22:09 |
| EmilienM | dprince: can you fix alignment? lint will fail | 22:09 |
| dprince | on it | 22:09 |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684297 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud update time" [Critical,Triaged] | 22:10 |
| openstack | Launchpad bug 1684297 in tripleo "Augeas[docker-daemon.json]: Could not evaluate. docker package removed /etc/docker/daemon.json" [Critical,In progress] - Assigned to Dan Prince (dan-prince) | 22:10 |
| dprince | boom | 22:10 |
| openstackgerrit | Dan Prince proposed openstack/puppet-tripleo master: Ensure /etc/docker/daemon.json https://review.openstack.org/458253 | 22:10 |
| *** newmember has quit IRC | 22:11 | |
| *** newmember has joined #tripleo | 22:11 | |
| *** Goneri has joined #tripleo | 22:12 | |
| EmilienM | dprince: thx | 22:12 |
| *** dmk0202 has quit IRC | 22:17 | |
| owalsh | dprince: one more time, lint failed on the "s | 22:18 |
| * dprince just needs to run puppet lint locally :/ | 22:19 | |
| * mwhahaha <3s his lint in vim | 22:19 | |
| *** cwolferh has joined #tripleo | 22:19 | |
| *** Goneri has quit IRC | 22:20 | |
| openstackgerrit | Dan Prince proposed openstack/puppet-tripleo master: Ensure /etc/docker/daemon.json https://review.openstack.org/458253 | 22:20 |
| *** jkilpatr has quit IRC | 22:21 | |
| *** colonwq has quit IRC | 22:21 | |
| *** mhenkel_ has joined #tripleo | 22:22 | |
| *** colonwq has joined #tripleo | 22:26 | |
| *** mhenkel_ has quit IRC | 22:27 | |
| openstackgerrit | Alex Schultz proposed openstack/puppet-tripleo master: Cover gnocchi api step 4 and 5 https://review.openstack.org/456376 | 22:27 |
| openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: Refactor SSHD config to allow both SSHD options and banner/motd to be set https://review.openstack.org/457585 | 22:30 |
| openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: WIP: restrict nova migration ssh tunnel https://review.openstack.org/458077 | 22:31 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/ocata: Ensure we configure ssl.conf https://review.openstack.org/458033 | 22:34 |
| *** limao has joined #tripleo | 22:37 | |
| *** Erming__ has joined #tripleo | 22:39 | |
| *** Erming_ has quit IRC | 22:41 | |
| EmilienM | stevebaker: in case I forgot to approve it tonight, please do it before the end of your day: https://review.openstack.org/#/c/458253/ | 22:42 |
| EmilienM | (if CI pass) | 22:42 |
| *** jerrygb has joined #tripleo | 22:49 | |
| *** jerrygb has quit IRC | 22:53 | |
| *** mhenkel_ has joined #tripleo | 22:53 | |
| *** morazi has quit IRC | 22:55 | |
| *** mburned is now known as mburned_out | 22:56 | |
| *** jkilpatr has joined #tripleo | 22:57 | |
| *** mhenkel_ has quit IRC | 22:58 | |
| *** dprince has quit IRC | 22:59 | |
| *** gfidente|afk has quit IRC | 23:04 | |
| *** jkilpatr has quit IRC | 23:05 | |
| *** pradk has quit IRC | 23:06 | |
| stevebaker | EmilienM: will do | 23:08 |
| *** noslzzp has quit IRC | 23:10 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684272 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1684297 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1684272 in tripleo "10 minute increase in overcloud update time" [Critical,Triaged] | 23:10 |
| openstack | Launchpad bug 1684297 in tripleo "Augeas[docker-daemon.json]: Could not evaluate. docker package removed /etc/docker/daemon.json" [Critical,In progress] - Assigned to Dan Prince (dan-prince) | 23:10 |
| *** dbecker has quit IRC | 23:11 | |
| *** newmember has quit IRC | 23:11 | |
| *** dbecker has joined #tripleo | 23:11 | |
| *** rlandy is now known as rlandy|bbl | 23:12 | |
| *** colonwq has quit IRC | 23:13 | |
| *** limao has quit IRC | 23:26 | |
| *** mburned_out is now known as mburned | 23:32 | |
| *** jkilpatr has joined #tripleo | 23:38 | |
| *** ansmith has joined #tripleo | 23:44 | |
| *** kjw3 has quit IRC | 23:45 | |
| *** dsneddon has quit IRC | 23:50 | |
| *** dsneddon has joined #tripleo | 23:52 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!