| *** emagana has quit IRC | 00:00 | |
| *** laron has joined #openstack-operators | 00:06 | |
| *** VW has joined #openstack-operators | 00:08 | |
| *** krobzaur has joined #openstack-operators | 00:09 | |
| *** elo has joined #openstack-operators | 00:09 | |
| *** laron has quit IRC | 00:10 | |
| *** lhcheng_ has joined #openstack-operators | 00:13 | |
| *** lhcheng has quit IRC | 00:14 | |
| *** VW has quit IRC | 00:14 | |
| mgagne | So I'm reading about federation. As a public cloud provider, how can I benefit from it? As a SP (service provider), I need to map users from a trusted IdP (IDentity Provider) to a local keystone group and assign roles to it. | 00:22 |
|---|---|---|
| mgagne | so far, is my understanding right? | 00:23 |
| *** sanjayu has quit IRC | 00:55 | |
| *** stanchan has quit IRC | 01:08 | |
| *** VW has joined #openstack-operators | 01:15 | |
| *** VW has quit IRC | 01:19 | |
| *** kencjohnston has joined #openstack-operators | 01:26 | |
| *** elo has quit IRC | 01:34 | |
| *** SimonChung has quit IRC | 01:35 | |
| *** markvoelker has joined #openstack-operators | 01:37 | |
| *** shimura_ken has joined #openstack-operators | 01:42 | |
| *** krobzaur has quit IRC | 01:43 | |
| *** gyee has quit IRC | 01:55 | |
| *** mriedem_away is now known as mriedem | 01:59 | |
| *** jmccrory has joined #openstack-operators | 02:05 | |
| *** fawadkhaliq has joined #openstack-operators | 02:05 | |
| *** mriedem has quit IRC | 02:07 | |
| *** alejandrito has joined #openstack-operators | 02:08 | |
| *** alejandrito has quit IRC | 02:08 | |
| *** alejandrito has joined #openstack-operators | 02:08 | |
| lhcheng_ | mgagne: yes, the mapping is made to the local keystone group. | 02:09 |
| lhcheng_ | mgagne: one use case for cloud provider is to allow support people to federate into their customer's cloud | 02:10 |
| lhcheng_ | so they can have access to troubleshoot issue | 02:10 |
| mgagne | lhcheng_: it raises more questions than it answers. Doesn't it mean I would have to create a mapping for each customers so one doesn't end up being granted access to all projects? | 02:10 |
| mgagne | lhcheng_: I mean, unless it's self-served, I don't see how a public cloud benefits from it without huge operational overhead. | 02:11 |
| *** krobzaur has joined #openstack-operators | 02:12 | |
| lhcheng_ | mgagne: I haven't had much experience with the mapping engine, perhaps you can ask in the keystone room. But I got your point, it does involve a lot of work to maps each users. | 02:14 |
| mgagne | lhcheng_: thanks, I'm starting to understand | 02:14 |
| lhcheng_ | maybe you could create a mapping of something like %username% to %username%-group | 02:15 |
| *** shimura_ken has quit IRC | 02:16 | |
| *** krobzaur has quit IRC | 02:20 | |
| *** alejandrito has quit IRC | 02:21 | |
| *** armax has joined #openstack-operators | 02:24 | |
| *** alejandrito has joined #openstack-operators | 02:33 | |
| *** bapalm has quit IRC | 03:06 | |
| *** harshs has quit IRC | 03:06 | |
| *** harshs has joined #openstack-operators | 03:16 | |
| *** VW has joined #openstack-operators | 03:17 | |
| *** albertom has quit IRC | 03:18 | |
| *** VW has quit IRC | 03:22 | |
| *** albertom has joined #openstack-operators | 03:23 | |
| *** lhcheng_ has quit IRC | 03:23 | |
| *** Marga__ has quit IRC | 03:24 | |
| *** Marga_ has joined #openstack-operators | 03:35 | |
| *** Marga_ has quit IRC | 03:36 | |
| *** harshs has quit IRC | 03:46 | |
| *** VW has joined #openstack-operators | 03:51 | |
| *** VW has quit IRC | 03:53 | |
| *** VW has joined #openstack-operators | 03:54 | |
| *** kencjohnston has quit IRC | 03:59 | |
| *** VW has quit IRC | 04:07 | |
| *** alejandrito has quit IRC | 04:16 | |
| *** lhcheng has joined #openstack-operators | 04:23 | |
| *** laron has joined #openstack-operators | 04:30 | |
| *** laron has quit IRC | 04:34 | |
| *** zhangjn has joined #openstack-operators | 04:35 | |
| *** zhangjn has quit IRC | 04:45 | |
| *** harshs has joined #openstack-operators | 04:46 | |
| *** lhcheng has quit IRC | 04:48 | |
| *** SimonChung has joined #openstack-operators | 04:49 | |
| *** SimonChung1 has joined #openstack-operators | 04:51 | |
| *** homegrown_ has joined #openstack-operators | 04:53 | |
| *** SimonChung has quit IRC | 04:54 | |
| *** baffle has joined #openstack-operators | 04:55 | |
| *** __nick has joined #openstack-operators | 04:57 | |
| *** cfarquhar_ has joined #openstack-operators | 04:58 | |
| *** wasmum- has joined #openstack-operators | 04:59 | |
| *** ruagair_ has joined #openstack-operators | 04:59 | |
| *** homegrown has quit IRC | 05:00 | |
| *** serverascode has quit IRC | 05:00 | |
| *** _nick has quit IRC | 05:00 | |
| *** codebauss has quit IRC | 05:00 | |
| *** baffle_ has quit IRC | 05:00 | |
| *** ruagair has quit IRC | 05:00 | |
| *** mgagne has quit IRC | 05:00 | |
| *** sbadia has quit IRC | 05:00 | |
| *** wasmum has quit IRC | 05:00 | |
| *** cfarquhar has quit IRC | 05:00 | |
| *** notmyname has quit IRC | 05:00 | |
| *** ruagair_ is now known as ruagair | 05:00 | |
| *** mgagne has joined #openstack-operators | 05:00 | |
| *** elo has joined #openstack-operators | 05:00 | |
| *** elo has quit IRC | 05:02 | |
| *** codebauss has joined #openstack-operators | 05:03 | |
| *** notmyname has joined #openstack-operators | 05:03 | |
| *** serverascode has joined #openstack-operators | 05:05 | |
| *** sbadia has joined #openstack-operators | 05:06 | |
| *** Marga_ has joined #openstack-operators | 05:10 | |
| *** Marga_ has quit IRC | 05:11 | |
| *** Marga_ has joined #openstack-operators | 05:11 | |
| *** B_Smith__ has joined #openstack-operators | 05:21 | |
| *** lhcheng has joined #openstack-operators | 05:22 | |
| *** B_Smith_ has quit IRC | 05:25 | |
| *** elo has joined #openstack-operators | 05:34 | |
| *** elo has quit IRC | 05:35 | |
| *** jmccrory has quit IRC | 05:48 | |
| *** jmccrory has joined #openstack-operators | 05:51 | |
| *** lhcheng has quit IRC | 05:54 | |
| *** aswadr has joined #openstack-operators | 05:54 | |
| *** lhcheng has joined #openstack-operators | 05:55 | |
| *** sanjayu has joined #openstack-operators | 05:57 | |
| *** harshs has quit IRC | 06:11 | |
| *** rcernin has joined #openstack-operators | 06:22 | |
| *** aswadr has quit IRC | 06:55 | |
| *** maishsk has quit IRC | 07:10 | |
| *** bvandenh has joined #openstack-operators | 08:01 | |
| *** maishsk has joined #openstack-operators | 08:01 | |
| *** matrohon has joined #openstack-operators | 08:15 | |
| *** bvandenh has quit IRC | 08:22 | |
| *** egonzalez has joined #openstack-operators | 08:36 | |
| *** liverpooler has joined #openstack-operators | 08:38 | |
| *** bvandenh has joined #openstack-operators | 08:40 | |
| *** liverpoo1er has joined #openstack-operators | 08:41 | |
| *** liverpooler has quit IRC | 08:43 | |
| *** egonzalez has quit IRC | 08:46 | |
| *** egonzalez90 has joined #openstack-operators | 08:47 | |
| *** elemoine has joined #openstack-operators | 08:48 | |
| *** elemoine has quit IRC | 08:53 | |
| *** elemoine has joined #openstack-operators | 08:54 | |
| *** elemoine has quit IRC | 08:57 | |
| *** elemoine has joined #openstack-operators | 08:57 | |
| *** bvandenh has quit IRC | 08:58 | |
| *** elemoine has quit IRC | 08:59 | |
| *** lhcheng has quit IRC | 09:00 | |
| *** elemoine has joined #openstack-operators | 09:01 | |
| *** Marga_ has quit IRC | 09:07 | |
| *** maishsk has quit IRC | 09:08 | |
| *** liverpoo1er has quit IRC | 09:11 | |
| *** elemoine has quit IRC | 09:11 | |
| *** liverpooler has joined #openstack-operators | 09:11 | |
| *** elemoine has joined #openstack-operators | 09:12 | |
| *** Marga_ has joined #openstack-operators | 09:16 | |
| *** cbrown2_ocf has joined #openstack-operators | 09:22 | |
| *** openstackgerrit has quit IRC | 09:31 | |
| *** openstackgerrit has joined #openstack-operators | 09:32 | |
| *** electrofelix has joined #openstack-operators | 09:32 | |
| *** sleinen-AS559 has joined #openstack-operators | 09:41 | |
| *** esker has joined #openstack-operators | 09:49 | |
| *** Marga_ has quit IRC | 09:50 | |
| *** belmoreira has joined #openstack-operators | 09:52 | |
| *** esker has quit IRC | 09:54 | |
| *** esker has joined #openstack-operators | 09:55 | |
| *** esker has quit IRC | 09:55 | |
| *** markvoelker has quit IRC | 10:05 | |
| *** cbrown2ocf has joined #openstack-operators | 10:16 | |
| *** egonzalez90 has quit IRC | 10:18 | |
| *** cbrown2_ocf has quit IRC | 10:19 | |
| *** cbrown2ocf is now known as cbrown2_ocf | 10:19 | |
| *** egonzalez has joined #openstack-operators | 10:20 | |
| *** subscope has joined #openstack-operators | 10:30 | |
| *** openstackgerrit has quit IRC | 10:31 | |
| *** openstackgerrit has joined #openstack-operators | 10:32 | |
| *** lhcheng has joined #openstack-operators | 10:48 | |
| *** subscope has quit IRC | 10:50 | |
| *** lhcheng has quit IRC | 10:54 | |
| *** markvoelker has joined #openstack-operators | 11:06 | |
| *** markvoelker has quit IRC | 11:11 | |
| *** zhangjn has joined #openstack-operators | 11:12 | |
| *** zhangjn has quit IRC | 11:12 | |
| *** zhangjn has joined #openstack-operators | 11:13 | |
| *** zhangjn has quit IRC | 11:14 | |
| *** zhangjn has joined #openstack-operators | 11:14 | |
| *** zhangjn has quit IRC | 11:15 | |
| *** zhangjn has joined #openstack-operators | 11:15 | |
| *** zhangjn has quit IRC | 11:15 | |
| *** zhangjn has joined #openstack-operators | 11:16 | |
| *** zhangjn has quit IRC | 11:16 | |
| *** zhangjn has joined #openstack-operators | 11:17 | |
| *** zhangjn has quit IRC | 11:18 | |
| *** zhangjn has joined #openstack-operators | 11:18 | |
| *** zhangjn has quit IRC | 11:19 | |
| *** zhangjn has joined #openstack-operators | 11:20 | |
| *** zhangjn has quit IRC | 11:20 | |
| *** zhangjn has joined #openstack-operators | 11:21 | |
| *** berendt has joined #openstack-operators | 12:21 | |
| *** elemoine_ has joined #openstack-operators | 12:24 | |
| *** elemoine_ has left #openstack-operators | 12:25 | |
| *** elemoine_ has joined #openstack-operators | 12:25 | |
| *** fawadkhaliq has quit IRC | 12:26 | |
| *** fawadkhaliq has joined #openstack-operators | 12:27 | |
| *** elemoine has quit IRC | 12:30 | |
| *** fawadkhaliq has quit IRC | 12:31 | |
| *** markvoelker has joined #openstack-operators | 12:37 | |
| *** lhcheng has joined #openstack-operators | 12:37 | |
| *** sleinen-AS5591 has joined #openstack-operators | 12:41 | |
| *** sleinen-AS559 has quit IRC | 12:42 | |
| *** markvoelker has quit IRC | 12:42 | |
| *** lhcheng has quit IRC | 12:42 | |
| *** alejandrito has joined #openstack-operators | 12:42 | |
| *** maishsk has joined #openstack-operators | 12:54 | |
| *** bvandenh has joined #openstack-operators | 12:58 | |
| *** lhcheng has joined #openstack-operators | 13:02 | |
| *** lhcheng has quit IRC | 13:07 | |
| *** csoukup has joined #openstack-operators | 13:11 | |
| *** signed8bit has joined #openstack-operators | 13:17 | |
| *** __nick is now known as _nick | 13:17 | |
| *** markvoelker has joined #openstack-operators | 13:27 | |
| *** csoukup has quit IRC | 13:31 | |
| *** elemoine_ has quit IRC | 13:33 | |
| *** elemoine has joined #openstack-operators | 13:34 | |
| *** regXboi has joined #openstack-operators | 13:34 | |
| *** cbrown2_ocf has quit IRC | 13:49 | |
| *** cbrown2_ocf has joined #openstack-operators | 13:51 | |
| *** cbrown2ocf has joined #openstack-operators | 13:52 | |
| *** cbrown2_ocf has quit IRC | 13:55 | |
| *** cbrown2ocf is now known as cbrown2_ocf | 13:55 | |
| *** signed8bit is now known as signed8bit_ZZZzz | 13:56 | |
| *** signed8bit_ZZZzz is now known as signed8bit | 14:00 | |
| *** mdorman has joined #openstack-operators | 14:09 | |
| *** signed8bit is now known as signed8bit_ZZZzz | 14:19 | |
| *** Marga_ has joined #openstack-operators | 14:19 | |
| *** laron has joined #openstack-operators | 14:20 | |
| *** signed8bit_ZZZzz is now known as signed8bit | 14:21 | |
| *** laron has quit IRC | 14:22 | |
| *** laron has joined #openstack-operators | 14:22 | |
| *** sanjayu has quit IRC | 14:26 | |
| *** kencjohnston has joined #openstack-operators | 14:27 | |
| *** liverpooler has quit IRC | 14:29 | |
| *** krobzaur has joined #openstack-operators | 14:32 | |
| *** mriedem has joined #openstack-operators | 14:32 | |
| *** nikhil_k is now known as nikhil | 14:39 | |
| *** krobzaur_ has joined #openstack-operators | 14:40 | |
| *** krobzaur has quit IRC | 14:43 | |
| *** krobzaur_ has quit IRC | 14:45 | |
| *** fawadkhaliq has joined #openstack-operators | 14:48 | |
| *** Marga_ has quit IRC | 14:52 | |
| *** mriedem has quit IRC | 14:55 | |
| *** dminer has joined #openstack-operators | 14:57 | |
| *** laron has quit IRC | 14:57 | |
| *** zhangjn has quit IRC | 14:59 | |
| *** mriedem has joined #openstack-operators | 15:03 | |
| *** laron has joined #openstack-operators | 15:04 | |
| *** ccarmack has joined #openstack-operators | 15:07 | |
| *** ccarmack has left #openstack-operators | 15:07 | |
| *** VW has joined #openstack-operators | 15:14 | |
| *** VW has quit IRC | 15:15 | |
| *** VW has joined #openstack-operators | 15:16 | |
| *** Guest80875 is now known as mfisch | 15:16 | |
| *** mfisch is now known as Guest20594 | 15:17 | |
| *** Guest20594 is now known as mfisch | 15:18 | |
| *** mfisch has quit IRC | 15:18 | |
| *** mfisch has joined #openstack-operators | 15:18 | |
| *** elemoine has quit IRC | 15:27 | |
| *** elemoine has joined #openstack-operators | 15:30 | |
| *** krobzaur_ has joined #openstack-operators | 15:36 | |
| *** elo has joined #openstack-operators | 15:39 | |
| *** armax has quit IRC | 15:49 | |
| *** Piet has quit IRC | 15:50 | |
| *** britthou_ has joined #openstack-operators | 15:51 | |
| *** VW has quit IRC | 15:53 | |
| *** britthouser has quit IRC | 15:53 | |
| *** krobzaur_ has quit IRC | 15:54 | |
| *** krobzaur_ has joined #openstack-operators | 16:03 | |
| *** bvandenh has quit IRC | 16:03 | |
| *** rcernin has quit IRC | 16:04 | |
| *** mriedem has quit IRC | 16:06 | |
| *** SimonChung1 has quit IRC | 16:09 | |
| *** mriedem has joined #openstack-operators | 16:11 | |
| *** belmoreira has quit IRC | 16:13 | |
| *** sleinen-AS559 has joined #openstack-operators | 16:14 | |
| *** sleinen-AS5591 has quit IRC | 16:14 | |
| krobzaur_ | Anybody here have experience with Fuel 6.1? I have a question about deploying the influxdb-grafana and elasticsearch-kibana plugins | 16:19 |
| *** armax has joined #openstack-operators | 16:28 | |
| *** kencjohnston has quit IRC | 16:29 | |
| *** kencjohnston has joined #openstack-operators | 16:30 | |
| *** Piet has joined #openstack-operators | 16:31 | |
| *** armax has quit IRC | 16:37 | |
| *** laron has quit IRC | 16:39 | |
| *** armax has joined #openstack-operators | 16:40 | |
| *** VW has joined #openstack-operators | 16:45 | |
| *** kencjohnston has quit IRC | 16:45 | |
| *** VW has quit IRC | 16:47 | |
| *** VW has joined #openstack-operators | 16:48 | |
| pasquier-s | krobzaur_, I can help | 16:48 |
| *** maishsk has quit IRC | 16:48 | |
| *** krobzaur_ has quit IRC | 16:49 | |
| *** swann has joined #openstack-operators | 16:51 | |
| *** elemoine has quit IRC | 16:53 | |
| *** kencjohnston has joined #openstack-operators | 16:54 | |
| *** gyee has joined #openstack-operators | 16:55 | |
| *** elemoine has joined #openstack-operators | 16:56 | |
| *** kencjohnston has quit IRC | 16:57 | |
| *** harshs has joined #openstack-operators | 16:58 | |
| *** mriedem is now known as mriedem_meeting | 16:59 | |
| *** kencjohnston has joined #openstack-operators | 16:59 | |
| *** laron has joined #openstack-operators | 17:00 | |
| *** britthou_ is now known as britthouser | 17:00 | |
| *** laron has quit IRC | 17:04 | |
| *** alop has joined #openstack-operators | 17:06 | |
| *** laron has joined #openstack-operators | 17:19 | |
| *** SimonChung has joined #openstack-operators | 17:23 | |
| *** matrohon has quit IRC | 17:25 | |
| *** elemoine has quit IRC | 17:28 | |
| *** _hanhart has joined #openstack-operators | 17:29 | |
| mgagne | if anyone is planning to support federation in their public cloud, I would like to hear about it | 17:30 |
| *** laron has quit IRC | 17:31 | |
| *** egonzalez has quit IRC | 17:34 | |
| *** rcernin has joined #openstack-operators | 17:36 | |
| *** mriedem_meeting is now known as mriedem | 17:39 | |
| jlk | mgagne: I think we have some very loose plans around allowing blue box private clouds to federate into IBM public cloud, or vice versa. But it's VERY early thoughts, nothing really fleshed out. | 17:40 |
| mgagne | jlk: which cloud will be the SP and IdP? | 17:41 |
| jlk | well I could see it going either way | 17:41 |
| mgagne | jlk: I'm trying to understand the user story around consuming federation enabled cloud | 17:41 |
| *** laron has joined #openstack-operators | 17:41 | |
| jlk | a private cloud customer may want to federate into the public cloud, so private owns the accounts | 17:41 |
| jlk | OR the other way, so the accounts really exist in public cloud, and it federates into the private cloud | 17:41 |
| *** Piet has quit IRC | 17:44 | |
| *** jmccrory has quit IRC | 17:44 | |
| *** lhcheng has joined #openstack-operators | 17:52 | |
| *** _hanhart has quit IRC | 17:54 | |
| mgagne | jlk: I'm interested in the user story: how does he setup this on the public cloud? support ticket? custom control panel? self-service? | 17:54 |
| *** markvoelker_ has joined #openstack-operators | 17:57 | |
| *** laron_ has joined #openstack-operators | 17:57 | |
| *** jmccrory has joined #openstack-operators | 17:58 | |
| jlk | mgagne: we don't know any of that right now | 17:58 |
| mgagne | jlk: alright, same here ^^' | 17:59 |
| *** SimonChung has quit IRC | 17:59 | |
| *** baffle has quit IRC | 17:59 | |
| *** kencjohnston has quit IRC | 17:59 | |
| *** laron has quit IRC | 18:00 | |
| *** markvoelker has quit IRC | 18:00 | |
| *** sbadia has quit IRC | 18:00 | |
| *** baffle has joined #openstack-operators | 18:00 | |
| *** sbadia has joined #openstack-operators | 18:05 | |
| *** armax has quit IRC | 18:06 | |
| *** laron_ has quit IRC | 18:10 | |
| *** sanjayu has joined #openstack-operators | 18:10 | |
| *** VW has quit IRC | 18:12 | |
| *** kencjohnston has joined #openstack-operators | 18:14 | |
| mriedem | jlk: you might be interested in testing this out in pre-prod kilo https://review.openstack.org/#/c/246530/ | 18:17 |
| mriedem | related to klindgren and the conductor load issues they are experiencing in kilo | 18:18 |
| *** sanjayu has quit IRC | 18:18 | |
| *** dansmith has joined #openstack-operators | 18:18 | |
| mgagne | hmm | 18:18 |
| mgagne | interesting | 18:18 |
| mgagne | can't a command line tool be used to set a flag in the BD once migration is completed? (since you can force the migration using CLI) | 18:19 |
| klindgren | apparently not :-/ | 18:19 |
| *** VW has joined #openstack-operators | 18:20 | |
| mgagne | but I guess it's just a change to test the impact of disabling online migration | 18:20 |
| klindgren | asked the same question | 18:20 |
| mgagne | how can someone detect that an online migration is completed? | 18:20 |
| *** harshs has quit IRC | 18:22 | |
| mriedem | mgagne: dansmith: i was going to say we could add a dry run option to nova-manage db migrate_flavor_data | 18:23 |
| dansmith | mriedem: for what purpose? it tells you how many matched/ran right? | 18:23 |
| mriedem | dansmith: to not actually do the migration | 18:23 |
| dansmith | right, I know what dry run means.. to what end? :) | 18:24 |
| dansmith | perhaps I'm missing some scrollback? | 18:24 |
| mriedem | (12:20:36 PM) mgagne: how can someone detect that an online migration is completed? | 18:24 |
| dansmith | mriedem: okay, right, my point was that if you try to run and don't get any hits, then you're done | 18:25 |
| dansmith | you might be able to run with --max=0 or something | 18:25 |
| mgagne | mriedem: it was more or less gateway question to: How can Nova knows it no longer needs to try to migrate data and stop doing lookups | 18:25 |
| mriedem | dansmith: i thought about that but don't think it will do anything https://github.com/openstack/nova/blob/stable/kilo/nova/db/sqlalchemy/api.py#L6074 | 18:25 |
| mriedem | you'll just get 0 results in your query | 18:25 |
| dansmith | can someone test that code to see if it even addresses the cpu problem? | 18:26 |
| *** fawadkhaliq has quit IRC | 18:26 | |
| *** VW has quit IRC | 18:26 | |
| *** armax has joined #openstack-operators | 18:26 | |
| dansmith | if it does, then we could explore doing things automatically, but it doesn't make sense unless that cuts out enough work to matter | 18:26 |
| mriedem | mgagne: there is a a db migration in liberty that forces it | 18:26 |
| mriedem | mgagne: https://github.com/openstack/nova/blob/stable/liberty/nova/db/sqlalchemy/migrate_repo/versions/291_enforce_flavors_migrated.py | 18:27 |
| *** liverpooler has joined #openstack-operators | 18:27 | |
| mriedem | mgagne: so i guess run this query manually https://github.com/openstack/nova/blob/stable/liberty/nova/db/sqlalchemy/migrate_repo/versions/291_enforce_flavors_migrated.py#L24 | 18:28 |
| mgagne | I'm more or less making the assumption that flavor migration is causing the CPU load (I might be wrong) and looking for a solution for an operator to not have to update a config once he is done with online migration but make it automatic | 18:28 |
| *** laron has joined #openstack-operators | 18:28 | |
| dansmith | mgagne: yes, I understand | 18:28 |
| mgagne | but lets get results first =) | 18:28 |
| dansmith | mgagne: prove that that patch makes it better and then we can talk about making it automatic | 18:28 |
| dansmith | mgagne: you understand this is gone in liberty right? | 18:29 |
| mgagne | dansmith: I'm not geared to perform upgrade every 6 months | 18:29 |
| dansmith | we need to balance the benefit of merging something that major into kilo vs. a config option just for one release | 18:29 |
| mgagne | which service uses this code path? | 18:32 |
| dansmith | anything that talks to the db directly | 18:32 |
| mgagne | nova-conductor or all services which happens to use condutor | 18:33 |
| dansmith | so if you're using real conductor, that's conductor and api and scheduler | 18:33 |
| dansmith | no, compute is unaffected | 18:33 |
| mgagne | ok, using conductor | 18:33 |
| mgagne | the real one | 18:33 |
| mgagne | applied and restarted nova-conductor. reduced CPU usage by ~0.5-1%. (down from 7%) can't say I ever had that problem with CPU usage so far, we tend to throw hardware at problems | 18:39 |
| dansmith | does that mean it was 7% and now it's 6%? | 18:41 |
| mgagne | yea | 18:41 |
| mgagne | ~6.5% | 18:41 |
| dansmith | any idea what you were at when you were on juno? or is the load even comparable? | 18:41 |
| mgagne | didn't try Juno, went straight from icehouse to kilo | 18:42 |
| *** mriedem has quit IRC | 18:42 | |
| dansmith | well, same difference | 18:42 |
| mgagne | no, I don't know tbh | 18:42 |
| mgagne | but klindgren might be the one that should test the change since he is the one complaining about CPU usage =) | 18:42 |
| dansmith | okay, well at that low it's hard to imagine that it's really measurable for you anyway | 18:42 |
| dansmith | yep | 18:42 |
| klindgren | working on it now... | 18:43 |
| dansmith | klindgren: please be exceedingly careful about deploying that, in case it's not clear | 18:43 |
| mgagne | faster! | 18:43 |
| * mgagne cracks the whip | 18:43 | |
| dansmith | klindgren: also, you could deploy it to one conductor node and compare | 18:43 |
| *** mriedem has joined #openstack-operators | 18:44 | |
| klindgren | mgagne, I was going to wait for an integration test to actually pass ;-) | 18:46 |
| mgagne | klindgren: :D | 18:46 |
| dansmith | klindgren: mgagne deployed, that's probably a better metric, FWIW | 18:46 |
| mgagne | klindgren: in our case, come back in ~1h30-2h00 :P | 18:46 |
| dansmith | klindgren: unless you mean internal tests | 18:47 |
| dansmith | jenkins won't exercise this code at all | 18:47 |
| mgagne | dansmith: I often have a cowboy hat on my head, you shouldn't trust my tests =) | 18:47 |
| dansmith | mgagne: if you applied, set the config, then at least you're running it | 18:47 |
| klindgren | I changed the default to true | 18:48 |
| klindgren | restarting now | 18:48 |
| klindgren | actually waiting a few minutes to get some 1 minute sar telemetry | 18:49 |
| *** electrofelix has quit IRC | 18:53 | |
| *** armax has quit IRC | 18:53 | |
| *** armax has joined #openstack-operators | 18:55 | |
| * dansmith assumes klindgren lit fire to the cloud | 18:57 | |
| *** SimonChung has joined #openstack-operators | 18:58 | |
| klindgren | yea | 18:59 |
| mdorman | anxiously awaiting the results here klindgren | 18:59 |
| klindgren | generally seeing a 4% decrease in cpu consumption across all cores | 19:00 |
| dansmith | does that mean four times the 30 cores you're running? | 19:00 |
| dansmith | klindgren: ^ | 19:00 |
| klindgren | http://paste.ubuntu.com/13316282/ | 19:01 |
| klindgren | 36% CPU reduction in total | 19:01 |
| klindgren | 4% * 8 cores | 19:01 |
| dansmith | that's good, yes? | 19:02 |
| klindgren | yes - much much closer to pre-juno numbers | 19:02 |
| dansmith | i.e. closer to pre-kilo numbers? | 19:02 |
| dansmith | okay | 19:02 |
| klindgren | s/juno/kilo | 19:02 |
| *** harshs has joined #openstack-operators | 19:02 | |
| dansmith | klindgren: is this in preprod or did you put on a cowboy hat? | 19:02 |
| klindgren | cowboy | 19:02 |
| dansmith | jeezus | 19:02 |
| klindgren | this is real prod with real work load | 19:02 |
| mdorman | klindgren: always has his cowboy hat on | 19:03 |
| klindgren | I dont always edit code - but when I do - I do it in production | 19:04 |
| mdorman | to be fair, there are two other boxes also running conductor that weren’t touched | 19:04 |
| dansmith | well, for what it's worth, I think that was pretty risky | 19:04 |
| dansmith | but whatever, it's your deal | 19:04 |
| dansmith | klindgren: I will clean this up to be less snarky, but I think we'll want to see that run for a while before we get too serious about actually applying it, okay? | 19:04 |
| klindgren | yea - agreed, I will let it bake on this node for a while | 19:05 |
| *** bvandenh has joined #openstack-operators | 19:08 | |
| *** berendt has quit IRC | 19:09 | |
| klindgren | with all cores shown : http://paste.ubuntu.com/13316425/ | 19:10 |
| *** Marga_ has joined #openstack-operators | 19:13 | |
| *** bvandenh has quit IRC | 19:14 | |
| dansmith | klindgren: are you happy enough yet to apologize about calling my answer shit? | 19:15 |
| *** laron has quit IRC | 19:17 | |
| dansmith | ..guess not.. :) | 19:19 |
| xavpaice | he just had to duck out and fix production ;) | 19:19 |
| dansmith | could be :) | 19:19 |
| dansmith | I warned him | 19:19 |
| klindgren | sorry was getting coffee | 19:19 |
| dansmith | xavpaice: actually, I expect my linkedin profile pic is posted somewhere in the office | 19:20 |
| klindgren | to be fair the shit comment was to the throw more hardware at the problem | 19:20 |
| xavpaice | good to see your priorities are right | 19:20 |
| klindgren | ;-) | 19:20 |
| dansmith | with lots of little holes in it | 19:20 |
| xavpaice | more hardware solves all performance proble.... oh | 19:21 |
| mgagne | http://blogs.msdn.com/cfs-filesystemfile.ashx/__key/communityserver-blogs-components-weblogfiles/00-00-01-32-02-metablogapi/8054.image_5F00_thumb_5F00_35C6E986.png | 19:21 |
| mgagne | haha better one with the cowboy hat: http://www.developermemes.com/wp-content/uploads/2013/03/i-dont-always-test-my-code-but-when-i-do-i-do-it-in-production-stay-on-call-my-friend-thumb-238x300.jpg | 19:21 |
| *** SimonChung has quit IRC | 19:22 | |
| *** SimonChung1 has joined #openstack-operators | 19:22 | |
| *** signed8__ has joined #openstack-operators | 19:23 | |
| *** SimonChung1 has quit IRC | 19:23 | |
| *** SimonChung has joined #openstack-operators | 19:23 | |
| klindgren | dansmith, seems like cpu usage is better - but still seeing conductor eat cpu | 19:24 |
| klindgren | http://paste.ubuntu.com/13316590/ | 19:24 |
| *** SimonChung has quit IRC | 19:24 | |
| *** SimonChung1 has joined #openstack-operators | 19:24 | |
| klindgren | so better - but not magic bullet | 19:25 |
| *** signed8bit has quit IRC | 19:25 | |
| dansmith | klindgren: but at pre-kilo levels, yes? | 19:25 |
| *** Piet has joined #openstack-operators | 19:26 | |
| *** krobzaur_ has joined #openstack-operators | 19:26 | |
| *** laron has joined #openstack-operators | 19:27 | |
| klindgren | most of the time seems to be closer to pre kilo-levels. Honestly didn't start looking at it until after kilo - when our cpu monitors started going off | 19:27 |
| klindgren | like I said - pre kilo we ran 40 conductor workers on 2 servers without issues | 19:28 |
| mdorman | we have also added a lot more compute servers since upgrading to kilo, too, so there are multi variables here | 19:28 |
| klindgren | I was under the impression that new compute went into another cell? | 19:29 |
| mdorman | ah yeah you’re right sorry | 19:29 |
| mdorman | mixing up my regions | 19:29 |
| dansmith | klindgren: you seem to think that conductor should not be doing a lot of work and thus not consuming a lot of CPU... why is that? | 19:29 |
| klindgren | I am not saying it shouldn't be doing work. I am saying that it eats by far the most cpu of any process in my cloud. | 19:30 |
| dansmith | it's doing all the database work on behalf of 250 compute nodes... | 19:31 |
| klindgren | for something that as I understand it is bascially acting as a database proxy? | 19:31 |
| dansmith | more than that, even, but at least that | 19:31 |
| *** laron has quit IRC | 19:31 | |
| dansmith | it is more than a database proxy, but that's the work you're seeing the most of I expect, yes | 19:32 |
| klindgren | to give a refrence point - I also campare this our existing provisioning system that hands building of VPS stuff | 19:33 |
| klindgren | where it deals with thousands of nodes, 60k+ VPS's and it uses about the same amount of hardware as our openstack control plane does | 19:33 |
| klindgren | for something about 1/200th the size | 19:34 |
| dansmith | heh | 19:35 |
| mdorman | yeah in general it would be nice if it could be more scalable | 19:36 |
| mdorman | but that kinda applies to everything | 19:36 |
| mdorman | maybe not more scalable. but smaller scaling ratio | 19:36 |
| mriedem | well, you can reduce traffice by turning down periodic tasks in the compute nodes, but it looks like you guys have already looked at some of that | 19:37 |
| mriedem | klindgren: mdorman: are you guys running db slaves at all? | 19:39 |
| dansmith | mriedem: what does that have to do with cpu load? | 19:39 |
| dansmith | on conductor | 19:39 |
| mriedem | nvm | 19:40 |
| mgagne | I find those numbers interesting and it would be nice it performance team could help projects to reach those numbers. 200x is an impressive number | 19:40 |
| klindgren | not saying I expect openstack to do the same thing - they provide different service levels | 19:41 |
| mgagne | yea, sure | 19:41 |
| klindgren | like the vps product doesn't do metadata | 19:41 |
| mgagne | I'm sure there is online migration stuff in play, object backports, periodic tasks, etc. | 19:41 |
| klindgren | but they do online migrations and the like all the time | 19:42 |
| *** elemoine has joined #openstack-operators | 19:42 | |
| mgagne | but find the increase cost in performance difficult to understand | 19:42 |
| dansmith | klindgren: mdorman: so I should abandon this patch right? | 19:42 |
| klindgren | dansmith, I think it helps - it did decrease cpu - but its not the main % of consumption or so it seems | 19:43 |
| mdorman | dansmith: no db slaves | 19:43 |
| dansmith | klindgren: it's clearly not the main database consumption.. that would be silly | 19:44 |
| dansmith | er, cpu consumption part of the database proxy | 19:44 |
| mdorman | i would rather see something where nova recognizes when all flavors (or whatever) are done migrated, and then skip that code on its own | 19:44 |
| mdorman | (or was that the piece that’s removed in Liberty+ ?) | 19:44 |
| dansmith | mdorman: yeah, this is gone in liberty | 19:44 |
| mdorman | gottcha | 19:44 |
| *** elemoine has quit IRC | 19:46 | |
| *** ccarmack has joined #openstack-operators | 19:49 | |
| *** britthouser has quit IRC | 19:54 | |
| *** armax has quit IRC | 19:57 | |
| *** laron has joined #openstack-operators | 19:59 | |
| *** britthouser has joined #openstack-operators | 20:00 | |
| *** krobzaur_ has quit IRC | 20:10 | |
| *** regXboi has quit IRC | 20:11 | |
| *** Marga_ has quit IRC | 20:11 | |
| *** laron has quit IRC | 20:12 | |
| *** harshs has quit IRC | 20:13 | |
| *** Marga_ has joined #openstack-operators | 20:17 | |
| *** bvandenh has joined #openstack-operators | 20:18 | |
| *** ccarmack has left #openstack-operators | 20:21 | |
| openstackgerrit | JJ Asghar proposed openstack/osops-tools-contrib: Adding the README https://review.openstack.org/245316 | 20:22 |
| *** cbrown2_ocf has quit IRC | 20:38 | |
| *** elemoine has joined #openstack-operators | 20:38 | |
| klindgren | dansmith, I think we are going to try temporarily setting the metadata_cache_expiration high to see if that reduces the work load on conductor | 20:43 |
| klindgren | my assumption is that the majority of load in our cloud on conductor comes from metadata | 20:44 |
| klindgren | we serv ~32k metadata request per 15 minutes (132k per hour) | 20:44 |
| mriedem | if anything interesting comes up it'd be good to post back on that performance thread in the ops ML | 20:54 |
| *** bvandenh has quit IRC | 20:55 | |
| *** VW has joined #openstack-operators | 21:00 | |
| *** VW has quit IRC | 21:01 | |
| *** VW has joined #openstack-operators | 21:02 | |
| *** cbrown2_ocf has joined #openstack-operators | 21:05 | |
| *** cbrown2ocf has joined #openstack-operators | 21:05 | |
| *** krobzaur_ has joined #openstack-operators | 21:21 | |
| *** Rockyg has joined #openstack-operators | 21:28 | |
| *** VW has quit IRC | 21:31 | |
| *** maishsk has joined #openstack-operators | 21:32 | |
| *** maishsk_ has joined #openstack-operators | 21:37 | |
| *** cbrown2ocf has quit IRC | 21:37 | |
| *** cbrown2_ocf has quit IRC | 21:37 | |
| *** SimonChung1 has quit IRC | 21:37 | |
| *** maishsk has quit IRC | 21:40 | |
| *** maishsk_ is now known as maishsk | 21:40 | |
| mgagne | klindgren: can metadata response be cached or would it defeat its purpose? (being dynamic) | 21:53 |
| *** Marga_ has quit IRC | 21:53 | |
| klindgren | default cache is 15 seconds | 21:59 |
| klindgren | a lot of stuff in metadata is not dynamic | 21:59 |
| klindgren | or is not very dynamic | 21:59 |
| *** SimonChung has joined #openstack-operators | 21:59 | |
| klindgren | like AZ, flavor, private ip, userdata (currently), image, vm uuid | 22:00 |
| *** elemoine has quit IRC | 22:02 | |
| *** kencjohnston_ has joined #openstack-operators | 22:09 | |
| mgagne | even with cache, conductor is taking that much CPU? | 22:11 |
| *** kencjohnston has quit IRC | 22:13 | |
| *** lhcheng has quit IRC | 22:14 | |
| *** kencjohnston_ has quit IRC | 22:15 | |
| *** harshs has joined #openstack-operators | 22:15 | |
| *** lhcheng has joined #openstack-operators | 22:16 | |
| klindgren | http://paste.ubuntu.com/13316590/ | 22:19 |
| *** rcernin has quit IRC | 22:21 | |
| balajin | klindgren: you mentioned about a quota patch that Cern had | 22:25 |
| balajin | i assume this was the hierarchial multi tenancy or was this quota per flavors? | 22:25 |
| balajin | any links to patches / details? | 22:25 |
| klindgren | yea - hrm - is might be in the ops etherpad - I know it was a topic on the nova unconfrence | 22:26 |
| balajin | https://etherpad.openstack.org/p/operator-local-patches | 22:27 |
| balajin | in this one? | 22:27 |
| klindgren | doubt its in there tbh | 22:27 |
| balajin | okay, any names to talk to | 22:28 |
| klindgren | https://etherpad.openstack.org/p/mitaka-nova-unconference | 22:28 |
| klindgren | line 32 - 40 | 22:28 |
| balajin | thanks | 22:30 |
| *** Marga_ has joined #openstack-operators | 22:33 | |
| *** SimonChung1 has joined #openstack-operators | 22:40 | |
| *** SimonChung has quit IRC | 22:40 | |
| *** liverpooler has quit IRC | 22:53 | |
| *** mriedem has quit IRC | 22:53 | |
| *** liverpooler has joined #openstack-operators | 22:54 | |
| *** liverpooler has quit IRC | 23:00 | |
| *** Rockyg has quit IRC | 23:03 | |
| *** xavpaice has quit IRC | 23:04 | |
| *** SimonChung1 has quit IRC | 23:07 | |
| *** SimonChung has joined #openstack-operators | 23:07 | |
| *** xavpaice has joined #openstack-operators | 23:08 | |
| *** SimonChung has quit IRC | 23:10 | |
| *** lhcheng has quit IRC | 23:25 | |
| *** lhcheng has joined #openstack-operators | 23:27 | |
| *** alejandrito has quit IRC | 23:31 | |
| *** SimonChung has joined #openstack-operators | 23:34 | |
| *** signed8__ has quit IRC | 23:42 | |
| *** SimonChung1 has joined #openstack-operators | 23:51 | |
| *** SimonChung has quit IRC | 23:51 | |
| *** SimonChung has joined #openstack-operators | 23:52 | |
| *** SimonChung has quit IRC | 23:52 | |
| *** SimonChung1 has quit IRC | 23:52 | |
| *** SimonChung2 has joined #openstack-operators | 23:52 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
