Tuesday, 2016-07-12

*** racedo_ has joined #openstack-operators00:11
*** racedo has quit IRC00:11
*** bapalm has quit IRC00:22
*** bapalm has joined #openstack-operators00:25
*** dtrainor has joined #openstack-operators00:28
*** ducttape_ has joined #openstack-operators00:33
*** gyee has quit IRC00:38
*** fragatina has quit IRC00:44
*** dtrainor has quit IRC01:04
*** ducttape_ has quit IRC01:09
*** saneax is now known as saneax_AFK01:12
*** Apoorva_ has joined #openstack-operators01:13
*** britthouser has joined #openstack-operators01:13
*** britthou_ has joined #openstack-operators01:14
*** ducttape_ has joined #openstack-operators01:15
*** Apoorva has quit IRC01:16
*** Apoorva_ has quit IRC01:17
*** britthouser has quit IRC01:18
*** zul has quit IRC01:50
*** piet has quit IRC01:52
*** zul_ has joined #openstack-operators02:01
*** zul has joined #openstack-operators02:02
*** zul_ has quit IRC02:10
*** jamesdenton has joined #openstack-operators02:28
*** ducttape_ has quit IRC02:35
*** piet has joined #openstack-operators02:43
*** piet has quit IRC02:54
*** piet has joined #openstack-operators02:54
*** kstev has quit IRC03:06
*** jamesdenton has quit IRC03:10
*** piet has quit IRC03:16
*** kstev has joined #openstack-operators03:17
*** jamesdenton has joined #openstack-operators03:17
*** armax has quit IRC03:29
*** markvoelker has joined #openstack-operators03:32
*** markvoelker_ has joined #openstack-operators03:35
*** markvoelker has quit IRC03:38
*** markvoelker has joined #openstack-operators03:46
*** markvoelker_ has quit IRC03:49
*** julim has quit IRC03:58
*** jamesdenton has quit IRC04:01
*** ducttape_ has joined #openstack-operators04:06
*** ducttape_ has quit IRC04:11
*** kstev has quit IRC04:31
*** saneax_AFK is now known as saneax04:32
*** d0ugal has quit IRC05:01
*** fawadkhaliq has joined #openstack-operators05:01
*** ducttape_ has joined #openstack-operators05:07
*** ducttape_ has quit IRC05:12
*** markvoelker_ has joined #openstack-operators05:16
*** zeih has quit IRC05:18
*** markvoelker has quit IRC05:19
*** breton has quit IRC05:26
*** liverpooler has joined #openstack-operators05:36
*** breton has joined #openstack-operators05:41
*** arif-ali has quit IRC05:45
*** markvoelker_ has quit IRC05:45
*** breton has quit IRC05:46
*** breton has joined #openstack-operators05:47
*** breton has quit IRC05:54
*** simon-AS5591 has quit IRC05:54
*** breton has joined #openstack-operators05:54
*** fawadkhaliq has quit IRC05:58
*** rcernin has joined #openstack-operators05:58
*** Rockyg has quit IRC05:59
*** d0ugal has joined #openstack-operators06:04
*** ducttape_ has joined #openstack-operators06:08
*** breton has quit IRC06:14
*** ducttape_ has quit IRC06:15
*** breton has joined #openstack-operators06:21
*** breton has quit IRC06:26
*** eddima has joined #openstack-operators06:28
*** breton has joined #openstack-operators06:28
*** d0ugal has quit IRC06:43
*** zeih has joined #openstack-operators06:46
*** simon-AS559 has joined #openstack-operators06:51
*** simon-AS5591 has joined #openstack-operators06:52
*** simon-AS559 has quit IRC06:55
*** fawadkhaliq has joined #openstack-operators06:56
*** britthou_ has quit IRC06:58
*** fawadkhaliq has quit IRC06:59
*** fawadkhaliq has joined #openstack-operators06:59
*** britthouser has joined #openstack-operators06:59
*** tesseract- has joined #openstack-operators07:00
*** pcaruana has joined #openstack-operators07:00
*** simon-AS5591 has quit IRC07:06
*** ducttape_ has joined #openstack-operators07:10
*** simon-AS5591 has joined #openstack-operators07:14
*** ducttape_ has quit IRC07:15
*** fragatina has joined #openstack-operators07:17
*** fragatina has quit IRC07:18
*** fragatina has joined #openstack-operators07:19
*** breton has quit IRC07:27
*** admin0 has joined #openstack-operators07:40
*** breton has joined #openstack-operators07:41
*** admin0 has quit IRC07:42
*** admin0 has joined #openstack-operators07:44
*** beddari has quit IRC07:48
*** breton has quit IRC07:49
*** breton has joined #openstack-operators07:50
*** dfflanders has quit IRC07:52
*** beddari has joined #openstack-operators07:52
*** breton has quit IRC07:55
*** breton has joined #openstack-operators07:56
*** zeih has quit IRC07:56
*** admin0 has quit IRC07:57
*** beddari has quit IRC07:58
*** fawadkhaliq has quit IRC07:59
*** dtrainor has joined #openstack-operators08:00
*** zeih has joined #openstack-operators08:01
*** breton has quit IRC08:01
*** fawadkhaliq has joined #openstack-operators08:01
*** breton has joined #openstack-operators08:02
*** beddari has joined #openstack-operators08:02
*** admin0 has joined #openstack-operators08:04
*** fawadkhaliq has quit IRC08:05
*** dbecker has joined #openstack-operators08:06
*** breton has quit IRC08:07
*** beddari has quit IRC08:08
*** dtrainor has quit IRC08:09
*** ducttape_ has joined #openstack-operators08:11
*** beddari has joined #openstack-operators08:12
*** ducttape_ has quit IRC08:15
*** beddari has quit IRC08:18
*** derekh has joined #openstack-operators08:20
*** beddari has joined #openstack-operators08:23
*** zeih has quit IRC08:24
*** zeih has joined #openstack-operators08:26
*** beddari has quit IRC08:28
*** admin0 has quit IRC08:28
*** beddari has joined #openstack-operators08:32
*** admin0 has joined #openstack-operators08:36
*** beddari has quit IRC08:38
*** beddari has joined #openstack-operators08:42
*** beddari has quit IRC08:48
*** beddari has joined #openstack-operators08:52
*** electrofelix has joined #openstack-operators08:56
*** beddari has quit IRC08:58
*** fawadkhaliq has joined #openstack-operators09:02
*** beddari has joined #openstack-operators09:02
*** fawadkhaliq has quit IRC09:06
*** ducttape_ has joined #openstack-operators09:12
*** ducttape_ has quit IRC09:16
*** fawadkhaliq has joined #openstack-operators09:24
*** dtrainor has joined #openstack-operators09:32
*** zeih has quit IRC09:36
*** admin0 has quit IRC09:46
*** d0ugal has joined #openstack-operators10:03
*** simon-AS5591 has quit IRC10:05
*** admin0 has joined #openstack-operators10:08
*** hieulq_ has quit IRC10:16
*** breton has joined #openstack-operators10:16
*** zeih has joined #openstack-operators10:30
*** hieulq_ has joined #openstack-operators10:31
*** arif-ali has joined #openstack-operators10:50
*** arnewiebalck has quit IRC10:58
*** arnewiebalck has joined #openstack-operators11:02
*** julim has joined #openstack-operators11:05
*** zeih has quit IRC11:11
*** britthouser has quit IRC11:22
*** julim has quit IRC11:23
*** admin0 has quit IRC11:27
*** bmwiedemann has joined #openstack-operators11:30
bmwiedemannHi, what is the easiest way to create a tenant/project with a certain project-id ? (would like to have that to make migration from another cloud easier) or could I just dump+import the appropriate table from the keystone DB?11:31
*** simon-AS559 has joined #openstack-operators11:31
*** simon-AS559 has quit IRC11:49
*** admin0 has joined #openstack-operators11:49
*** fawadkhaliq has quit IRC11:50
*** ducttape_ has joined #openstack-operators11:58
*** jokke_ has quit IRC11:59
*** jokke_ has joined #openstack-operators11:59
*** beddari has quit IRC12:08
*** zeih has joined #openstack-operators12:08
*** fawadkhaliq has joined #openstack-operators12:17
*** ducttape_ has quit IRC12:17
*** fawadkhaliq has quit IRC12:20
*** piet has joined #openstack-operators12:22
*** zeih has quit IRC12:25
*** zeih has joined #openstack-operators12:27
*** markvoelker has joined #openstack-operators12:34
*** piet has quit IRC12:40
*** piet has joined #openstack-operators12:40
*** makowals has joined #openstack-operators13:00
*** admin0 has quit IRC13:05
*** britthouser has joined #openstack-operators13:05
*** ducttape_ has joined #openstack-operators13:08
*** admin0 has joined #openstack-operators13:09
*** Trozz has quit IRC13:11
*** admin0 has quit IRC13:15
*** admin0 has joined #openstack-operators13:16
*** ducttape_ has quit IRC13:16
*** jamesdenton has joined #openstack-operators13:16
*** mriedem_away is now known as mriedem13:17
*** vinsh has quit IRC13:17
*** britthouser has quit IRC13:19
*** britthouser has joined #openstack-operators13:20
*** julim has joined #openstack-operators13:21
*** zeih has quit IRC13:21
*** julim has quit IRC13:23
*** dtrainor_ has joined #openstack-operators13:29
*** dbecker_ has joined #openstack-operators13:29
*** d0ugal_ has joined #openstack-operators13:30
*** dtrainor has quit IRC13:32
*** dbecker has quit IRC13:32
*** d0ugal has quit IRC13:33
*** d0ugal_ is now known as d0ugal13:35
*** d0ugal is now known as Guest6571513:36
*** Guest65715 has quit IRC13:36
*** d0ugal_ has joined #openstack-operators13:36
*** dtrainor_ has quit IRC13:38
*** racedo_ has quit IRC13:40
*** dtrainor has joined #openstack-operators13:42
*** racedo has joined #openstack-operators13:43
*** d0ugal_ has quit IRC13:44
*** d0ugal has joined #openstack-operators13:44
*** racedo has quit IRC13:47
*** kstev has joined #openstack-operators13:53
*** Fin1te has joined #openstack-operators14:01
med_\o14:02
med_crickets14:03
*** cdelatte has joined #openstack-operators14:03
med_I show a meeting in this channel now per tom email but it seems pretty vacant.14:04
med_clayton, concur?14:04
*** piet has quit IRC14:04
med_apparently not14:05
claytonI had it on my calendar for now14:09
*** markvoelker has quit IRC14:11
*** admin0 has quit IRC14:11
*** ducttape_ has joined #openstack-operators14:11
*** jlk has quit IRC14:15
*** jlk` has joined #openstack-operators14:15
bmwiedemannthese openstack channels seem pretty sleepy ;-)14:16
*** admin0 has joined #openstack-operators14:16
bmwiedemannbut now I got a simple question: if you are an openstack-operator, how big is your cloud? Mine is 10 nodes with ~700 MB RAM and 120 cores total.14:18
*** mriedem1 has joined #openstack-operators14:28
*** piet has joined #openstack-operators14:28
*** mriedem has quit IRC14:29
*** markvoelker has joined #openstack-operators14:30
*** julim has joined #openstack-operators14:31
*** mriedem1 is now known as mriedem14:31
bmwiedemannerr. meant 700GB RAM14:31
*** Fin1te has quit IRC14:41
*** egonzalez has joined #openstack-operators14:41
*** vinsh has joined #openstack-operators14:47
*** vinsh_ has joined #openstack-operators14:50
*** zeih has joined #openstack-operators14:51
*** vinsh has quit IRC14:53
*** vinsh_ has quit IRC14:54
*** vinsh has joined #openstack-operators14:54
*** vinsh has quit IRC14:55
*** Fin1te has joined #openstack-operators14:58
*** saneax is now known as saneax_AFK15:09
*** dronshaw has joined #openstack-operators15:13
*** Fin1te has quit IRC15:18
*** belmoreira has joined #openstack-operators15:18
*** egonzalez has quit IRC15:32
*** armax has joined #openstack-operators15:32
*** admin0 has quit IRC15:32
*** ducttape_ has quit IRC15:37
*** makowals has quit IRC15:38
*** ducttape_ has joined #openstack-operators15:40
*** belmoreira has quit IRC15:42
*** fragatina has quit IRC15:44
*** openfly has quit IRC15:47
*** fragatina has joined #openstack-operators15:49
*** mestery has quit IRC15:49
*** mriedem1 has joined #openstack-operators15:49
*** Guest52040 has joined #openstack-operators15:50
*** mestery has joined #openstack-operators15:50
*** mriedem has quit IRC15:51
*** mriedem has joined #openstack-operators15:52
*** mriedem1 has quit IRC15:54
*** gyee has joined #openstack-operators15:55
*** pcaruana has quit IRC16:00
*** rcernin has quit IRC16:03
*** eddima has quit IRC16:03
*** liverpooler has quit IRC16:08
*** britthouser has quit IRC16:09
*** tesseract- has quit IRC16:09
*** liverpooler has joined #openstack-operators16:10
*** ggillies has quit IRC16:13
*** david-lyle has joined #openstack-operators16:13
*** ggillies has joined #openstack-operators16:14
haukebrunobmwiedemann, currently our 2 plattforms are smaller than 10 compute nodes, but we ran one last year with 65 compute nodes16:16
*** david-lyle has quit IRC16:21
*** armax_ has joined #openstack-operators16:22
*** armax has quit IRC16:24
*** armax_ is now known as armax16:24
*** d0ugal has quit IRC16:24
*** dbecker_ has quit IRC16:25
*** jlk` is now known as jlk16:26
*** jlk has quit IRC16:27
*** jlk has joined #openstack-operators16:27
*** catintheroof has joined #openstack-operators16:28
*** dtrainor has quit IRC16:29
*** makowals has joined #openstack-operators16:30
*** fragatina has quit IRC16:31
*** Rockyg has joined #openstack-operators16:34
*** admin0 has joined #openstack-operators16:45
*** dbecker has joined #openstack-operators16:46
*** Apoorva has joined #openstack-operators16:55
*** Apoorva has quit IRC16:55
*** derekh has quit IRC16:56
*** Apoorva has joined #openstack-operators17:00
*** armax_ has joined #openstack-operators17:04
*** armax has quit IRC17:07
*** armax_ is now known as armax17:07
*** admin0 has quit IRC17:26
*** eddima has joined #openstack-operators17:32
*** makowals has quit IRC17:34
*** makowals has joined #openstack-operators17:34
*** makowals has quit IRC17:35
*** makowals has joined #openstack-operators17:36
*** eddima has quit IRC17:36
*** britthouser has joined #openstack-operators17:39
*** openfly has joined #openstack-operators17:44
*** electrofelix has quit IRC17:57
*** fawadkhaliq has joined #openstack-operators17:57
*** admin0 has joined #openstack-operators17:57
*** Apoorva_ has joined #openstack-operators17:59
*** Guest52040 has quit IRC18:00
*** Apoorva has quit IRC18:03
*** liverpooler has quit IRC18:11
*** ducttape_ has quit IRC18:11
*** ducttape_ has joined #openstack-operators18:16
*** dbecker has quit IRC18:19
*** mriedem1 has joined #openstack-operators18:27
*** Apoorva_ has quit IRC18:28
*** Apoorva has joined #openstack-operators18:28
*** mriedem2 has joined #openstack-operators18:29
*** mriedem has quit IRC18:29
*** ducttape_ has quit IRC18:30
*** mriedem1 has quit IRC18:32
*** mriedem2 is now known as mriedem18:35
*** admin0 has quit IRC18:38
*** piet has quit IRC18:48
*** krobzaur has joined #openstack-operators18:53
*** piet has joined #openstack-operators18:57
*** Andhra_Guy has joined #openstack-operators19:04
Andhra_Guyhaving issues ipv6 network in liberty19:04
Andhra_Guycreated ipv6 subnet with DHCPv6 stateful mode but instances not getting ipv6 address only link local address19:05
*** ducttape_ has joined #openstack-operators19:06
*** Andhra_Guy has left #openstack-operators19:15
*** gyee has quit IRC19:18
*** julim has quit IRC19:30
*** esker has joined #openstack-operators19:33
*** fragatina has joined #openstack-operators19:33
*** krobzaur has quit IRC19:43
*** fragatina has quit IRC19:44
*** maishsk has joined #openstack-operators19:52
*** fawadkhaliq has quit IRC19:53
*** Apoorva has quit IRC19:59
*** julim has joined #openstack-operators19:59
*** Apoorva has joined #openstack-operators20:00
*** zeih has quit IRC20:01
*** britthouser has quit IRC20:03
*** mriedem1 has joined #openstack-operators20:05
*** simon-AS559 has joined #openstack-operators20:05
*** mriedem has quit IRC20:08
*** mriedem has joined #openstack-operators20:10
*** britthouser has joined #openstack-operators20:11
*** mriedem1 has quit IRC20:12
*** cdelatte has quit IRC20:25
*** jamesdenton has quit IRC20:34
*** priteau has joined #openstack-operators20:47
*** esker has quit IRC20:48
*** markvoelker has quit IRC20:49
*** cdelatte has joined #openstack-operators20:50
*** piet has quit IRC20:56
*** simon-AS559 has quit IRC20:58
*** markvoelker has joined #openstack-operators21:00
*** zeih has joined #openstack-operators21:01
*** julian1 has joined #openstack-operators21:02
*** piet has joined #openstack-operators21:07
*** fragatina has joined #openstack-operators21:08
*** Trident has joined #openstack-operators21:13
*** fragatina has quit IRC21:15
*** julian1 has quit IRC21:20
*** rcernin has joined #openstack-operators21:21
*** maishsk_ has joined #openstack-operators21:21
*** esker has joined #openstack-operators21:22
*** zeih has quit IRC21:22
klindgrenwow - think I just solved a long running performance issue that we were seeing where eventually we got into a state where requests to keyston/neutron were getting hammered against a single server.21:23
*** maishsk has quit IRC21:24
*** maishsk_ is now known as maishsk21:24
klindgreninside haproxy configured neutron/keystone for option forceclose  (also enabled tw_reuse)21:24
klindgrenbefore we were getting "runs" against servers where it would recieve a request for 20 seconds, then get 30-40 in a second21:25
*** ducttape_ has quit IRC21:26
*** esker has quit IRC21:27
*** cdelatte has quit IRC21:28
klindgrenrequests weren't being balanced across all the servers effectively.  I think this is mainly from http keepalive and eventlet pooling and requests not being balanced well inside eventlet21:28
klindgrenso sessions were being kept open for upwards of 200 seconds before being reusued21:28
*** esker has joined #openstack-operators21:28
*** priteau has quit IRC21:32
*** priteau has joined #openstack-operators21:33
*** esker has quit IRC21:33
*** ducttape_ has joined #openstack-operators21:44
*** britthouser has quit IRC21:54
*** markvoelker has quit IRC21:59
*** markvoelker has joined #openstack-operators22:04
*** mriedem has quit IRC22:08
*** priteau has quit IRC22:09
*** priteau has joined #openstack-operators22:10
*** priteau has quit IRC22:13
*** zeih has joined #openstack-operators22:19
*** zeih has quit IRC22:25
*** rcernin has quit IRC22:35
*** zul has quit IRC22:36
*** zul has joined #openstack-operators22:37
jlkOkay, wtf is Senlin, and why is it different from Heat?22:38
jlkklindgren: interesting!22:38
jlkklindgren: are you running keystone in eventlet still?22:38
klindgrennope keystone is in uwsgi22:39
klindgrenbut the eventlet stuff I believe is coming from nova22:40
jlkso are you setting that globally then?22:40
klindgrenIE nova using some of the client stuff to talk to other services - IE keystone_middleware or nova -> neutron api for port info22:40
klindgrenso far just in haproxy in front of the servcies22:41
klindgrenletting it bake for a while before rolling everywhere22:41
klindgrenOur setup is Real LB -> haproxy on the api servers (terminates ssl) -> uwsgi|eventlet service22:42
*** _ducttape_ has joined #openstack-operators22:42
*** ducttape_ has quit IRC22:42
dmsimardjlk: re selin: I don't have an answer for you but you made me choke on my drink.22:42
klindgrenkeystone and nova are uwsgi - neutron is still eventlet22:42
jlkdmsimard: it appears to have been created to provide autoscaling features, but HEAT now does autoscaling, so ¯\_(ツ)_/¯22:43
*** ducttape_ has joined #openstack-operators22:43
dmsimardjlk: I think it's a legit question22:44
jlkthe repo is still getting commits, so somebody thinks it's useful22:44
klindgrendoes heat *really* do autoscaling?)22:44
dmsimardklindgren: yeah, it requires ceilometer/gnocchi/alarming22:44
dmsimardbasically you can tell it to spawn other things when cpu usage is too high or things like that22:45
klindgrenso... the answer is no then :D22:45
jlkklindgren: okay. We're using haproxy as our LB, fronting all the services. Keystone is behind apache and uwsgi, the rest eventlet22:45
jlkhttp://docs.openstack.org/developer/heat/template_guide/openstack.html#OS::Heat::AutoScalingGroup22:45
jlkso you can do autoscaling, with ceilometer (and now aodh) alarming, and even use lbaasv222:45
dmsimardklindgren: don't shoot the messenger, I hate heat :)22:45
jlkone of our partners landed on https://wiki.openstack.org/wiki/Heat/AutoScaling and started asking why we don't support Senlin for autoscale.22:46
klindgrenjlk - yea basically when I was watching the logs, I would see a server getting a bunch of requets, then nothing for a while (10 seconds to a few minutes) then getting slammed with requests) - bascially the load was very bursty and not spread out at all22:46
*** _ducttape_ has quit IRC22:47
klindgrenalso TQ time was in the 10ms-300000ms22:47
klindgrendepending on how long it had been doing a keepalive and the last time it was used22:47
klindgrennow they TQ time is all in the 10-200ms range22:48
jlkno noticeable ill effect from force close?22:48
*** ducttape_ has quit IRC22:52
*** mriedem has joined #openstack-operators22:53
*** markvoelker has quit IRC23:00
*** saneax_AFK is now known as saneax23:06
*** markvoelker has joined #openstack-operators23:06
klindgrennot yet23:07
klindgrenwe also enabled tw_re-use23:08
klindgrenhttps://github.com/openstack/oslo.service/blob/master/oslo_service/_options.py#L8123:08
klindgrenwe think might be the thing thats causing it23:08
klindgrenit was allegedly fixed, but apparently fix was - add doc notes - keep default the same23:08
klindgrenhttps://bugs.launchpad.net/nova/+bug/136136023:09
openstackLaunchpad bug 1361360 in neutron juno "Eventlet green threads not released back to the pool leading to choking of new requests" [Undecided,Fix committed]23:09
*** catintheroof has quit IRC23:09
harlowjai know klindgren !23:10
harlowjalol23:10
klindgrenlol - ha Josh has been helping debug this as well23:15
jlkso we could set wsgi_keep_alive to false across the services and also "fix" this?23:15
jlkand a reasonable timeout23:16
*** zeih has joined #openstack-operators23:20
*** fragatina has joined #openstack-operators23:21
*** zeih has quit IRC23:26
*** fragatina has quit IRC23:28
*** mjrichardson has quit IRC23:34
*** mjrichardson has joined #openstack-operators23:34
*** ducttape_ has joined #openstack-operators23:39

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!