| openstackgerrit | Nguyen Van Trung proposed openstack/neutron-lbaas master: Don't quote {posargs} in tox.ini https://review.openstack.org/608828 | 01:08 |
|---|---|---|
| openstackgerrit | Nguyen Van Trung proposed openstack/octavia master: Don't quote {posargs} in tox.ini https://review.openstack.org/608830 | 01:08 |
| openstackgerrit | coho proposed openstack/neutron-lbaas-dashboard stable/rocky: sni_container_refs needed if we want to use sni https://review.openstack.org/612222 | 01:29 |
| openstackgerrit | coho proposed openstack/neutron-lbaas-dashboard stable/queens: sni_container_refs needed if we want to use sni https://review.openstack.org/612223 | 01:30 |
| openstackgerrit | coho proposed openstack/neutron-lbaas-dashboard stable/pike: sni_container_refs needed if we want to use sni https://review.openstack.org/612224 | 01:31 |
| openstackgerrit | coho proposed openstack/neutron-lbaas-dashboard stable/ocata: sni_container_refs needed if we want to use sni https://review.openstack.org/612225 | 01:31 |
| *** annp has joined #openstack-lbaas | 01:58 | |
| *** hongbin has joined #openstack-lbaas | 02:19 | |
| *** hongbin has quit IRC | 02:35 | |
| *** hongbin has joined #openstack-lbaas | 02:35 | |
| *** hongbin_ has joined #openstack-lbaas | 02:41 | |
| *** hongbin has quit IRC | 02:43 | |
| *** hongbin_ has quit IRC | 04:22 | |
| *** ramishra has joined #openstack-lbaas | 04:23 | |
| *** hongbin has joined #openstack-lbaas | 04:24 | |
| *** hongbin has quit IRC | 04:38 | |
| *** yboaron_ has joined #openstack-lbaas | 04:58 | |
| *** ccamposr has joined #openstack-lbaas | 05:57 | |
| *** pcaruana has joined #openstack-lbaas | 06:21 | |
| *** bzhao__ has joined #openstack-lbaas | 06:42 | |
| *** yamamoto has quit IRC | 06:49 | |
| *** yamamoto has joined #openstack-lbaas | 06:49 | |
| *** yamamoto has quit IRC | 06:53 | |
| *** yamamoto has joined #openstack-lbaas | 06:53 | |
| *** yamamoto has quit IRC | 06:55 | |
| *** yamamoto has joined #openstack-lbaas | 06:58 | |
| *** pcaruana has quit IRC | 06:58 | |
| *** rcernin has quit IRC | 06:59 | |
| *** yamamoto has quit IRC | 07:01 | |
| *** yamamoto_ has joined #openstack-lbaas | 07:01 | |
| *** rcernin has joined #openstack-lbaas | 07:05 | |
| *** yboaron_ has quit IRC | 07:06 | |
| *** rcernin has quit IRC | 07:07 | |
| *** pcaruana has joined #openstack-lbaas | 07:13 | |
| openstackgerrit | ZhaoBo proposed openstack/octavia master: [WIP] Add client_ca_tls_container_ref to Octavia v2 listener API https://review.openstack.org/612267 | 07:33 |
| openstackgerrit | ZhaoBo proposed openstack/octavia master: WIP: Add an option to the Octavia V2 listener API for client cert https://review.openstack.org/612268 | 07:33 |
| openstackgerrit | ZhaoBo proposed openstack/octavia master: WIP: add more option for certification Optional type https://review.openstack.org/612269 | 07:33 |
| openstackgerrit | ZhaoBo proposed openstack/octavia master: WIP: add new ssl header for client certificate https://review.openstack.org/612270 | 07:33 |
| openstackgerrit | ZhaoBo proposed openstack/octavia master: WIP:L7rule support client certificate cases https://review.openstack.org/612271 | 07:33 |
| *** srini_ has quit IRC | 07:36 | |
| *** velizarx has joined #openstack-lbaas | 07:52 | |
| *** rpittau has joined #openstack-lbaas | 08:02 | |
| *** aojea has joined #openstack-lbaas | 08:05 | |
| *** phuoc has quit IRC | 08:06 | |
| *** celebdor has joined #openstack-lbaas | 08:13 | |
| *** yboaron_ has joined #openstack-lbaas | 08:35 | |
| openstackgerrit | Carlos Goncalves proposed openstack/octavia stable/queens: Healthmanager shouldn't update NO_MONITOR members https://review.openstack.org/612280 | 08:36 |
| *** yamamoto_ has quit IRC | 08:46 | |
| *** yamamoto has joined #openstack-lbaas | 08:46 | |
| *** yamamoto_ has joined #openstack-lbaas | 08:57 | |
| *** yamamoto has quit IRC | 09:00 | |
| *** yamamoto_ has quit IRC | 09:01 | |
| *** yamamoto has joined #openstack-lbaas | 09:01 | |
| openstackgerrit | Vadim Ponomarev proposed openstack/octavia master: Add notifications about changed status to worker https://review.openstack.org/611882 | 09:06 |
| *** salmankhan has joined #openstack-lbaas | 09:08 | |
| *** aojea has quit IRC | 09:18 | |
| *** celebdor has quit IRC | 09:43 | |
| *** yamamoto has quit IRC | 09:46 | |
| *** yamamoto has joined #openstack-lbaas | 09:47 | |
| openstackgerrit | Carlos Goncalves proposed openstack/octavia stable/queens: Create disabled members in haproxy https://review.openstack.org/612296 | 09:50 |
| *** celebdor has joined #openstack-lbaas | 09:52 | |
| *** yamamoto has quit IRC | 09:52 | |
| *** aojea has joined #openstack-lbaas | 09:55 | |
| openstackgerrit | Merged openstack/octavia stable/rocky: Ensure pool object contains the listener_id if passed https://review.openstack.org/611321 | 10:11 |
| *** yamamoto has joined #openstack-lbaas | 10:15 | |
| openstackgerrit | Vadim Ponomarev proposed openstack/octavia master: Remove unused methods. https://review.openstack.org/612305 | 10:20 |
| *** aojea has quit IRC | 10:29 | |
| openstackgerrit | Carlos Goncalves proposed openstack/octavia master: DNM: centos-minimal amphora https://review.openstack.org/612309 | 10:30 |
| *** annp has quit IRC | 10:49 | |
| openstackgerrit | Vadim Ponomarev proposed openstack/octavia master: Add a soft check for the allowed_address_pairs extension. https://review.openstack.org/568546 | 10:56 |
| openstackgerrit | Carlos Goncalves proposed openstack/octavia master: DNM: centos-minimal amphora https://review.openstack.org/612309 | 11:11 |
| *** aojea_ has joined #openstack-lbaas | 11:17 | |
| *** pcaruana has quit IRC | 11:26 | |
| *** pcaruana has joined #openstack-lbaas | 11:48 | |
| *** aojea_ has quit IRC | 11:53 | |
| *** pcaruana has quit IRC | 12:12 | |
| *** pcaruana has joined #openstack-lbaas | 12:12 | |
| openstackgerrit | Vadim Ponomarev proposed openstack/octavia master: Add notifications about changed status to worker https://review.openstack.org/611882 | 12:35 |
| *** aojea has joined #openstack-lbaas | 12:45 | |
| velizarx | Hi guys. Can you review this change (https://review.openstack.org/#/c/611987) faster? | 12:47 |
| velizarx | This problem affects all new changes :( | 12:48 |
| *** yamamoto has quit IRC | 12:50 | |
| *** yamamoto has joined #openstack-lbaas | 12:50 | |
| *** yamamoto has quit IRC | 12:55 | |
| openstackgerrit | Nir Magnezi proposed openstack/octavia master: Add posibilities to set default timeouts https://review.openstack.org/609418 | 12:55 |
| *** yboaron_ has quit IRC | 13:05 | |
| *** yboaron_ has joined #openstack-lbaas | 13:06 | |
| *** yamamoto has joined #openstack-lbaas | 13:06 | |
| *** aojea has quit IRC | 13:16 | |
| *** yboaron_ has quit IRC | 13:16 | |
| *** yboaron_ has joined #openstack-lbaas | 13:19 | |
| openstackgerrit | Merged openstack/octavia master: Update docs conf.py for openstackdocstheme change https://review.openstack.org/611987 | 13:43 |
| *** aojea has joined #openstack-lbaas | 13:50 | |
| openstackgerrit | Vlad Gusev proposed openstack/octavia master: Remove unused methods. https://review.openstack.org/612305 | 13:56 |
| *** yamamoto has quit IRC | 13:58 | |
| *** yamamoto has joined #openstack-lbaas | 13:59 | |
| *** yamamoto has quit IRC | 14:04 | |
| *** ivve has joined #openstack-lbaas | 14:12 | |
| *** aojea has quit IRC | 14:22 | |
| openstackgerrit | Vadim Ponomarev proposed openstack/octavia master: Support create amphora instance from volume based. https://review.openstack.org/570505 | 14:33 |
| *** yamamoto has joined #openstack-lbaas | 14:41 | |
| openstackgerrit | Vadim Ponomarev proposed openstack/octavia master: Add notifications about changed status to worker https://review.openstack.org/611882 | 14:55 |
| *** ramishra has quit IRC | 14:59 | |
| openstackgerrit | Vadim Ponomarev proposed openstack/octavia master: Add a soft check for the allowed_address_pairs extension. https://review.openstack.org/568546 | 15:01 |
| *** yboaron_ has quit IRC | 15:05 | |
| *** pcaruana has quit IRC | 15:12 | |
| *** ccamposr has quit IRC | 15:13 | |
| *** aojea has joined #openstack-lbaas | 15:15 | |
| *** emccormick has joined #openstack-lbaas | 15:42 | |
| *** aojea has quit IRC | 15:48 | |
| *** celebdor has quit IRC | 15:52 | |
| emccormick | Hi all. Is anyone here familiar with the kernel parameters that get set in /etc/sysctl.d/ on the amphora haproxy image? | 15:54 |
| xgerman_ | they should aLL be documented as part of the image build | 15:54 |
| emccormick | I was noticing lots of things in there that I would like to be set, however they don't actually seem to be getting loaded | 15:54 |
| xgerman_ | mmh, what OS are you on? | 15:55 |
| emccormick | Well, for example the image has net.netfilter.nf_conntrack_buckets.conf which sets net.netfilter.nf_conntrack_buckets = 125000 | 15:55 |
| emccormick | default ubuntu | 15:55 |
| xgerman_ | ok, that’s strange since we do most of our testing on ubuntu… | 15:56 |
| emccormick | I literally fed nothing to it. Just ran diskimage-create.sh with no arguments | 15:56 |
| xgerman_ | johnsom: ? | 15:56 |
| emccormick | yeah that's why I used it ;) | 15:56 |
| emccormick | didn't want to start off too adventurous | 15:56 |
| emccormick | what's actually loaded is just the system default | 15:56 |
| emccormick | net.netfilter.nf_conntrack_buckets = 16384 | 15:56 |
| xgerman_ | there is now actually some decent centOS support | 15:57 |
| johnsom | o/ | 15:57 |
| xgerman_ | o/ | 15:57 |
| emccormick | xgerman_ That's round 2 for me ;) | 15:57 |
| xgerman_ | :-) | 15:57 |
| emccormick | guessing johnson is Michael Johnson? | 15:58 |
| johnsom | Yes, that would be me. | 15:58 |
| xgerman_ | yep | 15:58 |
| emccormick | I'm the one that was asking the SSL questions on the ML. Thanks for your reply :) | 15:58 |
| emccormick | Got it resolved and replied back | 15:58 |
| johnsom | Not sure what the question is, but the current kernel tuning settings we are using are here: https://github.com/openstack/octavia/blob/master/elements/haproxy-octavia/post-install.d/20-haproxy-tune-kernel | 15:58 |
| johnsom | Note, some of these get applied inside the network namespace, some out depending on their scope | 15:59 |
| xgerman_ | the problem is they are not being applied | 15:59 |
| emccormick | The question is: I see the tuning files created. However they don't seem to get applied | 15:59 |
| emccormick | root@amphora-22809127-bad0-4cbe-a738-a83f70f2ef1c:/etc/sysctl.d# sysctl -n net.netfilter.nf_conntrack_buckets | 16:00 |
| emccormick | 16384 | 16:00 |
| johnsom | Does the LB have a listener? I would check after you have added a listener to the LB | 16:00 |
| emccormick | hmm. not yet. | 16:00 |
| emccormick | another lack of workflow understanding perhaps ;) | 16:00 |
| johnsom | Ah, conntrack, also note, depending on the version of the amphora you are using, we leave conntrack disabled, so those settings will stay at the default. | 16:00 |
| johnsom | I think only Rocky loads conntrack as we needed it for the UDP support. Prior to that, we didn't load it as it was just overhead. | 16:01 |
| johnsom | This is why there is a "|| true" in that file I linked. They will fail to set if conntrack is not active on the amphroa instance | 16:02 |
| emccormick | hmm. ok | 16:03 |
| emccormick | yeah that must be it. I'm using queens. | 16:04 |
| emccormick | I definitely need to set those for my use-case. I have a very busy site that dies without it | 16:04 |
| johnsom | Yeah, check this one: net.ipv4.ip_nonlocal_bind | 16:04 |
| johnsom | It is critical for our setup. If they are being applied, it will be 1 | 16:05 |
| emccormick | set to 1 | 16:05 |
| emccormick | I made a listener but haven't added any pool to it yet | 16:05 |
| johnsom | Ok, so they are being applied. Now, for those conntrack settings. If conntrack is not even loaded into the kernel, you don't need to set those. | 16:06 |
| johnsom | In fact, you can't set them | 16:07 |
| emccormick | I guess I've never tried flying without it ;) | 16:07 |
| johnsom | How busy is this busy site? requests per second and bandwidth wise? | 16:07 |
| emccormick | well, we've had up to about 15,000 concurrent users | 16:08 |
| emccormick | heavy media site with lots of page elements | 16:08 |
| emccormick | trying to see requests / second | 16:09 |
| emccormick | we hit that conntrack table full problem which is what led me to increasing it | 16:09 |
| emccormick | not very good with these analytics reports ;). Looks like a slow week is around 1M pageviews per week | 16:13 |
| emccormick | a more busy week is at least double that | 16:14 |
| johnsom | Ok, I understand. On queens you might be tight on what the amp can handle. There was some performance tuning patches that went in that boosts it to 30,000k with the stock amp. Let me look at where we backported that. | 16:14 |
| johnsom | emccormick You are going to want the jinja template changes from this patch: https://review.openstack.org/#/c/598379/ | 16:17 |
| johnsom | It's easy enough to apply those changes locally, either with the template override or just a local change/patch. | 16:18 |
| johnsom | That will boost your requests per second. | 16:18 |
| johnsom | Goiing forward, with the flavors changes we expect to be able to scale the standalone/active/standby amp more in Stein, and then in a later release Active/Active is still in progress. | 16:19 |
| emccormick | nice | 16:19 |
| emccormick | Even though I'm using Queens APIs, would I be better building the image off of Rocky? Or would that break things? | 16:20 |
| emccormick | I'm probably going to upgrade to Rocky shortly anyway. Just waiting the Kolla release ;) | 16:20 |
| johnsom | It *should* work, but two versions back we don't have gates testing (would be nice though). So, personally I would at least try it. Most of us run newer full stack of Octavia on older clouds. | 16:21 |
| johnsom | Ah, yeah, can't talk much to Kolla. One of the other cores uses it, but he is on vacation for a while. | 16:21 |
| emccormick | I hacked my way through those problem at least ;) | 16:22 |
| emccormick | Did you backport that patch anywhere or it's just in master? | 16:22 |
| johnsom | It looks like it is only in master at the moment | 16:25 |
| johnsom | As I remember, there was an issue with it and CentOS, since CentOS has such an old version of HAproxy. So it might be a bit complicated to backport. | 16:26 |
| openstackgerrit | German Eichberger proposed openstack/octavia master: Allows failover if port is not deallocated by nova https://review.openstack.org/585864 | 16:28 |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Bring up secondary IPs on member networks https://review.openstack.org/611460 | 16:29 |
| emccormick | johnson ok, thanks a lot. I'll try and pick out those changes and see what I get | 16:29 |
| johnsom | Ok, cool. Also, you are still looking for help on the certificate thing right? (not caught up on e-mail yet) | 16:30 |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Don't quote {posargs} in tox.ini https://review.openstack.org/608830 | 16:34 |
| *** aojea has joined #openstack-lbaas | 16:41 | |
| emccormick | johnson I got it figured out. Kolla deploys only one set of certificates, and apparently having the same CA for both client and server certificates breaks things | 17:01 |
| emccormick | Once I split out the certificates and hacked Kolla to deploy both it came up | 17:01 |
| johnsom | Ok | 17:01 |
| emccormick | I hope to document this stuff when I'm done, and also get Kolla doing thing properly. I stole pieces of OSA to make it all go. | 17:02 |
| johnsom | Yeah, I have been wanting to write the detailed install guide for years, it's just getting it to the top of the priority list.... | 17:06 |
| openstackgerrit | Michael Johnson proposed openstack/octavia-tempest-plugin master: Add traffic tests using an IPv6 VIP https://review.openstack.org/611980 | 17:07 |
| *** aojea has quit IRC | 17:15 | |
| *** pcaruana has joined #openstack-lbaas | 17:27 | |
| *** salmankhan has quit IRC | 17:29 | |
| *** irclogbot_3 has joined #openstack-lbaas | 17:34 | |
| emccormick | hmm. Looks like this would prevent me from running a newer amphora image anyway | 17:41 |
| emccormick | The fix for the hmac.compare_digest on python3 requires you to upgrade your health managers before updating the amphora image. The health manager is compatible with older amphora images, but older controllers will reject the health heartbeats from images with this fix. | 17:41 |
| emccormick | that's in Rocky | 17:41 |
| *** irclogbot_3 has quit IRC | 17:56 | |
| *** pcaruana has quit IRC | 17:57 | |
| *** irclogbot_3 has joined #openstack-lbaas | 18:06 | |
| *** aojea has joined #openstack-lbaas | 18:07 | |
| *** abaindur has joined #openstack-lbaas | 18:32 | |
| *** abaindur has quit IRC | 18:32 | |
| *** abaindur has joined #openstack-lbaas | 18:33 | |
| *** aojea has quit IRC | 18:39 | |
| openstackgerrit | Merged openstack/octavia master: Fix logging error in get_current_loadbalancer_from_db https://review.openstack.org/609964 | 19:03 |
| *** irclogbot_3 has quit IRC | 19:14 | |
| *** irclogbot_3 has joined #openstack-lbaas | 19:15 | |
| *** aojea has joined #openstack-lbaas | 19:17 | |
| openstackgerrit | German Eichberger proposed openstack/octavia master: Allows failover if port is not deallocated by nova https://review.openstack.org/585864 | 20:23 |
| *** irclogbot_3 has quit IRC | 20:27 | |
| *** ivve has quit IRC | 20:48 | |
| *** emccormick has quit IRC | 20:48 | |
| *** fnaval has joined #openstack-lbaas | 21:10 | |
| *** dmellado has quit IRC | 21:32 | |
| *** aojea has quit IRC | 21:41 | |
| *** fnaval has quit IRC | 22:12 | |
| *** colby_ has quit IRC | 22:16 | |
| *** salmankhan has joined #openstack-lbaas | 22:28 | |
| *** salmankhan has quit IRC | 22:43 | |
| *** rcernin has joined #openstack-lbaas | 22:46 | |
| *** rcernin_ has joined #openstack-lbaas | 23:28 | |
| *** rcernin has quit IRC | 23:30 | |
| *** celebdor has joined #openstack-lbaas | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!