*** yamamoto has joined #openstack-lbaas | 00:12 | |
*** yamamoto has quit IRC | 00:17 | |
*** AlexStaf has joined #openstack-lbaas | 00:17 | |
*** KeithMnemonic has quit IRC | 00:19 | |
*** longkb has joined #openstack-lbaas | 00:19 | |
*** annp has joined #openstack-lbaas | 01:22 | |
*** vnovakov2 has joined #openstack-lbaas | 01:45 | |
*** vnovakov has quit IRC | 01:47 | |
*** yamamoto has joined #openstack-lbaas | 01:58 | |
*** yamamoto has quit IRC | 02:04 | |
*** yamamoto has joined #openstack-lbaas | 02:20 | |
*** yamamoto has quit IRC | 02:25 | |
*** phuoc_ has joined #openstack-lbaas | 02:40 | |
*** phuoc has quit IRC | 02:42 | |
*** yamamoto has joined #openstack-lbaas | 02:43 | |
openstackgerrit | enikanorov proposed openstack/neutron-lbaas master: Make subnet_id an optional parameter for pool members https://review.openstack.org/569308 | 02:46 |
---|---|---|
*** yamamoto has quit IRC | 02:48 | |
Eugene_ | rm_work: hi. i still decided to fix it ^ | 02:49 |
*** yamamoto has joined #openstack-lbaas | 03:04 | |
*** yamamoto has quit IRC | 03:09 | |
*** yamamoto has joined #openstack-lbaas | 03:25 | |
*** yamamoto has quit IRC | 03:29 | |
*** ivve has quit IRC | 03:43 | |
*** yamamoto has joined #openstack-lbaas | 03:45 | |
*** yamamoto has quit IRC | 03:52 | |
*** rcernin has quit IRC | 04:03 | |
*** rcernin has joined #openstack-lbaas | 04:03 | |
*** yamamoto has joined #openstack-lbaas | 04:08 | |
*** yamamoto has quit IRC | 04:09 | |
*** yamamoto has joined #openstack-lbaas | 04:09 | |
*** phuoc has joined #openstack-lbaas | 04:59 | |
*** phuoc_ has quit IRC | 05:01 | |
*** vnovakov2 has quit IRC | 05:23 | |
*** annp has quit IRC | 05:37 | |
*** annp has joined #openstack-lbaas | 05:38 | |
*** yamamoto has quit IRC | 06:31 | |
*** kobis has joined #openstack-lbaas | 06:33 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/octavia-dashboard master: Imported Translations from Zanata https://review.openstack.org/569346 | 06:50 |
*** yamamoto has joined #openstack-lbaas | 06:51 | |
*** yamamoto has quit IRC | 06:56 | |
*** kobis has quit IRC | 06:57 | |
*** rcernin has quit IRC | 07:01 | |
*** kobis has joined #openstack-lbaas | 07:03 | |
*** ispp has joined #openstack-lbaas | 07:05 | |
*** yamamoto has joined #openstack-lbaas | 07:12 | |
*** tesseract has joined #openstack-lbaas | 07:14 | |
*** ispp has quit IRC | 07:16 | |
*** threestrands has quit IRC | 07:17 | |
*** kobis has quit IRC | 07:18 | |
*** yamamoto has quit IRC | 07:20 | |
openstackgerrit | Adam Harwell proposed openstack/octavia-tempest-plugin master: Create api+scenario tests for members https://review.openstack.org/566199 | 07:21 |
rm_work | forgot to add the right depends-on | 07:21 |
rm_work | which is why you were getting a fail | 07:21 |
rm_work | johnsom: ^^ | 07:22 |
*** yamamoto has joined #openstack-lbaas | 07:34 | |
*** AlexeyAbashkin has joined #openstack-lbaas | 07:46 | |
*** AlexStaf has quit IRC | 08:08 | |
*** kobis has joined #openstack-lbaas | 08:17 | |
*** ispp has joined #openstack-lbaas | 08:36 | |
*** kobis has quit IRC | 08:50 | |
*** salmankhan has joined #openstack-lbaas | 08:51 | |
*** ispp has quit IRC | 08:52 | |
*** ispp has joined #openstack-lbaas | 09:00 | |
*** ispp has quit IRC | 09:08 | |
*** ispp has joined #openstack-lbaas | 09:16 | |
*** salmankhan has quit IRC | 10:10 | |
*** ispp has quit IRC | 10:12 | |
*** salmankhan has joined #openstack-lbaas | 10:13 | |
*** kobis has joined #openstack-lbaas | 10:17 | |
*** kobis has quit IRC | 10:27 | |
*** yamamoto has quit IRC | 10:30 | |
*** salmankhan has quit IRC | 10:31 | |
*** salmankhan has joined #openstack-lbaas | 10:36 | |
*** kobis has joined #openstack-lbaas | 10:45 | |
*** kobis has quit IRC | 10:48 | |
*** longkb has quit IRC | 10:49 | |
*** kobis has joined #openstack-lbaas | 10:49 | |
*** annp has quit IRC | 10:51 | |
*** yamamoto has joined #openstack-lbaas | 10:51 | |
*** kobis has quit IRC | 10:52 | |
*** yamamoto has quit IRC | 10:56 | |
*** kobis has joined #openstack-lbaas | 10:58 | |
*** yamamoto has joined #openstack-lbaas | 11:02 | |
*** yamamoto has quit IRC | 11:06 | |
*** ispp has joined #openstack-lbaas | 11:08 | |
*** atoth has quit IRC | 11:12 | |
*** yamamoto has joined #openstack-lbaas | 11:24 | |
*** kobis has quit IRC | 11:29 | |
*** yamamoto has quit IRC | 11:29 | |
*** yamamoto has joined #openstack-lbaas | 11:41 | |
*** salmankhan1 has joined #openstack-lbaas | 11:50 | |
*** salmankhan has quit IRC | 11:50 | |
*** salmankhan1 is now known as salmankhan | 11:50 | |
*** atoth has joined #openstack-lbaas | 12:00 | |
*** samccann has joined #openstack-lbaas | 13:05 | |
*** phuoc_ has joined #openstack-lbaas | 13:12 | |
*** yamamoto has quit IRC | 13:13 | |
*** phuoc has quit IRC | 13:15 | |
*** kobis has joined #openstack-lbaas | 13:27 | |
*** yamamoto has joined #openstack-lbaas | 13:34 | |
*** yamamoto has quit IRC | 13:35 | |
*** yamamoto has joined #openstack-lbaas | 13:36 | |
*** ispp has quit IRC | 13:58 | |
*** ispp has joined #openstack-lbaas | 14:04 | |
*** ispp has quit IRC | 14:07 | |
*** phuoc has joined #openstack-lbaas | 14:08 | |
*** ispp has joined #openstack-lbaas | 14:09 | |
*** ispp has quit IRC | 14:09 | |
*** phuoc_ has quit IRC | 14:11 | |
*** kobis has quit IRC | 14:25 | |
*** phuoc has quit IRC | 14:35 | |
*** kobis has joined #openstack-lbaas | 14:39 | |
*** ispp has joined #openstack-lbaas | 14:57 | |
*** ispp has quit IRC | 14:58 | |
*** ispp has joined #openstack-lbaas | 15:15 | |
*** ispp has quit IRC | 15:17 | |
*** salmankhan has quit IRC | 15:18 | |
*** ispp has joined #openstack-lbaas | 15:23 | |
*** ispp has quit IRC | 15:24 | |
*** kobis has quit IRC | 15:26 | |
*** kobis has joined #openstack-lbaas | 15:26 | |
*** kobis has quit IRC | 15:33 | |
*** atoth has quit IRC | 15:36 | |
*** ispp has joined #openstack-lbaas | 15:36 | |
*** ispp has quit IRC | 15:40 | |
*** tesseract has quit IRC | 15:46 | |
*** AlexeyAbashkin has quit IRC | 16:06 | |
*** kobis has joined #openstack-lbaas | 16:38 | |
*** kobis has quit IRC | 16:43 | |
*** kobis has joined #openstack-lbaas | 16:47 | |
*** kobis has quit IRC | 16:47 | |
*** kobis has joined #openstack-lbaas | 16:48 | |
johnsom | Cores, can we get a review on this stable/ocata patch: https://review.openstack.org/#/c/562850/ | 17:00 |
johnsom | There was a mailing list email where someone ran into this issue | 17:00 |
*** imacdonn has quit IRC | 17:31 | |
*** imacdonn has joined #openstack-lbaas | 17:31 | |
*** salmankhan has joined #openstack-lbaas | 17:44 | |
johnsom | cgoncalves One quick fix and you have +2 from me: https://review.openstack.org/#/c/549654 | 17:50 |
johnsom | Awesome job BTW | 17:50 |
*** salmankhan has quit IRC | 17:51 | |
rm_work | noice | 17:52 |
johnsom | Not sure if he is around. Have tempted to just fix and get on with it | 17:52 |
rm_work | yeah i might, let me see | 17:52 |
johnsom | rm_work BTW, after I applied the HM patch I'm now getting 11 failures where before I just had 2. | 17:52 |
johnsom | Dumping logs now to dig in | 17:53 |
rm_work | ummmm wut | 17:53 |
rm_work | it's passing upstream O_o | 17:53 |
rm_work | I guess it's prolly in the APIs | 17:53 |
johnsom | Yeah, but I'm running live | 17:53 |
rm_work | which yeah | 17:53 |
rm_work | i'll look | 17:53 |
rm_work | i was running these against my staging cloud and they were all passing | 17:54 |
rm_work | i'll get my devstack back to Live | 17:54 |
rm_work | oh, it is | 17:54 |
*** leitan has joined #openstack-lbaas | 17:54 | |
johnsom | Yeah, maybe I did something wrong, not sure yet. For whatever reason the tempest.log file didn't have the tracebacks | 17:55 |
johnsom | So re-ran, dumping console to a log file | 17:55 |
rm_work | just the member ones? | 17:57 |
rm_work | like, what -r did you use | 17:57 |
johnsom | I ran everything | 17:57 |
rm_work | k | 17:59 |
rm_work | let me know when i can see a runlog / debuglog | 17:59 |
johnsom | Ah, hmm, might be a broken devstack, I'm seeing quota issues | 18:00 |
johnsom | I'm going to rebuild it. ~10 minutes | 18:00 |
johnsom | Are you fixing the grenade patch or should I? | 18:02 |
*** yamamoto has quit IRC | 18:02 | |
rm_work | it's literally just an edit to that regex? | 18:03 |
rm_work | you do it, i'm not 100% confident i know the right thing | 18:03 |
*** ivve has joined #openstack-lbaas | 18:03 | |
johnsom | yeah, let me get it then | 18:03 |
rm_work | but I am ready to review | 18:03 |
rm_work | yeah maybe my devstack is "reverse-broken"? :P | 18:03 |
rm_work | like, not working the same as upstream, in a way that causes everything to be good | 18:03 |
rm_work | lol | 18:03 |
rm_work | all passes here | 18:03 |
rm_work | but i usually run -t, i'll run wide-open | 18:04 |
johnsom | Yeah, this might just be past runs left something or ... let me rebuild | 18:05 |
openstackgerrit | Michael Johnson proposed openstack/octavia master: Add grenade support https://review.openstack.org/549654 | 18:05 |
johnsom | There we go. | 18:07 |
rm_work | o, lol | 18:07 |
johnsom | After that we just need an upgrade doc and we can assert cold upgrades | 18:07 |
rm_work | nice | 18:07 |
rm_work | then we can go for the next level! | 18:07 |
johnsom | The next one just needs some minor tweaks to the grenade job to test that we don't hose LBs during the upgrade | 18:08 |
johnsom | I think I have a decent stab at slides. I might record a dashboard demo just to have as backup just in case I somehow have extra time | 18:09 |
johnsom | Sent you guys a preview link to both presentations. Feel free to comment. | 18:11 |
johnsom | I really haven't done a polish run over them yet | 18:12 |
*** kobis has quit IRC | 18:18 | |
*** kobis has joined #openstack-lbaas | 18:20 | |
*** yamamoto has joined #openstack-lbaas | 18:23 | |
*** kobis has quit IRC | 18:24 | |
johnsom | Hmm, so this is definitely an problem: OverQuotaClient: Quota exceeded for resources: ['security_group']. | 18:28 |
*** yamamoto has quit IRC | 18:28 | |
johnsom | That is with a fresh devstack, running tempest with no settings, just defaults. | 18:31 |
*** yamamoto has joined #openstack-lbaas | 18:33 | |
*** yamamoto has quit IRC | 18:38 | |
rm_work | <_< | 18:46 |
rm_work | hmmm mine actually had some issues too running *all tests* in parallel | 18:47 |
rm_work | need to figure out what that is | 18:47 |
rm_work | there's ... too many ways to run this stuff | 18:48 |
rm_work | it's frustrating to try to swap between noop and live tests, for scenario and API | 18:48 |
rm_work | because i fix one thing but break it for the other mode, or something, or get confused about which mode is supposed to have what states | 18:48 |
rm_work | gonna have to figure out a way to streamline this at least | 18:48 |
rm_work | so i can run in live-mode and noop-mode at the same time, not have to switch VMs constantly (which involves setup work every time) | 18:49 |
rm_work | erg so mine runs member tests fine but is failing on pool tests somehow now | 18:50 |
*** yamamoto has joined #openstack-lbaas | 18:54 | |
*** leitan has quit IRC | 18:58 | |
johnsom | With concurrency 2 I don't get the quota issue, but still have failures: | 18:58 |
johnsom | https://www.irccloud.com/pastebin/9AZ7mwT5/ | 18:58 |
*** leitan has joined #openstack-lbaas | 18:58 | |
johnsom | I am going to run grab a sandwich and will be back in a bit to dig into this. | 18:58 |
*** yamamoto has quit IRC | 19:01 | |
*** leitan has quit IRC | 19:03 | |
rm_work | hmmm one pool API test seems wrong | 19:05 |
rm_work | "# Operating status for pools will always be offline without members" | 19:06 |
rm_work | I thought that was true | 19:06 |
rm_work | maybe it's not apparently | 19:06 |
*** yamamoto has joined #openstack-lbaas | 19:17 | |
*** yamamoto has quit IRC | 19:22 | |
*** salmankhan has joined #openstack-lbaas | 19:31 | |
*** salmankhan has quit IRC | 19:36 | |
*** yamamoto has joined #openstack-lbaas | 19:38 | |
*** yamamoto has quit IRC | 19:43 | |
johnsom | Grrrr, DIB http://logs.openstack.org/54/549654/32/check/octavia-grenade/25d9c16/logs/grenade.sh.txt.gz#_2018-05-18_18_37_32_512 | 19:50 |
*** yamamoto has joined #openstack-lbaas | 19:59 | |
*** yamamoto has quit IRC | 20:05 | |
johnsom | So, for whatever reason my LB is not going ONLINE. Like it didn't get a heartbeat. | 20:26 |
johnsom | ummm, hm, no HM log... | 20:27 |
rm_work | erm | 20:32 |
rm_work | which patches do you have | 20:32 |
rm_work | you need a few | 20:32 |
rm_work | https://review.openstack.org/568711 and https://review.openstack.org/567955 | 20:32 |
rm_work | speaking of https://review.openstack.org/#/c/568711/ that could use a merge I think | 20:33 |
rm_work | let me figure out this pool issue | 20:33 |
johnsom | Yeah, I think the HM issue was biting me | 20:34 |
*** yamamoto has joined #openstack-lbaas | 20:37 | |
*** yamamoto has quit IRC | 20:41 | |
*** samccann has quit IRC | 20:45 | |
*** yamamoto has joined #openstack-lbaas | 20:53 | |
*** yamamoto has quit IRC | 20:59 | |
rm_work | today has been meetings -> errands -> meetings | 21:00 |
rm_work | ugh | 21:00 |
johnsom | Yeah, So after fixing my HM, it's just this on: octavia_tempest_plugin.tests.api.v2.test_pool.PoolAPITest.test_pool_create_with_listener | 21:04 |
rm_work | yes same | 21:06 |
rm_work | that's the one i'm working | 21:06 |
johnsom | Ok | 21:06 |
rm_work | i think it's a legit bad expectation | 21:06 |
rm_work | [12:06:36] <rm_work>"# Operating status for pools will always be offline without members" | 21:07 |
rm_work | [12:06:39] <rm_work>I thought that was true | 21:07 |
rm_work | [12:06:42] <rm_work>maybe it's not apparently | 21:07 |
rm_work | but i literally have not had time to more than read that line since this morning <_< | 21:07 |
rm_work | still in a meeting | 21:07 |
johnsom | I'll let you look at that and I will try to figure out why image building for the grenade jobs is failing with out of disk space | 21:07 |
rm_work | k | 21:07 |
*** yamamoto has joined #openstack-lbaas | 21:07 | |
rm_work | do you know offhand the answer to that? | 21:07 |
rm_work | are pools supposed to go ONLINE with no members? O_o | 21:08 |
rm_work | gonna try to dig through now | 21:08 |
rm_work | I see pool_dict[constants.OPERATING_STATUS] = constants.OFFLINE | 21:09 |
rm_work | on the API side | 21:09 |
rm_work | so i assume it'd have to be in update_db | 21:09 |
johnsom | I don't know off my head | 21:10 |
rm_work | I guess if it has a listener... it gets processed | 21:10 |
rm_work | interesting | 21:10 |
*** yamamoto has quit IRC | 21:12 | |
johnsom | Ok, so figured it out. | 21:18 |
johnsom | stable/queens image building is broken | 21:18 |
rm_work | <_< | 21:18 |
rm_work | right | 21:18 |
johnsom | The ever bloating image has grown | 21:19 |
rm_work | don't we have that patch outstanding | 21:19 |
johnsom | yet again | 21:19 |
rm_work | ah different thing? | 21:19 |
johnsom | Yeah, new/different | 21:19 |
johnsom | I made master better by switching to ubuntu-minimal | 21:19 |
*** yamamoto has joined #openstack-lbaas | 21:19 | |
*** yamamoto has quit IRC | 21:24 | |
*** yamamoto has joined #openstack-lbaas | 21:32 | |
*** yamamoto has quit IRC | 21:38 | |
rm_work | johnsom: ok, testing all the PoolAPITest stuff in serial and then parallel | 21:43 |
rm_work | and then adding in the member tests on top | 21:44 |
rm_work | and then trying it all in parallel together | 21:44 |
johnsom | Ok, I'm still noodling on how to fix stable | 21:44 |
rm_work | maybe fixed the bad assumption | 21:44 |
rm_work | this is like, a matrix | 21:44 |
rm_work | has_listener * test_with_noop | 21:44 |
*** yamamoto has joined #openstack-lbaas | 21:47 | |
*** yamamoto has quit IRC | 21:52 | |
*** yamamoto has joined #openstack-lbaas | 21:58 | |
*** yamamoto has quit IRC | 22:02 | |
rm_work | errr, right. i think it's just not going to work for me to run the tests in parallel with *live amps* | 22:15 |
johnsom | I can | 22:15 |
johnsom | The gotcha for me was the neutron SG quota needs to be bumped if I run concurrency 4 | 22:16 |
*** yamamoto has joined #openstack-lbaas | 22:18 | |
rm_work | ah i'm running pre-traffic which means it doesn't have my optimizations for the members | 22:22 |
rm_work | so it spins members for each class | 22:22 |
*** yamamoto has quit IRC | 22:23 | |
*** yamamoto has joined #openstack-lbaas | 22:28 | |
rm_work | yeah my machine just can't do parallel at all | 22:33 |
*** yamamoto has quit IRC | 22:33 | |
johnsom | Post it and I can give it a spin | 22:33 |
rm_work | what are you running these on | 22:34 |
openstackgerrit | Adam Harwell proposed openstack/octavia-tempest-plugin master: Create api+scenario tests for members https://review.openstack.org/566199 | 22:34 |
openstackgerrit | Adam Harwell proposed openstack/octavia-tempest-plugin master: Create basic traffic balancing scenario test https://review.openstack.org/566700 | 22:34 |
openstackgerrit | Adam Harwell proposed openstack/octavia-tempest-plugin master: Create api+scenario tests for healthmonitors https://review.openstack.org/567688 | 22:34 |
rm_work | here's a rebased chain | 22:34 |
rm_work | i didn't actually fix anything besides the one pool bug | 22:34 |
rm_work | i know you had some comments on the traffic | 22:35 |
johnsom | My dev box, it's a vmware VM. 4 vcpu, 20GB RAM, 250GB disk | 22:35 |
rm_work | ah | 22:35 |
rm_work | i've got 8G of RAM in my stack VM, lol | 22:35 |
rm_work | i think | 22:36 |
rm_work | maybe 6 | 22:36 |
johnsom | Yeah, that is tight | 22:36 |
rm_work | nope, 4g lol | 22:36 |
johnsom | Ok, I have a workaround for the stables. If I uninstall the snapd stuff it frees up enough for DIB to continue. The apt cache purge is like the last step, so there is a lot of build bloat still there in the non-minimal images | 22:37 |
rm_work | lol | 22:37 |
johnsom | I didn't know you could run a devstack in 4GB | 22:37 |
rm_work | yes! :P | 22:38 |
rm_work | and you can even have ONE LB and two member VMs! :P | 22:38 |
rm_work | (this is why i complained about starting so many tiny member VMs) | 22:38 |
*** yamamoto has joined #openstack-lbaas | 22:38 | |
johnsom | https://review.openstack.org/569531 | 22:40 |
johnsom | ^^^ that is my stable branch image build fix | 22:40 |
rm_work | lol k | 22:42 |
johnsom | Did you add something to bump the neutron SG quota limit? | 22:42 |
*** yamamoto has quit IRC | 22:43 | |
johnsom | Removing snapd drops about 77MB | 22:43 |
rm_work | no | 22:46 |
rm_work | but it's not that | 22:46 |
rm_work | it can't boot enough amps lol | 22:46 |
rm_work | it dies just trying to get a third amp to come up | 22:46 |
rm_work | (combo of two classes + one of the tests that spins a second one) | 22:46 |
johnsom | Yeah, no, I know. I was talking about the stable branch issue | 22:46 |
rm_work | oh | 22:46 |
rm_work | hmmm | 22:47 |
rm_work | well, in gates they run in -t | 22:47 |
rm_work | don't they? | 22:47 |
johnsom | no | 22:47 |
rm_work | considering they all pass, they must be? | 22:47 |
rm_work | because... they all pass in gates | 22:47 |
rm_work | <_< | 22:47 |
rm_work | API are run in noop in the gate | 22:48 |
rm_work | and the most SG we could ever need in scenarios is like... | 22:48 |
rm_work | not many | 22:48 |
johnsom | It looks like the gates run concurrency 2 | 22:48 |
*** yamamoto has joined #openstack-lbaas | 22:48 | |
johnsom | But it is based off the number of cores on the host | 22:49 |
johnsom | instance I mean | 22:49 |
johnsom | Yeah, but if we don't set that people pulling this down are going to hit it and have a bad experience... | 22:49 |
rm_work | hmm | 22:50 |
rm_work | can look into it | 22:50 |
johnsom | I mean, this is for testing that a cloud is working right? They are going to run it live, probably on some beefy hardware | 22:50 |
johnsom | https://www.irccloud.com/pastebin/KkpQzsvo/ | 22:50 |
johnsom | https://github.com/openstack/tempest/blob/master/tempest/lib/services/network/quotas_client.py#L20 | 22:51 |
rm_work | yeah, i run it fully concurrent on my cloud | 22:52 |
rm_work | but ... | 22:52 |
rm_work | it doesn't run into that issue because in my cloud the users have sane quotas | 22:52 |
* rm_work shrugs | 22:52 | |
johnsom | Yeah, devstack default is 10 for SG | 22:53 |
rm_work | right so if they're running it on their cloud, they aren't going to be using devstack | 22:55 |
*** yamamoto has quit IRC | 22:55 | |
rm_work | so i'm not sure why this would be our problem :P | 22:56 |
johnsom | Yeah, but joe developer will do what I did.... | 22:56 |
rm_work | well, whatever, we can change these quotas easy enough right? | 22:56 |
rm_work | i assume tempest just has some setting for it we need to use | 22:56 |
johnsom | Yeah, we can just use the neutron service client like we do for nets and stuff. I can whip up a patch if you don't want to mess with it | 22:58 |
johnsom | However, the pools test still needs work: https://github.com/openstack/octavia/blob/master/octavia/controller/healthmanager/health_drivers/update_db.py#L247 | 22:58 |
johnsom | It's failing on line 147 | 22:58 |
johnsom | testtools.matchers._impl.MismatchError: 'ONLINE' != u'OFFLINE' | 22:58 |
rm_work | err | 22:58 |
rm_work | last time it failed because of OFFLINE != ONLINE | 22:59 |
rm_work | <_< | 22:59 |
johnsom | I think I left that comment in there about no members being up | 22:59 |
rm_work | right | 22:59 |
rm_work | originally my comment said exactly that | 22:59 |
rm_work | and just tested for OFFLINE | 22:59 |
rm_work | but running the test, I get a fail because it DOES come ONLINE | 23:00 |
rm_work | and i looked through the logic, and i think maybe I see how it could | 23:00 |
rm_work | so | 23:00 |
johnsom | The patch posted only checks for OFFLINE | 23:00 |
rm_work | err | 23:00 |
rm_work | ... | 23:00 |
johnsom | You didn't post a new pools patch | 23:00 |
*** yamamoto has joined #openstack-lbaas | 23:01 | |
rm_work | AUGH did i fix it in Members | 23:01 |
rm_work | I might have | 23:01 |
rm_work | yep | 23:01 |
rm_work | lol k well. | 23:01 |
rm_work | give me a few minutes | 23:01 |
johnsom | Did I pull the wrong thing down? | 23:02 |
rm_work | https://review.openstack.org/#/c/566199/20/octavia_tempest_plugin/tests/api/v2/test_pool.py | 23:02 |
rm_work | i fixed it in the NEXT patch | 23:02 |
rm_work | by accident | 23:02 |
rm_work | anyway, you should be running closer to the end | 23:02 |
rm_work | there's fixes for random stuff | 23:02 |
rm_work | like, the internals | 23:02 |
rm_work | for example, you won't spin a bunch of member VMs for the API tests if you have at least to https://review.openstack.org/#/c/566700/ | 23:03 |
johnsom | Yeah, I pulled the HM down | 23:03 |
rm_work | ok well, you have the fix then | 23:03 |
rm_work | notice it's looking for ONLINE | 23:03 |
rm_work | but getting OFFLINE | 23:03 |
rm_work | [15:58:44] <johnsom>testtools.matchers._impl.MismatchError: 'ONLINE' != u'OFFLINE' | 23:04 |
rm_work | Expected != Observed | 23:04 |
rm_work | lol yeah i can't even run the LB-List API test *alone* in serial mode | 23:04 |
johnsom | So I'm giving you differing results? | 23:05 |
rm_work | since it spins up 4 LBs total | 23:05 |
rm_work | yeah your result shows that it DIDN'T come online | 23:05 |
*** yamamoto has quit IRC | 23:05 | |
rm_work | but from what I understand and what you linked me in update_db, it should be coming ONLINE | 23:05 |
rm_work | probably I just need to add a Waiter | 23:05 |
rm_work | anyway, let me fix this patch in a minute | 23:06 |
johnsom | Yeah, no matter how I run it I'm getting OFFLINE | 23:06 |
johnsom | both singe or parallel | 23:06 |
rm_work | yeah prolly just it takes too long | 23:06 |
rm_work | one moment | 23:07 |
rm_work | waiting for test run to finish | 23:07 |
rm_work | prolly this: | 23:08 |
johnsom | May 18 16:08:41 devstackpy27-2 octavia-health-manager[117665]: DEBUG octavia.controller.healthmanager.health_drivers.update_db [-] pool d498ad08-3f1f-4cb0-8ab7-6854b0f2d70b status has changed from OFFLINE to ONLINE. Updating db and sending event. {{(pid=118557) _update_status_and_emit_event /opt/stack/octavia/octavia/controller/healthmanager/health_drivers/update_db.py:64}} | 23:09 |
johnsom | Yeah, it's got to be a timing thing, I see in HM with debug it does go online | 23:09 |
rm_work | yep | 23:09 |
rm_work | one sec | 23:09 |
openstackgerrit | Adam Harwell proposed openstack/octavia-tempest-plugin master: Create api+scenario tests for pools https://review.openstack.org/565640 | 23:09 |
openstackgerrit | Adam Harwell proposed openstack/octavia-tempest-plugin master: Create api+scenario tests for members https://review.openstack.org/566199 | 23:09 |
rm_work | had to type yes | 23:09 |
openstackgerrit | Adam Harwell proposed openstack/octavia-tempest-plugin master: Create basic traffic balancing scenario test https://review.openstack.org/566700 | 23:09 |
openstackgerrit | Adam Harwell proposed openstack/octavia-tempest-plugin master: Create api+scenario tests for healthmonitors https://review.openstack.org/567688 | 23:09 |
rm_work | that fixes it | 23:09 |
rm_work | just needs to do a Wait if it's going to actually compare a HM based status | 23:10 |
rm_work | i didn't design the API tests to run in live mode, really | 23:10 |
rm_work | which IS admittedly a problem | 23:10 |
rm_work | so i'll make sure there's no other weird stuff there | 23:10 |
rm_work | i think this was the only one | 23:10 |
*** yamamoto has joined #openstack-lbaas | 23:11 | |
johnsom | Passes in single, will let it rip now | 23:12 |
johnsom | Do you want me to create a neutron quota patch? | 23:12 |
rm_work | sure why not | 23:12 |
rm_work | i'm currently calling tire places trying to figure out who has tires in stock and can put them on my car *tomorrow* so I can get to Vancouver, lol | 23:13 |
*** yamamoto has quit IRC | 23:15 | |
johnsom | Joy | 23:16 |
*** hyang has joined #openstack-lbaas | 23:19 | |
*** yamamoto has joined #openstack-lbaas | 23:21 | |
hyang | Hi there, I ran into some devstack issues of Queens version of Octavia and after some investigation I found it is already fixed in master branch. May I know are there any plans to back-port these fixes to previous 'stable' branches? | 23:22 |
hyang | https://github.com/openstack/octavia/commit/53dc41d06f41c78f13fcdaef4d9e9a08609d8b34 | 23:22 |
hyang | https://github.com/openstack/octavia/commit/1020a3bceb296610c442ed0d445bba594f27b3de | 23:22 |
rm_work | anyone can propose backports | 23:23 |
hyang | any instruction for how? thanks | 23:24 |
rm_work | you can find the change in gerrit by searching for the change-id | 23:24 |
rm_work | which is in the commit message there | 23:24 |
rm_work | for example, if we search: I724f5064309d07fe05f86fcf2c7a488d9319e54c | 23:24 |
rm_work | it brings us to https://review.openstack.org/#/c/550487/ | 23:24 |
rm_work | you can then click the "cherry-pick" button, and type "stable/queens" | 23:25 |
rm_work | and click the button at the bottom left to do the cherry-pick! | 23:25 |
rm_work | that's all it takes | 23:25 |
hyang | thank you so much! will do that | 23:25 |
*** yamamoto has quit IRC | 23:26 | |
johnsom | rm_work I think I need to do this in the devstack plugin and not tempest. I would need to know the octavia service project_id to do it in tempest. | 23:30 |
rm_work | err | 23:31 |
rm_work | but the users are created by tempest | 23:31 |
rm_work | assuming dynamic creds | 23:31 |
johnsom | Yeah, the SGs are created by octavia | 23:31 |
rm_work | OH | 23:31 |
rm_work | you mean THOSE SGs | 23:31 |
johnsom | yep | 23:31 |
johnsom | It pukes in o-cw | 23:31 |
rm_work | lol I was totally not on the same page | 23:32 |
rm_work | then yes | 23:32 |
rm_work | but uhhh | 23:32 |
rm_work | if "tempest testing* is causing their octavia to break due to SG quota | 23:32 |
rm_work | I think they have a problem | 23:32 |
rm_work | lol | 23:32 |
johnsom | yea | 23:32 |
johnsom | https://www.irccloud.com/pastebin/1gpwBPaZ/ | 23:33 |
rm_work | woo, 42 passes :P | 23:33 |
johnsom | LGTM, ship it | 23:33 |
rm_work | yeah we really need to merge this stuff so I can start doing cleanup as I go, instead of juggling a bunch of patches | 23:34 |
rm_work | and then we can start merging your stuff | 23:34 |
rm_work | let's see... i think there was one more comment you had | 23:34 |
rm_work | Details: (TrafficOperationsScenarioTest:test_healthmonitor_traffic) show_member operating_status failed to update to OFFLINE within the required time 60. Current status of show_member: NO_MONITOR | 23:35 |
rm_work | is that still a problem? | 23:35 |
rm_work | or no? | 23:35 |
rm_work | did you run the scenarios just now too? | 23:35 |
johnsom | I ran everything, it's fixed | 23:35 |
rm_work | ok. I mean, i didn't change anything, so i guess it must have been an issue with your HM stuff | 23:36 |
rm_work | ? | 23:36 |
johnsom | Yeah, I was missing some other patches | 23:36 |
rm_work | k | 23:36 |
*** yamamoto has joined #openstack-lbaas | 23:40 | |
*** hyang has quit IRC | 23:40 | |
*** yamamoto has quit IRC | 23:47 | |
*** yamamoto has joined #openstack-lbaas | 23:53 | |
*** yamamoto has quit IRC | 23:57 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!