| openstackgerrit | Adam Gandelman proposed openstack/astara: WIP Add clustering capabilities to instance manager https://review.openstack.org/267263 | 01:22 |
|---|---|---|
| openstackgerrit | Adam Gandelman proposed openstack/astara: Rename self.driver to self.resource where applicable https://review.openstack.org/266590 | 01:22 |
| *** shashank_hegde has quit IRC | 01:32 | |
| *** phil_h has quit IRC | 02:08 | |
| *** reedip has joined #openstack-astara | 03:04 | |
| *** shashank_hegde has joined #openstack-astara | 03:42 | |
| *** xiayu1 has joined #openstack-astara | 06:12 | |
| *** xiayu has quit IRC | 06:13 | |
| *** xiayu has joined #openstack-astara | 06:20 | |
| *** xiayu1 has quit IRC | 06:20 | |
| *** xiayu1 has joined #openstack-astara | 06:26 | |
| *** xiayu has quit IRC | 06:27 | |
| *** shashank_hegde has quit IRC | 08:25 | |
| *** xiayu has joined #openstack-astara | 08:54 | |
| *** xiayu has left #openstack-astara | 08:54 | |
| *** xiayu1 has quit IRC | 08:54 | |
| *** phil_h has joined #openstack-astara | 12:25 | |
| *** phil_h has quit IRC | 12:26 | |
| *** phil_h has joined #openstack-astara | 12:27 | |
| *** openstackgerrit has quit IRC | 13:02 | |
| *** openstackgerrit has joined #openstack-astara | 13:03 | |
| *** shashank_hegde has joined #openstack-astara | 16:23 | |
| rods | adam_g you around? | 16:41 |
| adam_g | rods, hey | 16:41 |
| rods | adam_g hi | 16:41 |
| rods | I'm talking with drwahl to see how we can do some worker monitoring with the rug | 16:42 |
| rods | adam_g do you have a few minutes to join us? | 16:42 |
| adam_g | rods, g+ is probably not doable right now. im baby watching | 16:43 |
| adam_g | rods, what do you mean by worker monitoring? | 16:43 |
| drwahl | is that at all like bird watching? | 16:43 |
| rods | adam_g np, we can chat about it later | 16:43 |
| adam_g | drwahl, ya, im hiding in the bushes right now | 16:43 |
| drwahl | adam_g: we're conceptualizing a monitoring API for the rug | 16:43 |
| adam_g | rods, ive actually been thinking of ways to get some visibility into whats happening in the workers, from the outside | 16:43 |
| drwahl | that reports some metrics about things like how many messages a thread has processes since <some interval> | 16:44 |
| adam_g | yeah | 16:44 |
| adam_g | and current queue length and such | 16:44 |
| drwahl | yup | 16:44 |
| adam_g | essentially the output you get in logs from 'astara-ctl workers debug' | 16:44 |
| adam_g | but available via the API horizon uses | 16:44 |
| rods | adam_g cool, we where thinking about something like that too | 16:45 |
| drwahl | at $previous_employer, we had a .jsp page that every process generated and our monitoring systems could poll for data. something similar with an API would be fantastic | 16:45 |
| adam_g | https://git.openstack.org/cgit/openstack/astara/tree/astara/worker.py#n745 | 16:46 |
| adam_g | pumping something similar to that through the API and dumping as a json blob of stats would be an easy first step i think | 16:46 |
| rods | yeah, that would be a good start | 16:46 |
| drwahl | that would get us at least 80% of what we're looking for | 16:46 |
| adam_g | one thing to consider is that there'd be multiple endpoints to poll if running multiple orchestrators | 16:47 |
| drwahl | some other interesting metrics (from my perspective) would be total memory allocated, total routers being managed, and average time a message is in the queue | 16:48 |
| adam_g | drwahl, we already report on the number of state machines managed, the rest of that would need to be implemented somehow | 16:50 |
| *** phil_h has quit IRC | 16:50 | |
| rods | adam_g do you have a bp for the things we just taked about | 16:52 |
| rods | drwahl and I have time to work on it | 16:52 |
| rods | and would be nice to have something working for the mitaka cycle | 16:52 |
| adam_g | rods, i havent created anything, just an idea ive had kicking around to make some progress on this integratin test suite | 16:53 |
| rods | ok, so maybe later if you have time we can chat quickly and drwahl and I can start writing some code | 16:54 |
| adam_g | sure | 16:55 |
| *** phil_hop has quit IRC | 16:59 | |
| *** phil_h has joined #openstack-astara | 17:04 | |
| *** phil_hop has joined #openstack-astara | 17:17 | |
| openstackgerrit | Adam Gandelman proposed openstack/astara: Adds a functional test for router interfaces https://review.openstack.org/264515 | 17:46 |
| openstackgerrit | Adam Gandelman proposed openstack/astara: Add astara-ctl + API functional tests https://review.openstack.org/243874 | 17:46 |
| openstackgerrit | Adam Gandelman proposed openstack/astara: fail func tests early (do not merge) https://review.openstack.org/266666 | 17:46 |
| openstackgerrit | Adam Gandelman proposed openstack/astara: Adds a functional test for router interfaces https://review.openstack.org/264515 | 17:48 |
| openstackgerrit | Adam Gandelman proposed openstack/astara: Add astara-ctl + API functional tests https://review.openstack.org/243874 | 17:48 |
| openstackgerrit | Adam Gandelman proposed openstack/astara: fail func tests early (do not merge) https://review.openstack.org/266666 | 17:48 |
| *** shashank_hegde has quit IRC | 17:50 | |
| *** shashank_hegde has joined #openstack-astara | 18:14 | |
| openstackgerrit | Shilla Saebi proposed openstack/astara: minor changes to orchestrator astara file https://review.openstack.org/275897 | 19:19 |
| openstackgerrit | mark mcclain proposed openstack/astara: clean up Nova policy changes https://review.openstack.org/275904 | 19:55 |
| *** puranamr has joined #openstack-astara | 19:58 | |
| rods | adam_g are you available for that chat? | 20:05 |
| adam_g | rods, sure | 20:06 |
| *** puranamr has quit IRC | 20:06 | |
| rods | great, thx | 20:06 |
| rods | adam_g you should be able to join us here https://plus.google.com/hangouts/_/dreamhost.com/dhcteam | 20:08 |
| phil_h | adam_g, do you have a minute? | 20:38 |
| adam_g | phil_h, hi | 20:44 |
| phil_h | hello, getting some problems with setting up load balancers | 20:44 |
| phil_h | have one related to lb and one astara problem | 20:45 |
| phil_h | the LB problem - astara launched a LB vm but didn't configure it | 20:45 |
| phil_h | no config for nginx is there | 20:45 |
| phil_h | astara doesn't show any errors though | 20:46 |
| adam_g | phil_h, did you only create the LB and not the members/pool/listeners? it will wait till all of the config exists on the logical resource in neutron before actually configuring the server in the appliance | 20:46 |
| phil_h | I though I set all of that but let me double check | 20:47 |
| phil_h | something happened to the members, one I got the added it is working. | 20:52 |
| phil_h | thanks I have another problem but give me minute | 20:53 |
| adam_g | cool, good to hear | 20:54 |
| phil_h | are you testing using IPv4 or IPv^ addresses? | 20:58 |
| phil_h | IPv6 | 20:58 |
| phil_h | If I have astara set up neutron creates a IPv6 subnet whenever I create a network | 20:59 |
| phil_h | which is different but I have a problem with IPV4 stuff and getting the l2pop set up on the proper interface | 21:01 |
| markmcclain | right.. there's a setting in astara-neutron to auto create resources include v6 subnets | 21:07 |
| markmcclain | phil_h: ^ we've pushed a change to make this optional with the goal to turn it off before mitaka closes | 21:07 |
| markmcclain | phil_h: the auto resource creation generates too many surprises | 21:08 |
| markmcclain | phil_h: http://git.openstack.org/cgit/openstack/astara-neutron/tree/astara_neutron/plugins/decorators.py#n61 | 21:09 |
| phil_hop | Let me look and see if that fixes the problems I have seen | 21:14 |
| elopez | will need to back port the auto create resources to liberty | 21:19 |
| adam_g | markmcclain, why do we configur() in check_boot() ? https://git.openstack.org/cgit/openstack/astara/tree/astara/instance_manager.py#n271 | 22:23 |
| markmcclain | because if we can't push an initial config the boot likely failed | 22:24 |
| adam_g | markmcclain, why test that with a config push and not is_alive() ? | 22:25 |
| markmcclain | well a create event only boots an image | 22:25 |
| markmcclain | we'd have to split it into 2 | 22:25 |
| markmcclain | a create+update | 22:25 |
| adam_g | hm | 22:25 |
| adam_g | was always wondering why i'd see 2 config pushes on a new instance | 22:26 |
| adam_g | just stumbled on that | 22:26 |
| markmcclain | the 2nd push might be a mistake | 22:27 |
| adam_g | for this clustering stuff, checkboot is going to really only have to check the boot/liveliness, and let ConfigureInstance() actually do the confugiring | 22:27 |
| markmcclain | yeah really all depends on how we want to determine a successful boot | 22:27 |
| markmcclain | the HV might say the instance is active but if we can't push a config might be a sign that it's borked | 22:28 |
| adam_g | so configure() gets called once from instance_manager's check_boot, and also during ConfigureInstnace()'s execute() | 22:28 |
| * markmcclain isn't convicted on this subject | 22:28 | |
| adam_g | to me, is_alive() should be a sufficien t test of the boot | 22:28 |
| markmcclain | yeah seems like it would be enough | 22:29 |
| adam_g | esp. since configure is the next state after checkboot | 22:29 |
| adam_g | need to run out for a bit | 22:43 |
| drwahl | yo, akanda: we had a customer (last year) report that UDP traceroutes fail to floaters | 22:55 |
| drwahl | i suspect this is due to the way UDP traceroutes work (semi-random port) | 22:56 |
| drwahl | i'm not sure if there is really anything astara should be doing in this case, as it's probably more of a user error, but might be something worth keeping on your radar | 22:56 |
| *** elopez has quit IRC | 23:26 | |
| openstackgerrit | Merged openstack/astara: minor changes to orchestrator astara file https://review.openstack.org/275897 | 23:42 |
| markmcclain | drwahl: can file a bug here: https://bugs.launchpad.net/astara/+bugs | 23:55 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!