| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla: Updated from global requirements https://review.openstack.org/358287 | 00:04 |
|---|---|---|
| *** sdake has quit IRC | 00:11 | |
| sbezverk_ | sdake: I think I figured out how to make ovs stable, but I think kolla folks will not going to like it ;-) | 00:14 |
| *** dwalsh has joined #openstack-kolla | 00:14 | |
| *** dwalsh has quit IRC | 00:31 | |
| *** HyperJohnGraham has joined #openstack-kolla | 00:51 | |
| *** dave-mccowan has joined #openstack-kolla | 00:52 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Remove log propagate configuration https://review.openstack.org/358251 | 00:54 |
| *** HyperJohnGraham has quit IRC | 01:01 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: DO_NOT_MERGE: Use better URL for China https://review.openstack.org/329060 | 01:02 |
| *** salv-orlando has joined #openstack-kolla | 01:15 | |
| *** salv-orl_ has quit IRC | 01:18 | |
| *** daneyon has joined #openstack-kolla | 01:29 | |
| *** daneyon has quit IRC | 01:34 | |
| *** v1k0d3n has joined #openstack-kolla | 01:46 | |
| *** v1k0d3n has quit IRC | 01:51 | |
| *** mark-casey has joined #openstack-kolla | 02:00 | |
| *** zhurong has joined #openstack-kolla | 02:32 | |
| *** v1k0d3n has joined #openstack-kolla | 02:39 | |
| *** zhurong has quit IRC | 02:46 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Create the _member_ role in the horizon role https://review.openstack.org/358298 | 02:46 |
| *** rkrum has joined #openstack-kolla | 02:53 | |
| *** Jeffrey4l_ has quit IRC | 02:54 | |
| *** HyperJohnGraham has joined #openstack-kolla | 02:57 | |
| *** dave-mccowan has quit IRC | 03:02 | |
| *** rkrum has quit IRC | 03:04 | |
| *** daneyon has joined #openstack-kolla | 03:17 | |
| *** v1k0d3n has quit IRC | 03:19 | |
| *** rkrum has joined #openstack-kolla | 03:20 | |
| *** duonghq has joined #openstack-kolla | 03:21 | |
| duonghq | happy weekend everybody | 03:21 |
| *** daneyon has quit IRC | 03:22 | |
| *** v1k0d3n has joined #openstack-kolla | 03:23 | |
| *** rkrum has quit IRC | 03:40 | |
| *** kbaegis has quit IRC | 03:49 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair gnocchi dockerfile https://review.openstack.org/348388 | 03:58 |
| *** kbaegis has joined #openstack-kolla | 04:00 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair aodh image dockerfile https://review.openstack.org/350715 | 04:07 |
| *** v1k0d3n has quit IRC | 04:09 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair gnocchi dockerfile https://review.openstack.org/348388 | 04:11 |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair gnocchi dockerfile https://review.openstack.org/348388 | 04:12 |
| kbaegis | duonghq: Happy weekend :) | 04:14 |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add gnocchi ansible role https://review.openstack.org/349351 | 04:22 |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add aodh role https://review.openstack.org/351027 | 04:26 |
| *** v1k0d3n has joined #openstack-kolla | 04:27 | |
| *** v1k0d3n has quit IRC | 04:32 | |
| *** sdake has joined #openstack-kolla | 04:46 | |
| *** sdake_ has joined #openstack-kolla | 04:50 | |
| sdake_ | wznoinsk still around? | 04:53 |
| *** sdake has quit IRC | 04:53 | |
| sdake_ | wznoinsk if you read this later - its more then just compute that needs special attention with ironic, also nova-conductor and the entire interface into how nova operates | 04:55 |
| sdake_ | wznoinsk i think also neutron is affected | 04:55 |
| *** sdake_ has quit IRC | 05:15 | |
| *** unicell has joined #openstack-kolla | 05:42 | |
| openstackgerrit | Tin Lam proposed openstack/kolla: Add ansible role for vmtp container https://review.openstack.org/358305 | 06:20 |
| openstackgerrit | Shaun Smekel proposed openstack/kolla-kubernetes: Updated Pods and RCs to Deployments https://review.openstack.org/358306 | 06:26 |
| openstackgerrit | Shaun Smekel proposed openstack/kolla-kubernetes: [WIP] Updated Pods and RCs to Deployments https://review.openstack.org/358306 | 06:27 |
| *** duonghq has quit IRC | 06:57 | |
| openstackgerrit | Shaun Smekel proposed openstack/kolla-kubernetes: [WIP] Updated Pods and RCs to Deployments https://review.openstack.org/358306 | 07:00 |
| *** bootsha has joined #openstack-kolla | 07:00 | |
| *** hieulq_ has joined #openstack-kolla | 07:06 | |
| *** matrohon has joined #openstack-kolla | 07:10 | |
| *** salv-orl_ has joined #openstack-kolla | 07:16 | |
| *** salv-orlando has quit IRC | 07:18 | |
| *** matrohon has quit IRC | 07:59 | |
| *** bootsha has quit IRC | 08:06 | |
| *** bootsha has joined #openstack-kolla | 08:11 | |
| *** bootsha has quit IRC | 08:26 | |
| *** v1k0d3n has joined #openstack-kolla | 08:29 | |
| *** bootsha has joined #openstack-kolla | 08:30 | |
| *** hieulq_ has quit IRC | 08:33 | |
| *** v1k0d3n has quit IRC | 08:33 | |
| *** mark-casey has quit IRC | 08:37 | |
| *** sdake has joined #openstack-kolla | 08:38 | |
| *** sdake has quit IRC | 08:43 | |
| *** daneyon has joined #openstack-kolla | 08:43 | |
| *** hieulq_ has joined #openstack-kolla | 08:46 | |
| *** daneyon has quit IRC | 08:47 | |
| *** bootsha has quit IRC | 08:48 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Pin Ansible version to 2.1.1.0 in kolla-toolbox https://review.openstack.org/358320 | 08:56 |
| *** bootsha has joined #openstack-kolla | 09:36 | |
| *** dwalsh has joined #openstack-kolla | 09:49 | |
| *** hieulq_ has quit IRC | 09:55 | |
| *** hieulq_ has joined #openstack-kolla | 10:20 | |
| *** daneyon has joined #openstack-kolla | 10:31 | |
| *** daneyon has quit IRC | 10:35 | |
| *** bootsha has quit IRC | 10:52 | |
| *** bootsha has joined #openstack-kolla | 10:54 | |
| *** hieulq_ has quit IRC | 11:01 | |
| *** dmueller-evoila has joined #openstack-kolla | 11:04 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair gnocchi dockerfile https://review.openstack.org/348388 | 11:14 |
| *** dwalsh has quit IRC | 11:17 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair aodh image dockerfile https://review.openstack.org/350715 | 11:19 |
| *** sdake has joined #openstack-kolla | 11:46 | |
| *** sdake has quit IRC | 11:50 | |
| *** sdake has joined #openstack-kolla | 11:51 | |
| sdake | wznoinsk around? | 12:00 |
| sdake | morning folks | 12:01 |
| sdake | say I'll be in and out the next 2-3 days | 12:01 |
| sdake | moslty out | 12:01 |
| *** dave-mccowan has joined #openstack-kolla | 12:09 | |
| openstackgerrit | Shaun Smekel proposed openstack/kolla-kubernetes: [WIP] Updated Pods and RCs to Deployments https://review.openstack.org/358306 | 12:10 |
| *** bootsha has quit IRC | 12:12 | |
| *** bootsha has joined #openstack-kolla | 12:17 | |
| *** zhubingbing has joined #openstack-kolla | 12:20 | |
| kbaegis | Morning all | 12:26 |
| kbaegis | sdake: Good morning. Fix your laptop? | 12:26 |
| sdake | kbaegis ya - bad power supply | 12:26 |
| sdake | scabbed my wifes | 12:26 |
| sdake | about the e8th one i've been thorugh on that mac | 12:27 |
| sdake | 80 bucks a crack | 12:27 |
| kbaegis | Nice. I might be doing that soon. | 12:27 |
| sdake | terribly made | 12:27 |
| kbaegis | Yeah, Apple has crazy margins | 12:27 |
| sdake | it would beo k if it were 150 if it were made properly | 12:27 |
| sdake | but seriously their power blocks are junk | 12:28 |
| kbaegis | The older gen? | 12:28 |
| sdake | 85 watt | 12:28 |
| kbaegis | I've never had problems with magsafe2 | 12:28 |
| sdake | mine break all the time | 12:28 |
| sdake | cable breaks | 12:28 |
| kbaegis | Now they're all apparently usb | 12:28 |
| kbaegis | Oh, yeah | 12:28 |
| sdake | ya usbc - can't wait ;) | 12:28 |
| sdake | magsafe comes unplugged all the time on accident | 12:28 |
| kbaegis | I can. The port is tiny, and it comes unplugged as well | 12:29 |
| sdake | that is actually how i broke my screen | 12:29 |
| kbaegis | Really? Whipped it into the screen or something? | 12:29 |
| sdake | magsafe popped off, and it came unplugged whlie i was closing my laptop | 12:29 |
| zhubingbing | hello | 12:29 |
| kbaegis | Oh, dang | 12:29 |
| kbaegis | hello | 12:29 |
| sdake | this blew up the left corner of the display | 12:29 |
| sdake | but it is still usable | 12:29 |
| sdake | trying to find time ot get it repaired | 12:29 |
| kbaegis | Well I'm going to lose my job in two weeks, and I will lose my macbook then :) | 12:29 |
| sdake | i've read its 400-800 at apple store | 12:29 |
| sdake | kbaegis sorry to hear that | 12:30 |
| kbaegis | Yeah, crazy margins | 12:30 |
| sdake | kbaegis where do you work? | 12:30 |
| kbaegis | I'm @ Mirantis | 12:30 |
| zhubingbing | Mirantis？ | 12:30 |
| sdake | i see | 12:30 |
| sdake | what happens in 2 weeks? | 12:30 |
| sdake | just general layoff? | 12:30 |
| kbaegis | Asked to change departments, backfilled, and then my new job disappeared | 12:30 |
| kbaegis | Hiring freeze | 12:30 |
| *** v1k0d3n has joined #openstack-kolla | 12:30 | |
| kbaegis | Everyone's real sorry about it and all | 12:31 |
| sdake | i'm sure | 12:31 |
| kbaegis | I walk away w/ good references at least :) | 12:31 |
| zhubingbing | I'm sorry to hear that. have you found your new job？ | 12:31 |
| sdake | most of indsutry is in iring freeze atm | 12:31 |
| kbaegis | They're anticipating a recession. Which is smart | 12:32 |
| sdake | recesion doesn't have much ipact on tech spend | 12:32 |
| kbaegis | Why the freeze then? | 12:32 |
| sdake | world is very unstable atm | 12:33 |
| sdake | that happens in election years | 12:33 |
| sdake | once a new order happens, things should relax | 12:33 |
| sdake | brexit was sort of a noop in the us thus far | 12:33 |
| sdake | everyone was betting tht wouldnt happen | 12:34 |
| kbaegis | Yeah, big in Europe though | 12:34 |
| sdake | affects the us | 12:34 |
| sdake | moves around banking | 12:34 |
| sdake | brussels, ireland | 12:34 |
| kbaegis | Well, I'm expecting a recession. Student-debt, housing, etc | 12:34 |
| sdake | europe has negative interest rates | 12:34 |
| kbaegis | Really? | 12:34 |
| sdake | ya you pay the bank to safeguard your money... | 12:35 |
| *** v1k0d3n has quit IRC | 12:35 | |
| sdake | full fiath and credit fwiw :) | 12:35 |
| kbaegis | Open up a savings account with your mattress | 12:35 |
| sdake | i doubled down on my international portfolio | 12:35 |
| sdake | kbaegis the paper money could be no good | 12:36 |
| sdake | but the bank electronic reference is good protected by the faith and credit | 12:36 |
| sdake | this is the problem with paper money | 12:36 |
| *** zhurong has joined #openstack-kolla | 12:36 | |
| kbaegis | sdake You know what's awesome in a full currency collapse? | 12:36 |
| kbaegis | Cheap gold | 12:36 |
| kbaegis | 12k | 12:36 |
| sdake | a full currency collapse would cause gold to be worth 4-5k an ounce | 12:36 |
| sdake | i'm a supporter of audit the fed | 12:37 |
| kbaegis | exactly. But the secret is- and USSR was a case study here- there's a gap before people figure out how to test gold | 12:37 |
| sdake | but i'm afraid the country wont like the answers. | 12:37 |
| kbaegis | Well we used to have world wars over debt | 12:37 |
| kbaegis | Now we've got economics, ww 2.0 | 12:37 |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add Barbican ansible role https://review.openstack.org/352515 | 12:38 |
| kbaegis | The fed is simply an instrument of US imperialism. | 12:38 |
| sdake | the wars wren't over debt, they were for small iconsequential reasons which coudl have been resolved early on which escalated and escalated | 12:38 |
| sdake | in a convential war the us always wins | 12:38 |
| kbaegis | Well, there's a branch of scholarly historians who advocate that every war is about money and property | 12:39 |
| sdake | if war canbe won | 12:39 |
| sdake | the old wars prior to ww1 and 2 i agree completely | 12:39 |
| kbaegis | War is simply about subjugation. "A government is a group of people, most notably ungoverned" | 12:39 |
| kbaegis | We have the most favorable rules because we have the biggest guns | 12:40 |
| kbaegis | Why the dollar is still the reserve currency | 12:40 |
| sdake | yup if dollar drops off as world reserve currency | 12:40 |
| sdake | us in for a steep delcine | 12:40 |
| sdake | that is years out | 12:40 |
| kbaegis | It should've happened in '08 | 12:41 |
| sdake | surprised it didn't | 12:41 |
| sdake | but the us printed some more zeros | 12:41 |
| sdake | surprised quantative easing worked | 12:41 |
| sdake | was expecting a differenet outcome | 12:41 |
| sdake | that whole system was riggeed in favor of wallstreet - which is one of hte reasons it worked | 12:42 |
| kbaegis | Well when the fed finally breaks, it will be the end of our indirect empire. We'll have to a) go back to direct warfare, or b) end our empire | 12:42 |
| kbaegis | Yeah, wealth concentration is a huge issue. Our empire is going to eat itself | 12:42 |
| sdake | i need to concreate some welth :) | 12:42 |
| kbaegis | Yeah, but we aren't playing by the same rules. | 12:43 |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add sahara ansible role https://review.openstack.org/351294 | 12:43 |
| sdake | my uncle and cousin both self made 20millionaires | 12:43 |
| kbaegis | Middle class is gone mostly because of the housing fraud. How much you wanna bet that your job will get lost during a 30 year fixed? | 12:43 |
| sdake | father self-made 12 mil | 12:43 |
| kbaegis | Nice | 12:43 |
| kbaegis | How? | 12:43 |
| sdake | my grandgather moved to arizona with 10 cents in his pocket | 12:44 |
| sdake | literally 10 cents | 12:44 |
| sdake | he used it to get a job | 12:44 |
| sdake | he started a motorshop later - 5-10 years later | 12:44 |
| sdake | uncle and cousin grew it into quite an empire | 12:44 |
| sdake | all southwest motors were sold through their shop | 12:44 |
| sdake | my father went into law | 12:44 |
| sdake | and he saves far more then he made during his career | 12:45 |
| kbaegis | Will you buy me some equipment sdake? :) | 12:45 |
| sdake | he had 100k usd saved by 25 | 12:45 |
| sdake | they sold hte business | 12:45 |
| kbaegis | That's cool | 12:45 |
| sdake | to your question about job over 30 year period | 12:46 |
| sdake | that is highly unlikely to survive ffor 30 years in one company | 12:46 |
| *** v1k0d3n has joined #openstack-kolla | 12:46 | |
| sdake | i'm on my fourth stint | 12:47 |
| kbaegis | Yup. According to most mortgages, you have 90 days to find a new job. Add in recessions, and mortgages become an upward money engine (pumping it to the wealthy) | 12:47 |
| sdake | i have a family loan | 12:47 |
| sdake | and 120k emergency fund (1 year of expenses) | 12:48 |
| kbaegis | Amortization and defaults are the banks way of implementing the "house odds" | 12:48 |
| kbaegis | Huh, just realized that's a bad pun :) | 12:48 |
| sdake | banks would rather hvae your money then a default - trust me | 12:48 |
| sdake | you didn't borrow the house, you borrowed the money | 12:48 |
| sdake | thats what they want back | 12:49 |
| sdake | people use their houes at atm machines | 12:49 |
| sdake | bad bad move | 12:49 |
| sdake | at/as | 12:49 |
| kbaegis | I don't know if that's true. I think it's pretty effectively rigged | 12:49 |
| kbaegis | You can't blame homeowners | 12:49 |
| *** zhurong has quit IRC | 12:49 | |
| sdake | for using their house as an atm machine? | 12:50 |
| sdake | the fact that lenders let em do it - thats the problem | 12:50 |
| sdake | that is what 2008 was all about | 12:50 |
| sdake | then the great short | 12:50 |
| sdake | wallstreet profited heavily from the 2008 crash | 12:50 |
| *** zhurong has joined #openstack-kolla | 12:50 | |
| kbaegis | I thought '08 was about ninja loans and rating agency collusion | 12:50 |
| kbaegis | Not refinancing | 12:51 |
| sdake | all contributing factors | 12:51 |
| sdake | brb | 12:51 |
| kbaegis | Also, it's *automatic atm machine. ;) | 12:53 |
| *** v1k0d3n has quit IRC | 12:53 | |
| sdake | the lack of income verification - banks should have known better | 12:53 |
| sdake | agency collusion - they knew the debt was no good | 12:54 |
| sdake | homeowners DIDN"T know their risk was spread across the entire system | 12:54 |
| sdake | so they spent more then they had and refianced | 12:55 |
| sdake | to fix their finances | 12:55 |
| sdake | when the answer is consume less | 12:55 |
| sdake | i save 50% of my cross | 12:55 |
| sdake | gross | 12:55 |
| sdake | its painful | 12:55 |
| sdake | live paycheck to paycheck as a result | 12:55 |
| kbaegis | Yeah. I've been saving 80% for a downpayment. Going to be helpful since I'm now looking :) | 12:56 |
| sdake | i've only got 140k left on my mortgage | 12:57 |
| sdake | i should probablly just pay it off, but at3% apr | 12:57 |
| sdake | started at 240ish around 5 years ago | 12:57 |
| sdake | i just refinanced my car at 2.74% apr too | 12:58 |
| sdake | yay ;) | 12:58 |
| sdake | besides those hard assets, debt free and nearing 1mil in investments | 12:58 |
| kbaegis | Seriously? Nice | 12:59 |
| kbaegis | You're in the top 1% for assets for sure | 12:59 |
| sdake | thats in market, have about 150 oz of gold as well :) | 12:59 |
| *** v1k0d3n has joined #openstack-kolla | 13:00 | |
| sdake | by 2020 shoudl have about 2-3 mil | 13:01 |
| sdake | assuming i can stay employed | 13:01 |
| kbaegis | Nice | 13:01 |
| sdake | the key is starting early | 13:01 |
| sdake | 17k/yr at 20 for 10 years is enogh for retirement | 13:02 |
| kbaegis | Well, you're well positioned with openstack | 13:02 |
| sdake | compounding is a powerful natural force | 13:02 |
| sdake | i dont know about well positioned | 13:03 |
| sdake | i'm a contirbutor like everyone else | 13:03 |
| kbaegis | Yeah, but openstack is huge right now | 13:03 |
| sdake | keys words being right now | 13:05 |
| sdake | back on the 30 year jobs things | 13:05 |
| sdake | my strategy for that is to be a tip of the spear engineer | 13:05 |
| sdake | businesses rarely cut off the tip of the spear | 13:06 |
| sdake | and if they do, other businesses want it | 13:06 |
| sdake | hence job security through relevant skillset | 13:06 |
| sdake | i'm a jack of all trades _master_ of many | 13:06 |
| sdake | i used to subscribe to the philsophy do one thing do it well | 13:06 |
| sdake | but i've learned hybrid engineers are much more effective and various managers recognize it and WANT hyrids | 13:07 |
| sdake | sincethey are not all that common | 13:07 |
| kbaegis | So it's looking like my kolla-build has hung | 13:07 |
| sdake | to become a hybrid takes alot of OTJ training | 13:07 |
| sdake | kbaegis - centos? | 13:07 |
| sdake | it will time out eventually and try another mirror | 13:08 |
| sdake | we retry each build 3 times | 13:08 |
| sdake | (ocnfigurable) | 13:08 |
| kbaegis | last message is "INFO:kolla.image.build.nova-libvirt:Cleaning up list of fastest mirrors" and has been for ~10 minutes | 13:08 |
| sdake | yup that happens sometimes | 13:09 |
| sdake | which part of the world are ou in | 13:09 |
| sdake | non-responsive mirror lags up the build | 13:09 |
| sdake | anothe option is ou are out of disk space | 13:10 |
| kbaegis | I'm in the US | 13:10 |
| kbaegis | Definitely not a space issue | 13:10 |
| kbaegis | Maybe I should drop my threads back from 32 :) | 13:11 |
| kbaegis | Had some push timeouts | 13:11 |
| sdake | oh - use threads=8 | 13:11 |
| sdake | defaults should be solid on threads | 13:11 |
| *** dwalsh has joined #openstack-kolla | 13:12 | |
| sdake | we have had reports of build locksup on threads > 8 | 13:12 |
| sdake | probably an option we should deprecate | 13:12 |
| *** daneyon has joined #openstack-kolla | 13:13 | |
| sdake | thrads = 32 uses alot of ram | 13:14 |
| sdake | docker is limited to consuming 1gig iirc on most platforms | 13:14 |
| sdake | this is not enogh for that many thrasd | 13:14 |
| *** salv-orlando has joined #openstack-kolla | 13:15 | |
| sdake | docker ooms - bad things happen | 13:15 |
| *** v1k0d3n has quit IRC | 13:18 | |
| *** daneyon has quit IRC | 13:18 | |
| *** salv-orl_ has quit IRC | 13:18 | |
| kbaegis | Really? | 13:19 |
| kbaegis | Soo | 13:19 |
| kbaegis | If I have 128G, how do I expand dockers limit? | 13:19 |
| *** bootsha has quit IRC | 13:20 | |
| kbaegis | I really wish I weren't stuck with centos. Linux 3.x doesn't have proper cgroup isolation | 13:20 |
| kbaegis | I think they fixed that in 4.6.x, so I don't think Xenial has it either | 13:21 |
| *** dwalsh has quit IRC | 13:21 | |
| *** dwalsh has joined #openstack-kolla | 13:22 | |
| *** v1k0d3n has joined #openstack-kolla | 13:23 | |
| openstackgerrit | Tin Lam proposed openstack/kolla: Add ansible role for vmtp container https://review.openstack.org/358305 | 13:25 |
| *** bootsha has joined #openstack-kolla | 13:26 | |
| *** bootsha has quit IRC | 13:31 | |
| sbezverk_ | sdake: ping | 13:40 |
| *** v1k0d3n has quit IRC | 13:43 | |
| sdake | sup sbezverk_ | 13:48 |
| *** dave-mccowan has quit IRC | 13:49 | |
| *** v1k0d3n has joined #openstack-kolla | 13:51 | |
| sbezverk_ | sdake: can you check this and express your opinion on this approach: http://paste.openstack.org/show/561784/ | 13:51 |
| sdake | sbezverk_ that sort of breaks the universal nature of the containers | 13:53 |
| sdake | unless that shell command comes from json | 13:54 |
| sdake | then wfm | 13:54 |
| sdake | the json abi taht is | 13:54 |
| sbezverk_ | sdake: unfortunately that is the only way ovs components work with stability | 13:54 |
| sdake | how does that shell command enter the system? | 13:55 |
| sdake | the reason we have the json is to add flexibility for others | 13:55 |
| sbezverk_ | they are defined in initialization continaer | 13:55 |
| sbezverk_ | well it is a very kube specific | 13:55 |
| sbezverk_ | abd it does not change kolla image | 13:56 |
| sdake | json doesn't hve to be standard across system | 13:56 |
| sdake | could hvae one set of json for k8s one set for ansible | 13:56 |
| sdake | what is an init container | 13:57 |
| sbezverk_ | sdake: I think you are missing the main point.. all commands which used to be run before starting the main process are removed from the main container | 13:57 |
| sbezverk_ | it seems all these activities like copying config files, doing some actions makes kube container less stable | 13:58 |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add gnocchi ansible role https://review.openstack.org/349351 | 13:59 |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add trove role https://review.openstack.org/354901 | 13:59 |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair gnocchi dockerfile https://review.openstack.org/348388 | 13:59 |
| sbezverk_ | init container is kind of a job which run once before the main container | 13:59 |
| *** salv-orlando has quit IRC | 14:01 | |
| *** salv-orlando has joined #openstack-kolla | 14:01 | |
| *** dwalsh has quit IRC | 14:03 | |
| *** v1k0d3n has quit IRC | 14:12 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair gnocchi dockerfile https://review.openstack.org/348388 | 14:16 |
| wznoinsk | sdake: here now | 14:18 |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair gnocchi dockerfile https://review.openstack.org/348388 | 14:22 |
| *** zhurong has quit IRC | 14:22 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add gnocchi ansible role https://review.openstack.org/349351 | 14:28 |
| sdake | wznoinsk yo | 14:41 |
| sdake | wznoinsk so nova_compute_ironic is only one container | 14:41 |
| sdake | and your right it would provide correct isolation for us | 14:42 |
| sdake | the issue is nova_compute_ironic and nova_compute communicate with other parts of the system | 14:42 |
| sdake | and the otherparts of the system expect one type or the toher | 14:42 |
| sdake | when jpeeler worked on it, the other parts of the system were neutron and nova proper | 14:42 |
| sdake | such as scheduler and conductor etc | 14:43 |
| sdake | what needs to happen is these other parts of the system need tobe made to work with both types of compute managers | 14:43 |
| *** daneyon has joined #openstack-kolla | 15:01 | |
| *** daneyon has quit IRC | 15:06 | |
| *** vbel has quit IRC | 15:21 | |
| *** vbel has joined #openstack-kolla | 15:21 | |
| *** senk has joined #openstack-kolla | 15:52 | |
| *** ad_rien_ has joined #openstack-kolla | 16:01 | |
| sdake | sup vbel | 16:06 |
| openstackgerrit | zhubingbing proposed openstack/kolla: fix sahara dockerfile https://review.openstack.org/351320 | 16:11 |
| openstackgerrit | zhubingbing proposed openstack/kolla: fix sahara dockerfile https://review.openstack.org/351320 | 16:16 |
| zhubingbing | sdake | 16:17 |
| zhubingbing | help review mycode | 16:17 |
| sdake | zhubingbing brain not warmed up enough to review just yet | 16:25 |
| zhubingbing | I'm sorry, I don't know today is Sunday. | 16:26 |
| zhubingbing | - - | 16:26 |
| sdake | np | 16:26 |
| sdake | wznoinsk ^^ | 16:27 |
| sdake | see timestamp :42 | 16:27 |
| zhubingbing | -） | 16:28 |
| zhubingbing | sadake,I saw rally have dockerfile, I intend to integrate the kolla inside the rally. How do you feel? | 16:29 |
| sdake | wfm | 16:32 |
| zhubingbing | sure | 16:33 |
| sdake | although i thought rally had to run outside of openstack | 16:33 |
| sdake | not inside of it | 16:33 |
| zhubingbing | wf | 16:36 |
| zhubingbing | wfm what mean? | 16:37 |
| *** daneyon has joined #openstack-kolla | 16:50 | |
| *** daneyon has quit IRC | 16:54 | |
| *** brad[] has quit IRC | 17:11 | |
| *** brad[] has joined #openstack-kolla | 17:13 | |
| *** Administrator__ has joined #openstack-kolla | 17:30 | |
| *** zhubingbing has quit IRC | 17:33 | |
| *** zhugaoxiao has quit IRC | 17:33 | |
| *** jrist has quit IRC | 17:52 | |
| sdake | wfm = works or me | 17:59 |
| sdake | for me | 17:59 |
| *** jrist has joined #openstack-kolla | 18:07 | |
| *** senk has quit IRC | 18:25 | |
| *** unicell1 has joined #openstack-kolla | 18:39 | |
| *** unicell has quit IRC | 18:39 | |
| *** salv-orl_ has joined #openstack-kolla | 19:16 | |
| *** salv-orlando has quit IRC | 19:19 | |
| *** gfhellma has joined #openstack-kolla | 19:28 | |
| *** wirehead_ has quit IRC | 19:28 | |
| *** wirehead_ has joined #openstack-kolla | 19:30 | |
| *** jtriley has joined #openstack-kolla | 19:42 | |
| *** jtriley has quit IRC | 19:48 | |
| *** jtriley has joined #openstack-kolla | 19:50 | |
| *** rhallisey has joined #openstack-kolla | 20:04 | |
| *** jtriley has quit IRC | 20:07 | |
| *** gfhellma has quit IRC | 20:12 | |
| sdake | sup wirehead_ | 20:46 |
| sdake | sup gang | 20:46 |
| sbezverk_ | sdake: feel like checking one strace? I am chasing l3 RTNETLINK issue | 20:57 |
| sdake | sbezverk_ can you give 5-10 mins - am writing an email | 20:57 |
| sdake | sbezverk_ don't want to forgt about it | 20:57 |
| sbezverk_ | sdake: absolutamente | 20:57 |
| openstackgerrit | Shaun Smekel proposed openstack/kolla: Customizations for MariaDB https://review.openstack.org/354422 | 21:03 |
| sdake | sbezverk_ ok | 21:17 |
| sdake | sbezverk_ wound me ;) | 21:17 |
| sdake | what is the rtnetlink return code? | 21:18 |
| sbezverk_ | invalid argement | 21:19 |
| sbezverk_ | oops argument | 21:19 |
| *** daneyon has joined #openstack-kolla | 21:20 | |
| sdake | better then econnrefussed :) | 21:21 |
| sdake | sbezverk_ can you paste the backtrace or did you want to use webex | 21:24 |
| sbezverk_ | one sec | 21:25 |
| *** daneyon has quit IRC | 21:25 | |
| *** dmueller-evoila has quit IRC | 21:26 | |
| sbezverk_ | sdake: interesting here is back trace http://paste.openstack.org/show/561985/ | 21:28 |
| sbezverk_ | after I updated version of flanneld it looks like it got a bot better. | 21:29 |
| sdake | not inspring :) | 21:30 |
| sdake | so not doing an rtnetlink error anymore? | 21:30 |
| sdake | i see it fails to apply iptables rules | 21:31 |
| sdake | but dont see any rtnetlink problems | 21:31 |
| sdake | is this with --net=host? | 21:32 |
| sbezverk_ | sdake: there is still rtnetlink error | 21:32 |
| sdake | i just grepped that log for RTNETLINK - nothing | 21:33 |
| sdake | grepped for invalid argument- nothing | 21:34 |
| sbezverk_ | man, so ip netns is working but from within a container | 21:34 |
| sbezverk_ | http://paste.openstack.org/show/561987/ | 21:34 |
| sbezverk_ | what is not working then is sharing of namespaces that is why when I was running the same command from the host it failed | 21:35 |
| sdake | what about outside the container | 21:35 |
| sbezverk_ | [root@k8s-4 ~]# ip netns exec qrouter-baa3335b-0013-42dd-856a-64a5c2557a01 ip a | 21:35 |
| sbezverk_ | RTNETLINK answers: Invalid argument | 21:35 |
| sbezverk_ | setting the network namespace "qrouter-baa3335b-0013-42dd-856a-64a5c2557a01" failed: Invalid argument | 21:35 |
| sbezverk_ | sorry for pasting it here | 21:35 |
| sdake | try ip netns list | 21:36 |
| sdake | or whatever the equivalent ommand is | 21:36 |
| sbezverk_ | outside? | 21:36 |
| sdake | yup | 21:36 |
| sdake | i want tot see if you have any routeers or whatnot | 21:36 |
| sbezverk_ | ip netns list | 21:36 |
| sbezverk_ | RTNETLINK answers: Invalid argument | 21:36 |
| sbezverk_ | RTNETLINK answers: Invalid argument | 21:36 |
| sbezverk_ | qrouter-baa3335b-0013-42dd-856a-64a5c2557a01 | 21:36 |
| sbezverk_ | so outside it sees it but cannot properly access | 21:37 |
| sbezverk_ | since it is missing shared flag on bind | 21:37 |
| sdake | yup that would be my guess as well | 21:38 |
| sdake | the vms you aer startingdon't have shared speciied in docker | 21:38 |
| sbezverk_ | but kube does not support this flag yet | 21:38 |
| sdake | in docker's ystemd | 21:38 |
| sdake | can you point me at the relevant shared flag we pass in the ansible code? | 21:38 |
| sbezverk_ | but it is less of a problem now, as seeing namesapce outside of container is not required | 21:38 |
| sdake | seeing namespace outside container is helpful - oeprators will want that | 21:39 |
| sbezverk_ | the only problem might be if other containers need accessign it like dhcp-agent for example | 21:39 |
| sdake | sbezverk_ i know of only one place shared is specified | 21:39 |
| sbezverk_ | to overcome this issue I would need to run l3 and dhcp in the same pod | 21:39 |
| sdake | and that is in systemd file in the virtual machines where minions are running | 21:39 |
| wznoinsk | sdake: hi, a long shot but I'd imagine if nova_compute_ironic would be in a separate UTS namespace (isolate two system identifiers) and attaching to something as virtual as linux bridge/ovs should work but right I did hear it was a problem from ppl on summits, I may have a chance to look into it | 21:40 |
| sdake | wznoinsk that went way over my head ;) | 21:41 |
| sdake | make it work flowers will fall from the sky :) | 21:41 |
| wznoinsk | I hope you hold on to your words ;-) | 21:42 |
| sdake | i could drop them out of a window ;) | 21:42 |
| *** imcsk8_ has joined #openstack-kolla | 21:42 | |
| sdake | sbezverk_ what shared flag are you talking about | 21:43 |
| sdake | you mean this: https://github.com/openstack/kolla/blob/master/ansible/roles/neutron/tasks/start.yml#L167 | 21:45 |
| wznoinsk | sbezverk_: is there anyway of enforcing this mount to be always shared via systemd for example? | 21:46 |
| *** imcsk8 has quit IRC | 21:46 | |
| wznoinsk | AFAIR it was to be shared on the host (?) | 21:46 |
| sdake | wznoinsk docker is the one that shares it | 21:46 |
| sdake | but wonder if its possible to hardcode shares as wznoinsk identified | 21:47 |
| sbezverk_ | sdake wznoinsk docker shares it when it is told to do so, the problem here is kube does not tell it to docker | 21:47 |
| sdake | sbezverk_ does that :shared flag go into a mount operation in some way? | 21:48 |
| sbezverk_ | sdake: doubt kube builds docker file on the fly | 21:48 |
| sdake | sbezverk_ - couple options to see if your theory is correct | 21:48 |
| sdake | if docker popens to the mount commnd | 21:49 |
| sdake | it would be possible to hack mount to always share | 21:49 |
| sdake | if docker accesses the mount systemcall directly, could hack docker to do the same thing | 21:49 |
| sbezverk_ | sdake: that is not the big issue as long as we run dhcp agent in the same pod as l3 | 21:49 |
| sdake | as in not sharing with the host the netns moutnpoints? | 21:50 |
| sbezverk_ | I will add dhcp to my version of l3 agent container and test | 21:50 |
| sbezverk_ | right | 21:50 |
| sdake | this is not optimal | 21:50 |
| sbezverk_ | I mean it is just inconvenient | 21:50 |
| sbezverk_ | but it will be fixed eventually | 21:50 |
| sbezverk_ | anyhow I this colocating l3 and dhcp is less intrisuive than hacking docker ;-) | 21:51 |
| sbezverk_ | let me test it and I will let you know | 21:51 |
| sbezverk_ | sdake: http://paste.openstack.org/show/561988/ | 21:52 |
| sdake | so seems to work? | 21:53 |
| sdake | no backtraces? | 21:53 |
| sbezverk_ | there is one more backtrace but now it is not fatal | 21:53 |
| sbezverk_ | something to do with iptables for ipv6 | 21:54 |
| sdake | are you sure its only ipv6? | 21:54 |
| sbezverk_ | sdake: I think so but please double check: http://paste.openstack.org/show/561989/ | 21:54 |
| sdake | see if you can run ip6tables-restore in the container as root | 22:06 |
| sdake | without all the ip netns exec | 22:07 |
| sbezverk_ | sorry I killed already pod | 22:08 |
| sbezverk_ | I am adding dhcp, shoudl be done soon | 22:08 |
| sdake | so you want a fat container for this? | 22:08 |
| sbezverk_ | no, not fat container | 22:11 |
| sbezverk_ | 1 process per container but in the pod we can have mul;tiple containers | 22:12 |
| sbezverk_ | we already use this approach | 22:12 |
| sdake | how will they share the namespace? | 22:12 |
| sbezverk_ | because within pod | 22:12 |
| rhallisey | pods share namespaces | 22:12 |
| sbezverk_ | containers share | 22:13 |
| sbezverk_ | rhallisey hey man | 22:13 |
| rhallisey | hey | 22:13 |
| sdake | all namespaces? | 22:13 |
| sbezverk_ | sdake yep | 22:13 |
| sbezverk_ | well, in theory | 22:14 |
| sdake | wonder how they get docker to do thta | 22:14 |
| rhallisey | ya | 22:14 |
| sbezverk_ | I am about to test it | 22:14 |
| sdake | i find it super hard to believe that a pod can share the mount namesapce | 22:18 |
| sdake | which is where the files re that access the kernel | 22:19 |
| sdake | Applications within a pod also have access to shared volumes, which are defined as part of a pod and are made available to be mounted into each application’s filesystem. | 22:26 |
| sdake | from http://kubernetes.io/docs/user-guide/pods/ | 22:26 |
| sbezverk_ | sdake: DAMN :-( | 22:26 |
| sbezverk_ | it sees it but cannot access the same way as from the host | 22:26 |
| sdake | yup - easy solution - use a named volume | 22:27 |
| sdake | oh you said the files can be seen | 22:27 |
| sdake | so the way this works sbezverk_ | 22:27 |
| sdake | you are taking the namesapce tuff out of the host entirely | 22:28 |
| sdake | and putting it into he container namespace | 22:28 |
| sbezverk_ | sdake: we already have named volume mapped to host /run/netsh | 22:28 |
| sdake | the reason we have it on the host is for tools like tcpdump and ip to work properly | 22:28 |
| sdake | if you have named volulme mapped to /run/netns how did you expect the host binddmounts to work? | 22:29 |
| sbezverk_ | sdake: my mistake we have bind mount | 22:30 |
| sdake | where is your bindd mount to? | 22:30 |
| sdake | the host? | 22:30 |
| sbezverk_ | yes | 22:31 |
| sdake | what is a "shared volume" | 22:31 |
| sbezverk_ | if I do ls /run/netns on the host and on both containers I see the same info | 22:31 |
| sdake | use this: | 22:32 |
| sdake | http://kubernetes.io/docs/user-guide/volumes/ | 22:32 |
| sdake | instead of hte bindmounts for /run/netns | 22:32 |
| sdake | wow kubernetes has come a long way | 22:35 |
| sdake | so sbezverk_ use emptyDir volume type or /run/netns | 22:35 |
| sbezverk_ | sdake: I do use volume in pod definition | 22:35 |
| sdake | i dont know the syntax of the volume type | 22:36 |
| sbezverk_ | which is mapped to host's directory | 22:36 |
| sbezverk_ | I cannot use emptydir | 22:36 |
| sdake | why not? | 22:36 |
| sbezverk_ | as then host will not see it | 22:36 |
| sdake | one step at a time | 22:36 |
| sdake | goal is to get something working | 22:37 |
| sdake | not have it perfect | 22:37 |
| sdake | if k8s has gaps, we can ge thtose fixed down the road | 22:37 |
| sdake | i tink just 4 things need access to this emptydir | 22:38 |
| sdake | the neutron agents specifically | 22:38 |
| sdake | and ovs iirc | 22:38 |
| sdake | whatever the 4 things are related to networking :) | 22:38 |
| sdake | the pod needs to include these for networking things | 22:39 |
| sdake | wish i remembered what they were | 22:39 |
| sdake | four | 22:39 |
| sdake | how /query rhallisey | 22:39 |
| sdake | query rhallisey | 22:39 |
| sdake | grn | 22:39 |
| *** imcsk8_ is now known as imcsk8 | 22:40 | |
| *** salv-orl_ has quit IRC | 23:02 | |
| *** daneyon has joined #openstack-kolla | 23:09 | |
| *** daneyon has quit IRC | 23:13 | |
| *** rhallisey has quit IRC | 23:30 | |
| *** sdake has quit IRC | 23:34 | |
| *** bmace has quit IRC | 23:49 | |
| *** bmace has joined #openstack-kolla | 23:49 | |
| *** zhurong has joined #openstack-kolla | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!