*** rkrum has joined #openstack-kolla | 00:00 | |
*** zhurong has quit IRC | 00:06 | |
*** dmueller-evoila has joined #openstack-kolla | 00:23 | |
*** dmueller-evoila has quit IRC | 00:28 | |
*** fragatina has joined #openstack-kolla | 00:43 | |
*** duonghq has joined #openstack-kolla | 00:45 | |
*** dave-mccowan has joined #openstack-kolla | 01:02 | |
*** dave-mccowan has quit IRC | 01:02 | |
*** salv-orlando has joined #openstack-kolla | 01:06 | |
*** salv-orlando has quit IRC | 01:08 | |
*** zhurong has joined #openstack-kolla | 01:13 | |
*** HyperJohnGraham has quit IRC | 01:35 | |
*** fragatin_ has joined #openstack-kolla | 01:43 | |
*** fragatina has quit IRC | 01:47 | |
*** salv-orlando has joined #openstack-kolla | 02:16 | |
*** salv-orlando has quit IRC | 02:19 | |
*** Jeffrey4l has joined #openstack-kolla | 02:24 | |
*** daneyon has joined #openstack-kolla | 02:46 | |
*** yuanying has quit IRC | 02:46 | |
*** daneyon has quit IRC | 02:50 | |
*** yuanying has joined #openstack-kolla | 02:50 | |
*** kbaegis has quit IRC | 03:12 | |
openstackgerrit | Duong Ha-Quang proposed openstack/kolla: [wip] Add Ansible become to quick start guide https://review.openstack.org/358374 | 03:12 |
---|---|---|
*** chrichip has quit IRC | 03:23 | |
*** chrichip has joined #openstack-kolla | 03:23 | |
*** salv-orlando has joined #openstack-kolla | 03:26 | |
*** salv-orlando has quit IRC | 03:33 | |
*** HyperJohnGraham has joined #openstack-kolla | 03:40 | |
*** ayoung has quit IRC | 03:43 | |
*** yuanying has quit IRC | 03:45 | |
*** yuanying has joined #openstack-kolla | 03:48 | |
openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Dockerfiles for footer related changes https://review.openstack.org/357746 | 03:52 |
openstackgerrit | Tin Lam proposed openstack/kolla: Fix typo in iscsi pull playbook https://review.openstack.org/358390 | 04:07 |
openstackgerrit | Hui Kang proposed openstack/kolla: Add etcd container https://review.openstack.org/355156 | 04:10 |
duonghq | Pavo: are you there? | 04:12 |
openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Vagrant doc updates https://review.openstack.org/353240 | 04:23 |
*** michauds has joined #openstack-kolla | 04:34 | |
*** salv-orlando has joined #openstack-kolla | 04:36 | |
*** david-lyle has quit IRC | 04:37 | |
*** salv-orlando has quit IRC | 04:43 | |
*** diga has joined #openstack-kolla | 04:44 | |
*** fragatina has joined #openstack-kolla | 05:20 | |
*** fragatin_ has quit IRC | 05:20 | |
*** aswadr_ has joined #openstack-kolla | 05:22 | |
*** HyperJohnGraham has quit IRC | 05:39 | |
*** diga has quit IRC | 05:44 | |
*** cloudnull has quit IRC | 05:46 | |
*** irtermite has quit IRC | 05:46 | |
*** salv-orlando has joined #openstack-kolla | 05:46 | |
*** salv-orlando has quit IRC | 05:54 | |
*** duonghq has quit IRC | 05:57 | |
*** caowei has joined #openstack-kolla | 05:58 | |
*** duonghq has joined #openstack-kolla | 05:59 | |
*** senk has joined #openstack-kolla | 06:01 | |
*** salv-orlando has joined #openstack-kolla | 06:07 | |
*** cloudnull has joined #openstack-kolla | 06:07 | |
*** michauds has quit IRC | 06:15 | |
*** coolsvap has joined #openstack-kolla | 06:15 | |
*** coolsvap has quit IRC | 06:16 | |
*** coolsvap has joined #openstack-kolla | 06:16 | |
*** zhiwei has joined #openstack-kolla | 06:25 | |
*** ad_rien_ has quit IRC | 06:31 | |
openstackgerrit | Duong Ha-Quang proposed openstack/kolla: Check if ansible_user is in docker group https://review.openstack.org/358432 | 06:57 |
*** ad_rien_ has joined #openstack-kolla | 07:01 | |
*** duonghq has quit IRC | 07:02 | |
*** Mr_Broken has joined #openstack-kolla | 07:02 | |
*** egonzalez90 has joined #openstack-kolla | 07:03 | |
egonzalez90 | morning o/ | 07:03 |
*** zhurong has quit IRC | 07:09 | |
*** zhurong has joined #openstack-kolla | 07:13 | |
*** narasimha_SV has joined #openstack-kolla | 07:14 | |
openstackgerrit | Duong Ha-Quang proposed openstack/kolla: Check if ansible_user is in docker group https://review.openstack.org/358432 | 07:15 |
narasimha_SV | after adding dockerfile for ironic to create /var/lib/ironic folder with ironic user permission | 07:15 |
narasimha_SV | still I see that folder is not getting created. | 07:15 |
*** salv-orl_ has joined #openstack-kolla | 07:15 | |
narasimha_SV | i tried by editing in ironic-base and also ironic-conductor docker files | 07:15 |
*** duonghq has joined #openstack-kolla | 07:16 | |
*** daneyon has joined #openstack-kolla | 07:16 | |
*** salv-orlando has quit IRC | 07:18 | |
*** bootsha has joined #openstack-kolla | 07:19 | |
*** daneyon has quit IRC | 07:21 | |
duonghq | Pavo, kbaegis: anybody there? | 07:26 |
*** b_bezak has joined #openstack-kolla | 07:30 | |
*** salv-orl_ has quit IRC | 07:32 | |
*** berendt has joined #openstack-kolla | 07:33 | |
*** rkrum has quit IRC | 07:34 | |
*** bootsha has quit IRC | 07:36 | |
*** narasimha_SV has quit IRC | 07:37 | |
egonzalez90 | narasimha_SV, PS link? | 07:47 |
openstackgerrit | Jeffrey Zhang proposed openstack/kolla: DO NOT MERGE: TEST UBUNTU MULTIGATE https://review.openstack.org/326307 | 08:06 |
*** matrohon has joined #openstack-kolla | 08:06 | |
openstackgerrit | Wei Cao proposed openstack/kolla: Add solum container https://review.openstack.org/355408 | 08:07 |
*** gfidente has joined #openstack-kolla | 08:08 | |
*** athomas has joined #openstack-kolla | 08:10 | |
*** salv-orlando has joined #openstack-kolla | 08:14 | |
*** coolsvap is now known as _coolsvap_ | 08:18 | |
*** bootsha has joined #openstack-kolla | 08:26 | |
*** Serlex has joined #openstack-kolla | 08:31 | |
*** yuanying has quit IRC | 08:34 | |
*** yuanying has joined #openstack-kolla | 08:35 | |
*** papacz has quit IRC | 08:35 | |
*** Mr_Broken has quit IRC | 08:42 | |
*** Mr_Broken has joined #openstack-kolla | 08:43 | |
*** yuanying has quit IRC | 08:47 | |
*** qbjolof has quit IRC | 08:48 | |
*** qbjolof has joined #openstack-kolla | 08:48 | |
*** yuanying has joined #openstack-kolla | 08:49 | |
openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix Ubuntu binary build and deploy gate https://review.openstack.org/358488 | 08:59 |
*** daneyon has joined #openstack-kolla | 09:05 | |
*** daneyon has quit IRC | 09:09 | |
*** Guest_48844 has joined #openstack-kolla | 09:18 | |
*** pbourke has joined #openstack-kolla | 09:18 | |
*** bootsha has quit IRC | 09:20 | |
*** bootsha has joined #openstack-kolla | 09:23 | |
*** pbourke has quit IRC | 09:25 | |
*** athomas has quit IRC | 09:29 | |
*** athomas has joined #openstack-kolla | 09:34 | |
openstackgerrit | Wei Cao proposed openstack/kolla: Add solum container https://review.openstack.org/355408 | 09:48 |
*** Guest_48844 has quit IRC | 09:49 | |
*** Guest_48844 has joined #openstack-kolla | 09:51 | |
Guest_48844 | allah is doing | 09:51 |
Guest_48844 | sun is not doing allah is doing | 09:51 |
Guest_48844 | moon is not doing allah is doing | 09:51 |
Guest_48844 | stars are not doing allah is doing | 09:51 |
berendt | Guest_48844 go ahead | 09:51 |
Guest_48844 | planets are not doing allah is doing | 09:51 |
Guest_48844 | galaxies are not doing allah is doing | 09:52 |
Guest_48844 | oceans are not doing allah is doing | 09:52 |
Guest_48844 | mountains are not doing allah is doing | 09:52 |
Guest_48844 | trees are not doing allah is doing | 09:52 |
Guest_48844 | mom is not doing allah is doing | 09:52 |
Guest_48844 | dad is not doing allah is doing | 09:53 |
Guest_48844 | boss is not doing allah is doing | 09:53 |
Guest_48844 | job is not doing allah is doing | 09:53 |
Guest_48844 | dollar is not doing allah is doing | 09:53 |
Guest_48844 | degree is not doing allah is doing | 09:53 |
Guest_48844 | medicine is not doing allah is doing | 09:53 |
Guest_48844 | customers are not doing allah is doing | 09:53 |
Guest_48844 | you can not get a job without the permission of allah | 09:54 |
Guest_48844 | you can not get married without the permission of allah | 09:54 |
Guest_48844 | nobody can get angry at you without the permission of allah | 09:55 |
Guest_48844 | light is not doing allah is doing | 09:55 |
Guest_48844 | fan is not doing allah is doing | 09:55 |
Guest_48844 | businessess are not doing allah is doing | 09:55 |
Guest_48844 | america is not doing allah is doing | 09:56 |
berendt | sombody here who can kick the spammer? it is annoying... | 09:56 |
Guest_48844 | fire can not burn without the permission of allah | 09:56 |
Guest_48844 | knife can not cut without the permission of allah | 09:56 |
Guest_48844 | rulers are not doing allah is doing | 09:56 |
Guest_48844 | governments are not doing allah is doing | 09:57 |
Guest_48844 | sleep is not doing allah is doing | 09:57 |
Guest_48844 | hunger is not doing allah is doing | 09:57 |
Guest_48844 | food does not take away the hunger allah takes away the hunger | 09:57 |
Guest_48844 | water does not take away the thirst allah takes away the thirst | 09:57 |
Guest_48844 | seeing is not doing allah is doing | 09:58 |
Guest_48844 | hearing is not doing allah is doing | 09:58 |
Guest_48844 | seasons are not doing allah is doing | 09:58 |
*** rkrum has joined #openstack-kolla | 09:58 | |
Guest_48844 | weather is not doing allah is doing | 09:58 |
egonzalez90 | coolsvap ping | 09:58 |
Guest_48844 | humans are not doing allah is doing | 09:58 |
Guest_48844 | animals are not doing allah is doing | 09:58 |
*** zhurong has quit IRC | 09:58 | |
Guest_48844 | the best amongst you are those who learn and then teach quran | 09:59 |
Guest_48844 | one letter read from book of allah amounts to one good deed and allah multiplies one good deed ten times | 10:00 |
Guest_48844 | hearts get rusted as does iron with water to remove rust from heart recitation of quran and rememberance of death | 10:00 |
Guest_48844 | heart is likened to a mirror | 10:01 |
Guest_48844 | when a person commits one sin a black dot sustains the heart | 10:01 |
*** rkrum has quit IRC | 10:02 | |
Guest_48844 | to accept islam say that i bear witness that there is no deity worthy of worship except allah and muhammad peace be upon him is his slave and messenger | 10:03 |
Guest_48844 | read book http://www.fazaileamaal.com | 10:04 |
Guest_48844 | read book http://www.muntakhabahadith.com | 10:04 |
Guest_48844 | need spiritual teacher visit http://www.alhaadi.org.za | 10:05 |
egonzalez90 | berendt, just mark as ignored until some op can kick him | 10:05 |
berendt | egonzalez90 this way the spam is still in the channel logs | 10:05 |
Guest_48844 | allah created the sky without any pillars | 10:05 |
berendt | I wrote a mail to one of the guys from the freenode staff team, maybe they can help. | 10:06 |
*** FireFly has joined #openstack-kolla | 10:06 | |
berendt | FireFly thanks for joining, it is Guest_48844 ... | 10:06 |
*** Guest_48844 has quit IRC | 10:06 | |
FireFly | Well | 10:06 |
berendt | FireFly looks good. thank you! | 10:07 |
FireFly | I didn't even do anything :D | 10:07 |
FireFly | Feel free to poke me if they return | 10:07 |
berendt | Hmm. I thought you kicked him :) | 10:07 |
berendt | Maybe it is sufficient to highlight the bot to kick him.. | 10:07 |
FireFly | Judging by the nickname.. was it one of them 'allah' spammers? | 10:08 |
FireFly | or something else? | 10:08 |
berendt | yes, "humans are not doing allah is doing" ... | 10:08 |
FireFly | Right | 10:12 |
*** pbourke_ has joined #openstack-kolla | 10:18 | |
pbourke_ | #openstack-ekko | 10:18 |
openstackgerrit | Duong Ha-Quang proposed openstack/kolla: Specify 'become' to tasks in common and destroy roles https://review.openstack.org/358539 | 10:18 |
*** kbaegis has joined #openstack-kolla | 10:19 | |
*** pbourke_ is now known as pbourke | 10:19 | |
*** duonghq has quit IRC | 10:20 | |
*** bootsha has quit IRC | 10:22 | |
*** bootsha has joined #openstack-kolla | 10:23 | |
*** imcsk8 has quit IRC | 10:35 | |
*** imcsk8 has joined #openstack-kolla | 10:35 | |
*** ad_rien_1 has joined #openstack-kolla | 10:40 | |
*** ad_rien_ has quit IRC | 10:40 | |
*** imcsk8_ has joined #openstack-kolla | 10:48 | |
*** imcsk8 has quit IRC | 10:51 | |
*** daneyon has joined #openstack-kolla | 10:53 | |
*** nick-ma has quit IRC | 10:54 | |
*** nick-ma has joined #openstack-kolla | 10:54 | |
*** daneyon has quit IRC | 10:57 | |
*** openstackgerrit has quit IRC | 11:03 | |
*** openstackgerrit has joined #openstack-kolla | 11:03 | |
openstackgerrit | Wei Cao proposed openstack/kolla: Add solum container https://review.openstack.org/355408 | 11:06 |
*** bootsha has quit IRC | 11:10 | |
*** FireFly has left #openstack-kolla | 11:10 | |
*** bootsha has joined #openstack-kolla | 11:15 | |
*** fragatina has quit IRC | 11:15 | |
*** ajpower has quit IRC | 11:16 | |
*** fragatina has joined #openstack-kolla | 11:16 | |
*** afranc has quit IRC | 11:17 | |
*** imcsk8_ has quit IRC | 11:18 | |
*** afranc has joined #openstack-kolla | 11:18 | |
*** imcsk8 has joined #openstack-kolla | 11:19 | |
*** salv-orlando has quit IRC | 11:19 | |
*** salv-orlando has joined #openstack-kolla | 11:21 | |
*** ajpower has joined #openstack-kolla | 11:21 | |
*** salv-orl_ has joined #openstack-kolla | 11:22 | |
*** Guest52040 has joined #openstack-kolla | 11:23 | |
*** salv-orlando has quit IRC | 11:25 | |
*** zhubingbing has joined #openstack-kolla | 11:27 | |
*** zhiwei has quit IRC | 11:29 | |
*** openstackstatus has quit IRC | 11:36 | |
*** openstackstatus has joined #openstack-kolla | 11:38 | |
*** ChanServ sets mode: +v openstackstatus | 11:38 | |
*** bootsha has quit IRC | 11:39 | |
*** sean-k-mooneyAFK is now known as sean-k-mooney | 11:42 | |
*** bootsha has joined #openstack-kolla | 11:43 | |
*** kbaegis has quit IRC | 11:43 | |
*** kbaegis has joined #openstack-kolla | 11:45 | |
*** salv-orl_ has quit IRC | 11:53 | |
*** salv-orlando has joined #openstack-kolla | 11:54 | |
*** kbaegis1 has joined #openstack-kolla | 11:56 | |
*** kbaegis1 has quit IRC | 11:56 | |
*** kbaegis2 has joined #openstack-kolla | 11:56 | |
*** kbaegis has quit IRC | 11:56 | |
openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix Ubuntu binary build and deploy gate https://review.openstack.org/358488 | 12:00 |
*** dave-mccowan has joined #openstack-kolla | 12:02 | |
*** ad_rien_1 has quit IRC | 12:07 | |
*** ad_rien_ has joined #openstack-kolla | 12:08 | |
*** Jeffrey4l_ has joined #openstack-kolla | 12:15 | |
*** Jeffrey4l has quit IRC | 12:16 | |
*** zhubingbing has quit IRC | 12:28 | |
*** caowei has quit IRC | 12:28 | |
*** godleon has joined #openstack-kolla | 12:30 | |
*** srwilkers_ has joined #openstack-kolla | 12:33 | |
*** rhallisey has joined #openstack-kolla | 12:37 | |
*** daneyon has joined #openstack-kolla | 12:41 | |
*** daneyon has quit IRC | 12:46 | |
*** srwilkers_ has quit IRC | 12:49 | |
*** salv-orlando has quit IRC | 12:50 | |
*** dwalsh has joined #openstack-kolla | 12:51 | |
*** dwalsh has quit IRC | 12:56 | |
*** ad_rien_ has quit IRC | 12:57 | |
*** egonzalez90 has quit IRC | 12:58 | |
*** ad_rien_ has joined #openstack-kolla | 12:58 | |
*** mgoddard has joined #openstack-kolla | 13:00 | |
*** srwilkers_ has joined #openstack-kolla | 13:02 | |
*** srwilkers_ has left #openstack-kolla | 13:02 | |
openstackgerrit | Prithiv proposed openstack/kolla: Dockerfiles for collectd-ceilometer-plugin Container https://review.openstack.org/358632 | 13:09 |
*** jtriley has joined #openstack-kolla | 13:10 | |
*** Guest37793 is now known as zeus | 13:24 | |
*** zeus has joined #openstack-kolla | 13:24 | |
*** dmueller-evoila has joined #openstack-kolla | 13:26 | |
*** dmueller-evoila has quit IRC | 13:26 | |
*** Guest52040 has quit IRC | 13:30 | |
*** papacz has joined #openstack-kolla | 13:30 | |
*** dwalsh has joined #openstack-kolla | 13:33 | |
*** sould has joined #openstack-kolla | 13:38 | |
*** sould has quit IRC | 13:45 | |
*** inc0 has joined #openstack-kolla | 13:46 | |
*** sdake has joined #openstack-kolla | 13:46 | |
inc0 | good morning | 13:47 |
inc0 | hey sdake, still in the land of living? | 13:47 |
*** dwalsh has quit IRC | 13:47 | |
pbourke | hey inc0 | 13:47 |
sdake | morning peeps | 13:47 |
sdake | me sure | 13:47 |
sdake | my friend not so much | 13:47 |
inc0 | hey Paul, I failed at rally:/ | 13:48 |
sdake | only going tto be in a little bit today | 13:48 |
sdake | wanted to ssee where osic si | 13:48 |
pbourke | inc0: where'd you get to? | 13:49 |
inc0 | I ran stuff, made ~/root/rally-results | 13:50 |
*** sdake_ has joined #openstack-kolla | 13:50 | |
inc0 | redeployed with ceph afterwards | 13:50 |
inc0 | but rally shown lots of errors | 13:50 |
inc0 | rally task list -> tons of fails | 13:50 |
pbourke | did it work before you deployed with ceph? | 13:51 |
inc0 | not sure, I ran tasks outputted them to file and happily moved on | 13:52 |
inc0 | only afterwards Ive noticed something is off | 13:52 |
inc0 | but issues sounded keystone-ish | 13:53 |
inc0 | look at tasks error log | 13:53 |
pbourke | there are a handful that fail because no cinder | 13:53 |
*** sdake has quit IRC | 13:53 | |
inc0 | I did deploy cinder | 13:53 |
sdake_ | inc0 which email | 13:54 |
pbourke | inc0: looking now | 13:56 |
pbourke | inc0: unrelated but, have you ever seen neutron-openvswitch-agent having permission trouble accessing /var/run/openvswitch/db.sock | 13:56 |
inc0 | no | 13:57 |
pbourke | inc0: as when you look at it that file is owned by root but the container is running as neutron | 13:57 |
inc0 | I know sbezverk_ seen this while porting neutron to k8s | 13:57 |
inc0 | however I am baffled | 13:57 |
pbourke | i cant even find where this is bind mounted despite it showing up on the host filesystem | 13:58 |
pbourke | inc0: anyway, it seems almost all rally failures are down the image name in glance | 13:58 |
*** sould has joined #openstack-kolla | 13:58 | |
pbourke | inc0: dont see anything related to keystone | 13:59 |
inc0 | I added uec image | 13:59 |
*** mgoddard_ has joined #openstack-kolla | 13:59 | |
pbourke | im not seeing it... | 14:00 |
*** sould has quit IRC | 14:01 | |
sbezverk_ | inc0: remember we had similar discussion last week about the socket? | 14:02 |
*** mgoddard has quit IRC | 14:03 | |
pbourke | sbezverk_: what was the summary | 14:04 |
sdake_ | pbourke did the osic cluster suffer this permissions problem for ovs? | 14:04 |
pbourke | inc0: did you archive the results for the first scenario somewhere | 14:04 |
pbourke | sdake_: no osic is ok | 14:05 |
pbourke | sdake_: its in a deployment i have im seeing it | 14:05 |
inc0 | pbourke, yes, first dir in ~/root/rally-results | 14:05 |
sbezverk_ | pbourke: I noticed that neutron-ovs-agent container runs with use neutron, but ovsdb as root | 14:05 |
sbezverk_ | so the socket gets created with only root priv | 14:06 |
pbourke | sbezverk_: yeah. so its strange it ever works | 14:06 |
sbezverk_ | pbourke: exactly was my qurestion to inc0 | 14:06 |
*** JoseMello has joined #openstack-kolla | 14:06 | |
sbezverk_ | there is a theory that it worked because it was done via rootwrap | 14:06 |
pbourke | sbezverk_: actually yeah I have something hacked around rootwrap let me check. thanks | 14:09 |
*** kbaegis2 has quit IRC | 14:11 | |
*** cinerama has quit IRC | 14:11 | |
pbourke | inc0: going to fix up the image name and re-run for ceph | 14:11 |
*** haplo37__ has joined #openstack-kolla | 14:11 | |
inc0 | ok | 14:11 |
sdake_ | berendt around? | 14:13 |
berendt | sdake_ jup | 14:13 |
sdake_ | sup | 14:13 |
pbourke | sbezverk_: rootwrap fixed it! | 14:13 |
sdake_ | so re irc permissions | 14:13 |
sdake_ | wfm - i dont care if folks that are trusted have operator permissions | 14:13 |
sdake_ | but we dont run with operator permissions in channel in general | 14:14 |
berendt | the spam bot was just annoying and it would be nice if it is possible to kick/ban in such a situation | 14:15 |
sdake_ | right | 14:15 |
sdake_ | just setting groundrules :) | 14:15 |
sbezverk_ | pbourke: great, you confirmed the theory then :-) | 14:16 |
pbourke | inc0: how did you generate the json/html | 14:17 |
pbourke | inc0: seems we only have stats for one task | 14:17 |
*** sdake_ is now known as sdake | 14:18 | |
*** ChanServ sets mode: +o sdake | 14:18 | |
sdake | [07:18:13] <sdake>flags #openstack-kolla berendt +o | 14:18 |
sdake | [07:18:14] -ChanServ-You are not authorized to execute this command. | 14:18 |
*** ad_rien_ has left #openstack-kolla | 14:19 | |
*** bootsha has quit IRC | 14:20 | |
sbezverk_ | sdake: I keeplooking at rtnetlink problem with namespaces and noticed that namespaces do not get mounted on the host, and only one dhcp namespace got mounted inside of dhcp container. do you know at what point namespace gets mounted? | 14:25 |
*** berendt has quit IRC | 14:25 | |
*** zhiwei has joined #openstack-kolla | 14:26 | |
*** berendt has joined #openstack-kolla | 14:26 | |
*** zhiwei has quit IRC | 14:30 | |
sdake | berendt try /msg chanserv #kolla op berendt | 14:33 |
*** bootsha has joined #openstack-kolla | 14:33 | |
*** cinerama has joined #openstack-kolla | 14:34 | |
pbourke | inc0: can you confirm the command(s) you used to generate the report for deploy #1 (without ceph) | 14:36 |
*** ChanServ sets mode: +o berendt | 14:36 | |
berendt | sdake /msg chanserv OP #openstack-kolla berendt worked. thanks. | 14:37 |
*** ChanServ sets mode: -o berendt | 14:37 | |
*** ChanServ sets mode: +o berendt | 14:38 | |
*** ChanServ sets mode: +o berendt | 14:38 | |
*** ayoung has joined #openstack-kolla | 14:41 | |
*** b_bezak has quit IRC | 14:48 | |
*** senk has quit IRC | 14:49 | |
*** bootsha has quit IRC | 14:53 | |
*** salv-orlando has joined #openstack-kolla | 14:54 | |
*** salv-orlando has quit IRC | 14:57 | |
*** ayoung has quit IRC | 14:59 | |
*** HyperJohnGraham has joined #openstack-kolla | 15:02 | |
*** sould has joined #openstack-kolla | 15:06 | |
*** sould has quit IRC | 15:12 | |
*** Jeffrey4l_ has quit IRC | 15:13 | |
*** mgoddard has joined #openstack-kolla | 15:14 | |
*** mgoddard_ has quit IRC | 15:15 | |
pbourke | latest openstackclient seems to break kolla for me | 15:21 |
*** vhosakot has joined #openstack-kolla | 15:21 | |
pbourke | wonder what our gates will think | 15:21 |
*** vbel has quit IRC | 15:21 | |
*** vbel has joined #openstack-kolla | 15:22 | |
*** Jeffrey4l_ has joined #openstack-kolla | 15:22 | |
*** daneyon has joined #openstack-kolla | 15:23 | |
*** berendt has quit IRC | 15:24 | |
*** _coolsvap_ is now known as coolsvap | 15:24 | |
sdake | was going to climb humphreys today | 15:27 |
sdake | and woke up with terible backache | 15:27 |
*** daneyon has quit IRC | 15:28 | |
inc0 | sdake, vodka is good medicine for that | 15:30 |
inc0 | also makes trip more interesting | 15:31 |
sdake | inc0 this is humprehys: http://www.allflagstaff.com/mountains/mt_humphreys.php | 15:34 |
sdake | 3 hrs my ass | 15:35 |
sdake | more like 5 up 7 down | 15:35 |
inc0 | looks great | 15:36 |
sdake | when i was in perfect shape i could do humphreys in about 3.5 hours | 15:36 |
*** Administrator__ has quit IRC | 15:36 | |
inc0 | 9 miles total n 3 hrs? fuck. I don't know who made this measurement | 15:37 |
*** Administrator__ has joined #openstack-kolla | 15:37 | |
inc0 | and it's mountains | 15:37 |
sdake | with the last mile anklebreakers everywhere | 15:37 |
sdake | i can do 10 miles in 3 hours over flat terrain | 15:37 |
sdake | but this is straight up then straight back down | 15:38 |
inc0 | pbourke, sorry I was on call | 15:39 |
inc0 | did you move anything forward? | 15:39 |
*** harlowja_at_home has joined #openstack-kolla | 15:40 | |
*** matrohon has quit IRC | 15:42 | |
jrich523_ | sdake: so if its half up and half down, shouldnt the time even out? :) | 15:44 |
*** jrich523_ is now known as jrich523 | 15:44 | |
*** aNupoisc has joined #openstack-kolla | 15:45 | |
sdake | jrich523 going down is harder then up | 15:45 |
sdake | in many domains - not just hiking :) | 15:46 |
jrich523 | yeah, im mostly just giving you crap, i use to hike a lot, and it always felt that would be better, until you start to come down :) | 15:46 |
jrich523 | lot of stress on your feet to maintain speed/control | 15:46 |
sdake | and knees | 15:47 |
jrich523 | yeah, those too, as i get older the more they hurt | 15:47 |
sdake | i'm 42 - not sure if i can hike this mountain anymore | 15:47 |
jrich523 | so lets see, you're all banged up, some feet/knee problems and tooth geesh, rough weekend :) | 15:47 |
jrich523 | im 36.. im not far behind you | 15:47 |
jrich523 | my hikes have gotten easier over the years lol | 15:48 |
jrich523 | hmm is there anything stopping you from using a top level domain for internal DNS? | 15:49 |
jrich523 | i feel like there isnt anything stopping you from doing system.mydomain | 15:49 |
*** jemcevoy has joined #openstack-kolla | 15:49 | |
*** michauds has joined #openstack-kolla | 15:50 | |
*** sdake has quit IRC | 15:54 | |
*** sdake has joined #openstack-kolla | 15:55 | |
*** Jeffrey4l_ has quit IRC | 16:00 | |
*** mgoddard_ has joined #openstack-kolla | 16:01 | |
pbourke | inc0: I just want to confirm what you've done | 16:02 |
*** dgonzalez has quit IRC | 16:03 | |
*** bradjones has quit IRC | 16:03 | |
pbourke | inc0: so I dont do anything unnecessary | 16:03 |
pbourke | inc0: it seems you generate the reports for just the one task rather than all in the rally deployment? | 16:03 |
inc0 | nah | 16:04 |
inc0 | I generated for all of them | 16:04 |
*** mgoddard has quit IRC | 16:04 | |
*** ssurana has joined #openstack-kolla | 16:04 | |
*** dgonzalez has joined #openstack-kolla | 16:05 | |
*** bradjones has joined #openstack-kolla | 16:06 | |
*** bradjones has quit IRC | 16:06 | |
*** bradjones has joined #openstack-kolla | 16:06 | |
pbourke | inc0: ~/rally-results/19-08-4pmCST/output.html only contains one task | 16:06 |
inc0 | ahh | 16:07 |
inc0 | yeah in this one | 16:07 |
inc0 | in 5pm I learned my lesson;) | 16:07 |
inc0 | I'm in tmux | 16:07 |
*** dave-mccowan has quit IRC | 16:08 | |
*** dave-mcc_ has joined #openstack-kolla | 16:08 | |
pbourke | actually its no big deal as we still have the info saved in the rally db | 16:09 |
pbourke | so you just need to remember which rally deployment it was | 16:09 |
inc0 | so i probably ran on same deployment | 16:10 |
pbourke | the kolla-rally-tests.sh script creates a new one each time | 16:10 |
pbourke | are we on different tmuxes? | 16:11 |
inc0 | ok | 16:11 |
*** salv-orlando has joined #openstack-kolla | 16:11 | |
*** wirehead_ has quit IRC | 16:11 | |
*** wirehead_ has joined #openstack-kolla | 16:12 | |
pbourke | inc0: can you grab all reports from your previous deploy before we re-do the ceph ones? | 16:13 |
*** Jeffrey4l_ has joined #openstack-kolla | 16:14 | |
*** egonzalez90 has joined #openstack-kolla | 16:18 | |
*** fragatina has quit IRC | 16:19 | |
*** fragatina has joined #openstack-kolla | 16:19 | |
sbezverk_ | sdake: I can ping between dhcp agent and neutron router over ovs bridge :-) | 16:21 |
*** vhosakot has quit IRC | 16:24 | |
kfox1111 | sbezverk_: awesome. :) | 16:24 |
sbezverk_ | kfox1111: do not think so, if you see my container you will do -100 ;-) | 16:25 |
*** vhosakot has joined #openstack-kolla | 16:25 | |
kfox1111 | hehe. | 16:25 |
kfox1111 | well, its a step in the right direction. :) | 16:25 |
kfox1111 | cleanup next. :) | 16:25 |
sbezverk_ | kfox1111: do you want to have a heartattack? I can show you the pod definition | 16:26 |
kfox1111 | sure. :) | 16:26 |
kfox1111 | I need something to wake me up this morning. :) | 16:26 |
sbezverk_ | kfox1111: here you go, I warned you: http://paste.openstack.org/show/562247/ | 16:27 |
kfox1111 | not horible. needs some work. but good you got it working and identified the stuff like hostIPC needed to get it working. | 16:29 |
kfox1111 | is there a reason you coupled l3 agent and dhcp together? | 16:29 |
sbezverk_ | kfox1111: yes | 16:29 |
sbezverk_ | issues with namespaces | 16:30 |
kfox1111 | how so? | 16:30 |
sbezverk_ | kfox1111: it is still not fixed | 16:30 |
sbezverk_ | host sees namespaces but cannot access them :-( | 16:30 |
kfox1111 | not sure I follow. what is complainging? | 16:31 |
*** vhosakot has quit IRC | 16:31 | |
inc0 | what happend to kolla-ansible cleanup? pbourke | 16:32 |
pbourke | inc0: its now destroy | 16:32 |
pbourke | inc0: not sure who pushed that change | 16:32 |
kfox1111 | do the two network namespaces just cant talk to each other if in seperate containers, or does something actually complain? | 16:33 |
inc0 | not working well too | 16:33 |
pbourke | whats the problem | 16:33 |
kfox1111 | shoot. I blew out my docker devicem mapper trying to fix it... gota reboot. :/ | 16:33 |
*** Jeffrey4l_ has quit IRC | 16:33 | |
inc0 | ceph_mon volumes are stale | 16:33 |
pbourke | inc0: yeah... I've been meaning to log that bug | 16:33 |
*** mark-casey has joined #openstack-kolla | 16:33 | |
pbourke | :/ | 16:33 |
sbezverk_ | inc0: have you seen ip6tables-restore traceback in kolla before? | 16:34 |
*** gfhellma has joined #openstack-kolla | 16:34 | |
inc0 | sbezverk_, nope... | 16:34 |
sbezverk_ | it seems l3-agent uses ip6tables-restore -n | 16:34 |
sbezverk_ | but -n option does no exist | 16:34 |
sbezverk_ | in package we have with kolla | 16:35 |
Mech422 | Morning | 16:35 |
sbezverk_ | anyway I filed a bug with neutron | 16:36 |
*** athomas has quit IRC | 16:36 | |
*** harlowja_at_home has quit IRC | 16:37 | |
*** Serlex has quit IRC | 16:40 | |
kfox1111 | back. | 16:40 |
kfox1111 | maybe that's a new feature in newton? | 16:41 |
*** zhiwei has joined #openstack-kolla | 16:42 | |
sbezverk_ | kfox1111: I use master | 16:43 |
kfox1111 | yeah. so it might be a new thing with newton (trunk) | 16:44 |
kfox1111 | maybe try switchign the image to 2.0.1 instead of 3.0.0? | 16:44 |
*** vhosakot has joined #openstack-kolla | 16:45 | |
sbezverk_ | kfox1111: it does not seem to impact atm.. my main pain is namespaces | 16:45 |
kfox1111 | k. | 16:45 |
*** mgoddard_ has quit IRC | 16:46 | |
kfox1111 | so whats the namespace issue? does ping just not work, or is it something else? | 16:46 |
*** zhiwei has quit IRC | 16:46 | |
*** gfidente has quit IRC | 16:46 | |
sbezverk_ | kfox1111: It is hard to explain.. it seems connectivity depends upon a working namespace | 16:47 |
*** fragatina has quit IRC | 16:47 | |
kfox1111 | there are multiple types of namepaces. which one are you refering to? | 16:47 |
kfox1111 | ip netns ? | 16:48 |
sbezverk_ | kfox1111: when running commands withis name space stop working I loose connectivity | 16:48 |
sbezverk_ | kfox1111: yes | 16:48 |
kfox1111 | ok. lets see... | 16:48 |
kfox1111 | ip netns's are a bit different then even a kernen net namespace. | 16:48 |
kfox1111 | kernel. | 16:48 |
kfox1111 | they do some special filesystem thing, wich I think you might have to share with the host, | 16:49 |
kfox1111 | which is why its working when in the same pod maybe... | 16:49 |
*** mgoddard has joined #openstack-kolla | 16:49 | |
sbezverk_ | kfox1111: all net name spaces are listed in /run/netns | 16:50 |
*** godleon has quit IRC | 16:50 | |
kfox1111 | yeah. /var/netns. | 16:50 |
sbezverk_ | and we do share it | 16:50 |
kfox1111 | /var/lib/netns | 16:50 |
kfox1111 | which I think is a symlink.. | 16:50 |
*** aNupoisc has quit IRC | 16:50 | |
*** egonzalez90 has quit IRC | 16:50 | |
*** egonzalez90 has joined #openstack-kolla | 16:50 | |
kfox1111 | yeah. to /run/netns | 16:51 |
sbezverk_ | kfox1111: do not think so, I checked with classical kolla | 16:51 |
sbezverk_ | and everything stays in /run/netns | 16:51 |
kfox1111 | /var/run is a symlink on my box to ../run | 16:52 |
*** unicell1 has quit IRC | 16:53 | |
sbezverk_ | kfox1111: ok | 16:53 |
sbezverk_ | kfox1111: but since we share /run/netns to all neutron containers they and host should have access | 16:54 |
*** dims has quit IRC | 16:54 | |
sbezverk_ | what happens is I see that name space created example in l3 agent is accessible from within I can run ip netns exec qrouter ip a | 16:55 |
sbezverk_ | but the same command run from host generate error | 16:55 |
sbezverk_ | but host see l3-agent namespace as it shows it in "ip netns" | 16:56 |
*** dims has joined #openstack-kolla | 16:56 | |
jrich523 | hows the current master branch, did the horizon issue get resolved? | 16:57 |
rhallisey | sbezverk_, so outside the container you cant see anything you created? | 16:58 |
sbezverk_ | rhallisey: I can see but cannot taste | 16:59 |
sbezverk_ | let me illustrate it with some logs, 1 sec | 17:00 |
rhallisey | does kubernetes support shared volume mounts? | 17:01 |
rhallisey | this seems like mount propogation is failing | 17:01 |
kfox1111 | gota a meeting. bbiab. | 17:02 |
kfox1111 | yeah. did you do the mount propigation tweak from the install docs in docker? | 17:02 |
sbezverk_ | rhallisey: http://paste.openstack.org/show/562251/ | 17:02 |
sbezverk_ | do you mean shared flag in docker startup file> | 17:03 |
sbezverk_ | ? | 17:03 |
rhallisey | that mount needs to be shared | 17:03 |
sbezverk_ | it is | 17:03 |
rhallisey | so that all the containers and the host can change it and the changed get seen everywhere | 17:04 |
sbezverk_ | - name: host-run-netsh | 17:04 |
sbezverk_ | hostPath: | 17:04 |
sbezverk_ | path: /run/netsh | 17:04 |
rhallisey | I don't think that | 17:05 |
rhallisey | that's shared | 17:05 |
sbezverk_ | rhallisey: that might be a problem because when namespace gets created inside of container, host sees it like this: | 17:05 |
rhallisey | it doing normal mounting | 17:05 |
rhallisey | so it's taking the copy /run/netns and mounting it per container | 17:05 |
sbezverk_ | ----------. 1 root root 0 Aug 22 11:17 qrouter-baa3335b-0013-42dd-856a-64a5c2557a01 | 17:05 |
sbezverk_ | rhallisey: I was told that since MountFlag=Shared | 17:06 |
rhallisey | interesting... | 17:06 |
sbezverk_ | it does sharring by default | 17:06 |
rhallisey | hostPath has a different definition that I expected | 17:07 |
sbezverk_ | rhallisey: that thing screwed up my whole weekend, because I could not get my head around it | 17:07 |
rhallisey | because how does it store data if there are multiple users of the hostPath | 17:07 |
rhallisey | sbezverk_, we can test this | 17:07 |
*** inc0 has quit IRC | 17:07 | |
rhallisey | start like mariadb and create a file in there | 17:07 |
rhallisey | instead of starting the service | 17:08 |
rhallisey | then start another mariadb | 17:08 |
rhallisey | and create a different file | 17:08 |
rhallisey | ls the hostPath | 17:08 |
*** inc0 has joined #openstack-kolla | 17:08 | |
rhallisey | I"m wondering what is there | 17:08 |
sbezverk_ | I can do it with already existing containers | 17:08 |
rhallisey | ok touch a file in a shared dir | 17:08 |
rhallisey | in multiple containers | 17:09 |
rhallisey | and ls that dir | 17:09 |
rhallisey | see what shows up there | 17:09 |
*** fragatina has joined #openstack-kolla | 17:09 | |
sbezverk_ | rhallisey: yep I created files from dhcp and from l3 and host sees both files | 17:11 |
rhallisey | hmm | 17:12 |
rhallisey | does the l3 container show both files too | 17:12 |
sbezverk_ | and I can even read the content on the host | 17:12 |
sbezverk_ | yes | 17:12 |
rhallisey | ok its shared then | 17:13 |
sbezverk_ | http://paste.openstack.org/show/562253/ | 17:13 |
rhallisey | weird what is the difference in those envs | 17:14 |
rhallisey | host vs container.. | 17:14 |
sbezverk_ | rhallisey: no idea, if you want I can setup webex to show you.. | 17:15 |
rhallisey | could it be the kernel version | 17:16 |
sbezverk_ | rhallisey: I suspect the issue is the way namespace get created, probably in kube there is something specific which is not considered in the current code | 17:16 |
rhallisey | sbezverk_, I think other have this working though | 17:17 |
sbezverk_ | rhallisey: hm, I would really like to see it if in fact it is working | 17:17 |
rhallisey | well it may not be idk | 17:18 |
sbezverk_ | I had to do so much twicking to make ovsdb/vswitchd working, it is just crazy | 17:18 |
rhallisey | we're close | 17:19 |
*** unicell has joined #openstack-kolla | 17:20 | |
sbezverk_ | rhallisey: and the only way I managed to get them working with perfect stability was not using kolla abi :-( it was breaking them constantly | 17:20 |
rhallisey | sbezverk_, that' ok | 17:20 |
rhallisey | we can change it | 17:20 |
rhallisey | this exposes a flaw in it | 17:20 |
*** berendt has joined #openstack-kolla | 17:21 | |
*** aNupoisc has joined #openstack-kolla | 17:23 | |
jrich523 | hey can someone give me a hand with dealing with the missing 3.0 tag on docker.io/kolla/centos-binary-kolla-toolbox? | 17:25 |
*** inc0 has quit IRC | 17:28 | |
*** aNupoisc has quit IRC | 17:31 | |
*** fragatina has quit IRC | 17:33 | |
*** fragatina has joined #openstack-kolla | 17:33 | |
*** mntdevops has quit IRC | 17:36 | |
*** inc0 has joined #openstack-kolla | 17:39 | |
rhallisey | jrich523, 3.0 will be pushed after the N release | 17:44 |
jrich523 | until then, how do i tell it to deploy whatever is current? | 17:44 |
rhallisey | jrich523, you can build them | 17:45 |
rhallisey | clone master | 17:45 |
jrich523 | thats what i've done | 17:45 |
rhallisey | look in the tools directory | 17:45 |
rhallisey | ls /tools/build.py | 17:45 |
rhallisey | that script will build all the images you need | 17:45 |
jrich523 | thats script is pretty basic | 17:47 |
jrich523 | jsut calling another module to do the work | 17:47 |
rhallisey | yup | 17:47 |
rhallisey | that will get you going | 17:47 |
jrich523 | hmm | 17:48 |
*** berendt has quit IRC | 17:48 | |
*** haplo37__ has quit IRC | 17:49 | |
jrich523 | looks like its in diag | 17:52 |
jrich523 | so i found that my controler has the 3.0.0 docker image (someone helped me get this going, might have manually pulled it?) | 17:57 |
*** sould has joined #openstack-kolla | 17:57 | |
*** gfhellma has quit IRC | 17:58 | |
rhallisey | if you want latest master you'll have to build it. Maybe it came for a local registry? | 17:59 |
jrich523 | yeah see.. someone helped build this | 18:00 |
jrich523 | and i saw most of it over webex, but i missed a few parts | 18:00 |
jrich523 | one of which was how he fixed this | 18:00 |
jrich523 | i have a all in one setup right now, but needed to add a compute node | 18:00 |
*** gfhellma has joined #openstack-kolla | 18:00 | |
jrich523 | which, the only problem i have left is this pesky tag | 18:00 |
jrich523 | so im thinking of exporting the image and importing it on the other box... probably not idea... but.. should cut it | 18:00 |
jrich523 | im using an inventory file with kolla-ansible -i <file> deploy | 18:01 |
*** sould has quit IRC | 18:01 | |
rhallisey | jrich523, https://github.com/openstack/kolla/blob/master/doc/multinode.rst#deploy-a-registry-required-for-multinode | 18:02 |
rhallisey | you can use a local docker registry | 18:02 |
rhallisey | or build the images on the new host | 18:02 |
jrich523 | based on what docker images return, looks like i've got them all locally, but, i dont believe i ever did the registry stuff (Because it was originally all in one) | 18:03 |
inc0 | jrich523, you suggested me ATH right? | 18:03 |
jrich523 | i sure did inc0, and i think you ordered it up? | 18:03 |
*** kbaegis has joined #openstack-kolla | 18:03 | |
inc0 | yeah, just got it and testing it | 18:03 |
inc0 | so far it's cool | 18:03 |
jrich523 | oh damn that was fast | 18:03 |
inc0 | amazon <3 | 18:04 |
kbaegis | I'm glad sahara is already around :) | 18:04 |
jrich523 | lol yeah right, a abuse that so bad | 18:04 |
inc0 | wanted to say thanks:) I like midtones and highs | 18:04 |
jrich523 | i get same day delivery a lot, for free, just cuz | 18:04 |
jrich523 | yeah its a well rounded set | 18:04 |
jrich523 | which is what i was looking for since i use them for games/calls/music | 18:04 |
jrich523 | you got it with the v-moda and adapter? or just rocking the headphones so far | 18:04 |
inc0 | yeah, also I tend to listen to music that needs these highs, like classical | 18:05 |
inc0 | full set, but it seems mic is borked | 18:05 |
inc0 | it doesn't seem to be working, can't hear anything over mics cable | 18:05 |
inc0 | will debug later todat | 18:05 |
jrich523 | hmm i've had good luck with the mic's, the adapters tho can be suspect | 18:06 |
kfox1111 | sbezverk_: rhallisey: something to follow if you arn't already: https://github.com/kubernetes/kubernetes/pull/20698 | 18:06 |
inc0 | yeah, will try these too | 18:06 |
jrich523 | i assume you took them apart ok to get the adapter in correctly? | 18:06 |
inc0 | hmm...I might not...took what apart? | 18:07 |
sbezverk_ | kfox1111: thank you intresting thread | 18:07 |
jrich523 | so there is a locking... tab... inside the headphone | 18:07 |
rhallisey | kfox1111, it seems like the mount is propogating though | 18:08 |
jrich523 | if you look at the cable that came with it, it has a groove to push in and twist to keep it there | 18:08 |
jrich523 | which gets in the way with that adapter | 18:08 |
jrich523 | so you have to take it apart and remove the tab (super easy, its a screwed on part) | 18:08 |
rhallisey | sbezverk_, run that same test you did on the host in the dhcp and l3 containers | 18:08 |
inc0 | ahh I see what you mean | 18:08 |
jrich523 | super easy, just... needs to be done is all | 18:08 |
inc0 | I've missed that one | 18:08 |
jrich523 | i tried to force it past that nub, no luck, def have to take it out | 18:09 |
inc0 | will do it later, probably that's the case | 18:09 |
inc0 | or make a dent in adapter | 18:09 |
jrich523 | meh i considered that too | 18:09 |
rhallisey | sbezverk_, ok checked your old paste l3 worked | 18:09 |
jrich523 | peeling off the outer case | 18:09 |
kfox1111 | rhallisey: yeah. I'm trying to get my docker pieced back together so I can look at it. | 18:09 |
jrich523 | but they are so cheaply made i was scared to do anything to it | 18:09 |
rhallisey | kfox1111, ya same | 18:09 |
kfox1111 | it ate is back end storage. :/ | 18:09 |
rhallisey | kfox1111, getting to multonode is a pita here.. | 18:09 |
inc0 | yeah, I'd rather mess with 5$ adapter than 150$ headphones;) | 18:10 |
* rhallisey is 9 vms in | 18:10 | |
jrich523 | honestly its super easy | 18:10 |
jrich523 | pretty sure there is a video out there too | 18:10 |
inc0 | I'll use a drill probably, especially that I have 2 of them | 18:10 |
jrich523 | ahh nice | 18:10 |
kfox1111 | rhallisey: yeah. I thin this is one of the darker sides of containers right now. it needs to use some things that are not very well documented. | 18:10 |
rhallisey | kfox1111, aio need some improvment | 18:10 |
jrich523 | i will say, based on where the mic clip is (mic mute/vol) i sometimes get that caught under the desk and it rips out the plug | 18:10 |
kfox1111 | hell. docker's use of thin provisioned device mapper things are also very hard to debug. :) | 18:10 |
rhallisey | and for multinode we need a single node ceph | 18:11 |
jrich523 | so it might be good that its not locked in there :) | 18:11 |
kfox1111 | as there are very little docs on the topic. | 18:11 |
kfox1111 | yeah. We need a tutorial for using kolla-ansible to deploy just a single node ceph for testing. | 18:11 |
inc0 | yeah, that's why you want removable cable | 18:11 |
kfox1111 | until we can get kube standing it up. | 18:11 |
sbezverk_ | kfox1111: once I got ceph going for my kube cluster, I try not to touch it ;-) | 18:13 |
rhallisey | sbezverk_, how did you deploy multinode kube? | 18:13 |
rhallisey | taking recommendations | 18:13 |
sbezverk_ | it gave me some really nasty surprises, like FS corruptions | 18:13 |
kfox1111 | sbezverk_: yeah, I got it working in mine with kubernetes managed. but had to manually do some of the job stuff, | 18:14 |
sbezverk_ | rhallisey: everything was done manually.. | 18:14 |
kfox1111 | and I've got a loopback/volume mount thing I have to manually do before launching kubelet. | 18:14 |
sbezverk_ | that was the only way to make sure everything works as expected | 18:14 |
kfox1111 | I need to document all that stuff. | 18:14 |
sdake | sbezverk_ nice | 18:14 |
sbezverk_ | kfox1111: as per your comment I splitted l3 and dhcp, dod not make any difference | 18:15 |
sbezverk_ | but agree it is more flexible | 18:15 |
rhallisey | sbezverk_, I think you might have answered this, but they are all in the same pod or no? | 18:15 |
sbezverk_ | rhallisey: now I have thin pod | 18:16 |
rhallisey | hm | 18:16 |
sbezverk_ | 1 service per pod | 18:16 |
sdake | sbezverk_ kfox has a systemd workaround in there | 18:16 |
sdake | then it would appear the mounts show up on the host | 18:16 |
kfox1111 | sbezverk_: cool. what are the differences between your split stuff, and whats in this review: https://review.openstack.org/#/c/35755 | 18:17 |
sbezverk_ | sdake: mounts always appeared on the host, they were not just usable | 18:17 |
sdake | well finally got all my gear for humphrys | 18:17 |
sdake | sbezverk_ this makes them usable... | 18:17 |
kfox1111 | just the ipc flag? | 18:17 |
sdake | now its 11am - too late to climb - tomorrow it is | 18:17 |
sbezverk_ | kfox1111: I am not going to push it for review | 18:17 |
kfox1111 | yeah, just wondering what to fix in #35755. | 18:18 |
sbezverk_ | because it is major change in architecture | 18:18 |
sdake | kfox1111 in that kubernetes github pull request, you commented that you used a systemd workaround | 18:18 |
sbezverk_ | I will keep it as plan B if normal way will not work out | 18:18 |
sdake | I've got a workaround for now. Say you want to share out /export/foo and make it remountable: | 18:18 |
sdake | I made a systemd unit that does the following after mounts happen but before docker starts: | 18:18 |
sdake | mount --bind --make-shared /export /export | 18:18 |
kfox1111 | I did? | 18:18 |
kfox1111 | sdake: oh. yeah. that. :) | 18:18 |
sdake | july 23 | 18:18 |
kfox1111 | yeah. it works pretty well. | 18:19 |
rhallisey | kfox1111, I think the mounts are shared at the moment though | 18:19 |
sdake | rhallisey no - default is private not shared | 18:19 |
rhallisey | we tested it earlier | 18:19 |
sbezverk_ | sdake: we tested and they are shared | 18:19 |
sbezverk_ | I created file on share mount | 18:20 |
rhallisey | if they were private, you would create a file in each container and it would not be seen | 18:20 |
*** HyperJohnGraham has quit IRC | 18:20 | |
sbezverk_ | and host and all containers who mounted that share saw it right away | 18:20 |
rhallisey | as they would be slave mounts | 18:20 |
kfox1111 | sdake: http://pastebin.com/UUS5hybJ | 18:20 |
rhallisey | s/slave/private | 18:21 |
kfox1111 | it worked one way though. the container mounts are still slaves. | 18:21 |
kfox1111 | so I couldn't get cvmfs to run in containers and share back outwards like I wanted. | 18:21 |
rhallisey | I think docker defaults as slave not private | 18:21 |
kfox1111 | I think it might work with the kolla-kubernetes workaround though. I've been meaning to try that. | 18:22 |
sdake | rhallisey definatelly private | 18:22 |
rhallisey | I think it's private if you use volume in the dockerfile | 18:22 |
kfox1111 | then I can get cvmfs containerized. | 18:22 |
rhallisey | I think -v is a slave mount | 18:22 |
rhallisey | s/volume/VOLUME/ | 18:22 |
rhallisey | kfox1111, if you run that container with cvmfs:/cvmfs:shared it should work | 18:23 |
sbezverk_ | have meeting will be back in 2 hours | 18:23 |
rhallisey | sbezverk_, kk | 18:24 |
rhallisey | sbezverk_, ping when you're back and we can keep looking | 18:24 |
*** vhosakot has quit IRC | 18:24 | |
kfox1111 | rhallisey: not by default. as its a slave mount back to the host. but I think the settings listed in kolla-kubernetes to change the default to shared might work around the issue for me. | 18:24 |
kfox1111 | cause I want to run the cvmfs fuse filesystem in a container by daemonset, and then consume it in a different pod. | 18:25 |
*** vhosakot has joined #openstack-kolla | 18:25 | |
rhallisey | you have to run docker with mountflags=shared | 18:25 |
kfox1111 | there we go.. docker is starting to rebuild now. | 18:25 |
rhallisey | and use -v cvmfs:/cvmfs:shared | 18:25 |
kfox1111 | rhallisey: you have to do both? | 18:25 |
sdake | rhallisey - k8s does not permit passing the shared flag to a host mount | 18:25 |
kfox1111 | cause k8s can't do the latter step yet. | 18:26 |
rhallisey | it may not support it | 18:26 |
sdake | it does not support it | 18:26 |
kfox1111 | rhallisey: https://github.com/kubernetes/kubernetes/pull/20698 <- no support yet. :/ | 18:26 |
rhallisey | but we were seeing behavior of a shared mount | 18:26 |
sdake | kfox1111 has a link above which shows htei issue pull reuqeet | 18:26 |
rhallisey | right I know, but idk how sbezverk_ was able to share files in the dir | 18:26 |
rhallisey | this needs more testing | 18:26 |
sdake | rhallisey the way was via an Empty volume mount | 18:26 |
rhallisey | no he used hostPath | 18:27 |
sdake | of /run/netns | 18:27 |
sdake | but then the namespace doesn't appear on the host | 18:27 |
kfox1111 | docker images pulling... | 18:27 |
kfox1111 | ok. I should be able to help test soon. | 18:27 |
rhallisey | well think of this | 18:27 |
kfox1111 | well... except, I cant... :/ | 18:27 |
kfox1111 | I need to run opevswitch on the host, and share it out. | 18:27 |
rhallisey | if you use hostPath, it makes sense by default that everything is slave by default | 18:28 |
rhallisey | hostPath is meant to store files on the host | 18:28 |
kfox1111 | as its backing docker, and other native bridges. | 18:28 |
rhallisey | but if you have multiple users of a since hostPath then what? | 18:28 |
rhallisey | has to be shared | 18:28 |
kfox1111 | rhallisey: yeah. | 18:28 |
rhallisey | the sharing may be wrong | 18:28 |
*** sdake has quit IRC | 18:29 | |
kfox1111 | can you update a sharing flag after its mounted? | 18:29 |
kfox1111 | --remount,shared? | 18:29 |
rhallisey | not sure | 18:29 |
kfox1111 | if so, we could work around it until k8s gains native support. | 18:29 |
rhallisey | kfox1111, we need to poke around here and see what the kernel thinks of that mount | 18:31 |
rhallisey | and what we can actually do with it | 18:31 |
kfox1111 | oh... well, the neutron-l3-agent is the one making the namespaces though. so I think I can test that. | 18:31 |
rhallisey | kfox1111, well all we need to test is 2 pods that use the same hostPath | 18:32 |
kfox1111 | hmm... | 18:32 |
kfox1111 | actually I think I can do it in 1. launch the l3-agent and see if I can see it on the host? | 18:32 |
kfox1111 | if so, it shoudl work over to another pod too. | 18:33 |
rhallisey | inc0, https://github.com/kubespray/kargo you tried that right? | 18:33 |
inc0 | yeah | 18:33 |
sbezverk_ | kfox1111: seeing and using it two differenet things | 18:33 |
rhallisey | kfox1111, possibly | 18:33 |
inc0 | it's cool | 18:33 |
rhallisey | kfox1111, if it's seen on the host then that means the mount is at least a slave | 18:33 |
sbezverk_ | I can see all namespaces but running command against these namespaces from the host does not work | 18:33 |
kfox1111 | sbezverk_: ip netns exec qrouter-.... ip a #should do it. | 18:33 |
rhallisey | if you can see it on the host and in another container then it would be shared | 18:34 |
sbezverk_ | but it does not | 18:34 |
sbezverk_ | kfox1111: ip netns | 18:34 |
sbezverk_ | shows namespace | 18:34 |
sbezverk_ | ip netns exec qrouter ip a | 18:34 |
kfox1111 | the exec lets you enter it. which is what I think breaks? | 18:34 |
sbezverk_ | gives RTNETLINK | 18:34 |
sbezverk_ | error | 18:34 |
kfox1111 | yeah. | 18:34 |
kfox1111 | that makes sense. | 18:35 |
sbezverk_ | the same command within container returns correct output | 18:35 |
sbezverk_ | kfox1111: would be good if you can confirm it, maybe it is "feature" of my cluster | 18:36 |
rhallisey | inc0, k trying kubespray | 18:36 |
kfox1111 | yeah. trying to launch an l3 agent now... | 18:37 |
kfox1111 | hmm... need to get the configmap created... | 18:38 |
kfox1111 | ok... got one... | 18:39 |
*** zhiwei has joined #openstack-kolla | 18:39 | |
*** gfhellma has quit IRC | 18:40 | |
kfox1111 | k.. don't have that container handy... | 18:41 |
kfox1111 | gota build it. :/ | 18:41 |
kfox1111 | and its gota rebuild the base image.... | 18:42 |
kfox1111 | this might take a bit. | 18:42 |
rhallisey | kk | 18:42 |
*** zhiwei has quit IRC | 18:43 | |
kfox1111 | base is almost done building... | 18:49 |
kfox1111 | oh. now on to openstack-base. | 18:50 |
jemcevoy | Mech422, sean-k-mooney, I got past my ceph install problem and now can deploy a healthy ceph... It turned out to be the firewall. The doc did not say anything about turning off the firewall. Is anyone looking into firewall rules for the install? | 18:50 |
Mech422 | jemcevoy: sorry - I don't configure firewalling on my cluster machines, so its never come up | 18:51 |
jemcevoy | I was just running with what CentOS7.2 has out of the box. ie minimal kickstart... | 18:52 |
jrich523 | hey rhallisey, that link is what i need i think however stuff isnt adding up (re: docker registry) | 18:53 |
jrich523 | mostly im having a problem setting it to be insecure, the file referenced isnt there, and googling makes me thinks there are more steps to it than just creating that file | 18:53 |
kfox1111 | arg... snapshotting isn't working. | 18:54 |
rhallisey | jrich523, did you setup systemd correctly? | 18:54 |
kfox1111 | do you have to tell docker your using btrfs? | 18:54 |
rhallisey | also be sure to do a systemd daemon-reload | 18:54 |
jrich523 | i've done that, but not recently, so let me try that | 18:55 |
jrich523 | also, the only reason i didnt try that first, was because it was in the next step (figured it needed the setting before restarting stuff) | 18:55 |
kfox1111 | and, gota restart all over again building images. :/ | 18:56 |
rhallisey | jrich523, sure. Let me know if that helps | 18:56 |
jrich523 | nope, didnt create that file | 18:56 |
jemcevoy | Now I have gotten stuck at Neutron. Right after the the step "neutron | Starting neutron-server container] " I loose all network access to the "public" side of the network... I need to use VLAN 10 to access anything outside of my dev rack... This is my log http://droplet.salemherbals.com/jmcevoy/20160821-kollaInstall.html | 18:57 |
jrich523 | https://docs.docker.com/engine/admin/systemd/ | 18:57 |
jrich523 | that is what i read that made me believe there is more to it | 18:57 |
*** haplo37__ has joined #openstack-kolla | 18:57 | |
jrich523 | systemctl show docker | grep EnvironmentFile | 18:57 |
jrich523 | that returns nothing | 18:57 |
jrich523 | which is why i tought more needed to be done than just creating that file | 18:58 |
rhallisey | jrich523, check /etc/systemd/system/docker.service | 18:58 |
rhallisey | do you have environmentFile there? | 18:58 |
jrich523 | its docker.service.d and no | 18:58 |
jrich523 | jsut kolla.conf (shared mount stuff) | 18:59 |
rhallisey | how about /usr/lib/systemd/system/docker.service | 19:00 |
*** daneyon has joined #openstack-kolla | 19:00 | |
jrich523 | that exists | 19:01 |
rhallisey | jrich523, does environmenFile exist in there? | 19:02 |
jrich523 | nope | 19:02 |
rhallisey | jrich523, at it there | 19:04 |
rhallisey | and add $DOCKER_OPTS | 19:04 |
*** daneyon has quit IRC | 19:05 | |
jrich523 | does the env file belong in the [Service] section? | 19:05 |
rhallisey | yes | 19:06 |
*** HyperJohnGraham has joined #openstack-kolla | 19:06 | |
jrich523 | and am i adding the $DOCKER_OPTS to the ExecStart? | 19:06 |
rhallisey | yup | 19:08 |
rhallisey | EnvironmentFile=-/etc/default/docker | 19:08 |
rhallisey | ExecStart=/usr/bin/docker daemon -H fd:// $DOCKER_OPTS | 19:08 |
openstackgerrit | Eduardo Gonzalez proposed openstack/kolla: Add nova-serialproxy container https://review.openstack.org/358822 | 19:09 |
openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Customization for base https://review.openstack.org/358042 | 19:14 |
jrich523 | rhallisey: so i then follow the Ubuntu directions (this is CentOS 7), which tells me to enter, in the env file, DOCKER_OPTS="--insecure-registry 192.168.1.100:4000" | 19:15 |
*** salv-orl_ has joined #openstack-kolla | 19:16 | |
rhallisey | jrich523, I think /etc/sysconfig/docker use to be a default | 19:17 |
rhallisey | or it's a global systemd default idk | 19:17 |
rhallisey | but, the ubuntu piece will work regardless | 19:18 |
jrich523 | ok | 19:18 |
jrich523 | actually it failed, but i think i got it | 19:18 |
jrich523 | just going to test stuff first | 19:18 |
rhallisey | on by a default I mean default in the systemd start file | 19:18 |
rhallisey | jrich523, ok gotcha | 19:18 |
*** salv-orlando has quit IRC | 19:19 | |
jrich523 | so, in the end, what i did is added the env file path to the service as you said, left the exec alone (the one you provided didnt fly) and then added the insecure gateway stuff to the file (just as the centos directions say) | 19:19 |
rhallisey | jrich523, did it work? | 19:19 |
jrich523 | basically all i had to do outside of the directions was to specify the env file, which, i might not have to do if it was an assume default locaiton... but docker is running, now just need to check registry | 19:19 |
jrich523 | well i dont know if the registry is working yet, but docker started back up this time | 19:19 |
rhallisey | cool | 19:20 |
rhallisey | check the status | 19:20 |
rhallisey | should see insecure-registyry | 19:20 |
jrich523 | i see it running in docker ps | 19:20 |
jrich523 | k | 19:20 |
*** senk has joined #openstack-kolla | 19:20 | |
*** berendt has joined #openstack-kolla | 19:20 | |
rhallisey | ya iirc /etc/sysconfig/docker was a default as an environemnt file and is no longer | 19:21 |
rhallisey | jrich523, if you can, update the docs to reflect that | 19:21 |
rhallisey | the ubuntu systemd method will work for centos | 19:21 |
jrich523 | hmm if it does, im missing something | 19:21 |
jrich523 | it didnt want to start | 19:21 |
rhallisey | + | 19:21 |
jrich523 | based on the exec command you gave (which is what the docs talked about for ubuntu) | 19:22 |
rhallisey | can you paste the output of systemctl status docker | 19:22 |
jrich523 | http://paste.openstack.org/show/562270/ | 19:23 |
jrich523 | i dont see any reference to the param | 19:23 |
rhallisey | run a systemctl daemon-reload | 19:23 |
rhallisey | and restart it | 19:23 |
jrich523 | i had before starting it, but i'll give it another go | 19:23 |
rhallisey | hm | 19:23 |
rhallisey | can you paste /usr/lib/systemd/system/docker.service | 19:24 |
jrich523 | http://paste.openstack.org/show/562271/ | 19:24 |
rhallisey | and paste /etc/sysconfig/docker | 19:25 |
*** berendt has quit IRC | 19:25 | |
jrich523 | http://paste.openstack.org/show/562272/ | 19:25 |
rhallisey | #ExecStart=/usr/bin/docker daemon -H fd:// $DOCKER_OPTS | 19:26 |
rhallisey | it's commented out | 19:26 |
rhallisey | anbd | 19:26 |
jrich523 | ... i should probalby update that ip :-/ | 19:26 |
jrich523 | right, because that failed | 19:26 |
rhallisey | DOCKER_OPTS="--insecure-registry 192.168.1.100:4000" is the var | 19:26 |
rhallisey | oh when you ran with that | 19:26 |
jrich523 | which is commented out as well, since it pairs up with the commented out start | 19:26 |
jrich523 | the service fails to start when i use the commented out method | 19:26 |
*** haplo37__ has quit IRC | 19:26 | |
rhallisey | what was the failure? | 19:27 |
jrich523 | let me swap it back, not sure it gave much other than a failure code of 1 i think, but give me a sec | 19:27 |
rhallisey | ok | 19:27 |
*** senk has quit IRC | 19:28 | |
*** senk has joined #openstack-kolla | 19:29 | |
*** salv-orl_ has quit IRC | 19:29 | |
jrich523 | http://paste.openstack.org/show/562274/ | 19:29 |
*** david-lyle has joined #openstack-kolla | 19:30 | |
rhallisey | ugh | 19:30 |
jrich523 | seems like its not allowed to run docker this way, it has to be dockerd? | 19:31 |
jrich523 | which... doesnt support the params in this way | 19:31 |
rhallisey | systemctl unmask docker.socket | 19:32 |
rhallisey | then start it again | 19:32 |
jrich523 | err, as root its saying i dont have permissions... | 19:32 |
*** HyperJohnGraham has quit IRC | 19:33 | |
rhallisey | can you paste /etc/systemd/system/docker.service.d/docker_options.conf | 19:34 |
senk | jrich523 wat is `systemctl status docker.socket` saying? | 19:34 |
*** gfhellma has joined #openstack-kolla | 19:34 | |
jrich523 | senk: not found | 19:34 |
jrich523 | rhallisey: that isnt there, only kolla.conf | 19:35 |
senk | jrich523 looks like a problem with systemd socket activation.. any problems with using `-H unix:///var/run/docker.sock` | 19:35 |
*** HyperJohnGraham has joined #openstack-kolla | 19:36 | |
rhallisey | jrich523, can you paste what's in kolla.conf | 19:36 |
jrich523 | senk: where/how do i use that? | 19:36 |
jrich523 | [Service] MountFlags=shared | 19:37 |
senk | jrich523 instead of `-H fd://` | 19:37 |
jrich523 | k | 19:37 |
senk | for socket activation https://github.com/docker/docker/blob/master/contrib/init/systemd/docker.socket should be present in /etc/systemd/system | 19:39 |
jrich523 | that got it running | 19:40 |
rhallisey | nice | 19:40 |
senk | for more information on socket activation: http://0pointer.de/blog/projects/socket-activation.html | 19:40 |
rhallisey | thanks senk | 19:41 |
jrich523 | http://paste.openstack.org/show/562277/ | 19:41 |
jrich523 | thats the status, dont see any mention of the insecure setup tho | 19:41 |
*** david-lyle has quit IRC | 19:42 | |
*** sdake has joined #openstack-kolla | 19:42 | |
sdake | inc0 ping | 19:43 |
rhallisey | did you reload the daemon? I wonder if the /usr/lib/systemd/system/docker.serivce changed | 19:43 |
jrich523 | i did, i've been doing: systemctl daemon-reload;systemctl restart docker | 19:43 |
inc0 | sdake, here | 19:44 |
*** david-lyle has joined #openstack-kolla | 19:46 | |
rhallisey | jrich523, is docker.service still have $DOCKER_OPTS in place? | 19:46 |
rhallisey | s/is/does/ | 19:47 |
jrich523 | yeah | 19:47 |
rhallisey | weird | 19:47 |
jrich523 | i commented out the 'centos' syntax and uncommented the docker_opts | 19:47 |
rhallisey | I'm not sure | 19:48 |
jrich523 | i think i at least have a handle on whats going on now | 19:48 |
senk | jrich523 do you have output of `docker info` | 19:48 |
jrich523 | oh, that would show insecure registries, which, lists it as 127.0.0.0/8 | 19:49 |
jrich523 | http://paste.openstack.org/show/562278/ | 19:49 |
senk | what is currently in /etc/systemd/system/docker.service.d | 19:51 |
jrich523 | just kolla.conf | 19:52 |
senk | and content of /etc/systemd/system/docker.service | 19:52 |
senk | and `systemctl show docker` maybe :D | 19:53 |
jrich523 | i dont have docker.service, just docker.service.d | 19:54 |
kfox1111 | hmm.... | 19:55 |
kfox1111 | are we assigning uid's explicitly in containers? | 19:55 |
senk | what about /lib/systemd/system/docker.service? | 19:55 |
kfox1111 | how do we know neutron's uid isn't going to change? | 19:55 |
jrich523 | http://paste.openstack.org/show/562280/ | 19:56 |
jrich523 | it has the 3 start methods (yours is active) and the env config path set to /etc/sysconfig/docker | 19:56 |
jrich523 | oh wait | 19:56 |
jrich523 | so | 19:56 |
senk | jrich523 make line 14 look like this | 19:56 |
senk | ExecStart=/usr/bin/docker daemon -H unix://var/run/docker.sock | 19:56 |
senk | ah wait | 19:57 |
senk | like ExecStart=/usr/bin/docker daemon -H unix://var/run/docker.sock $DOCKER_OPTS | 19:57 |
jrich523 | k, that makes sense | 19:57 |
*** matrohon has joined #openstack-kolla | 19:57 | |
kfox1111 | do we just rely on whatever rpm/deb creates for the user? | 19:57 |
*** inc0 has quit IRC | 19:57 | |
*** inc0 has joined #openstack-kolla | 19:58 | |
jrich523 | nice | 19:58 |
inc0 | guys I need an advice | 19:58 |
jrich523 | when i do docker info i see the repo there | 19:58 |
inc0 | I'm thinking of customization of sources.list | 19:58 |
senk | jrich523 great | 19:59 |
inc0 | and I have an idea I'd like to run with you before I'll do this | 19:59 |
jrich523 | thanks senk! | 19:59 |
inc0 | macro edit_file | 19:59 |
senk | jrich523 now i can sleep in peace :D | 19:59 |
jrich523 | just in time for me to get lunch, now that its 1pm :) | 19:59 |
jrich523 | lol | 19:59 |
inc0 | that will act like customizable, only with lines:) | 19:59 |
senk | 10pm here :D | 19:59 |
jrich523 | lol just in time then :) | 19:59 |
senk | jrich523 oh yes.. bye :) | 20:00 |
jrich523 | thanks again | 20:00 |
jrich523 | cya | 20:00 |
*** senk has quit IRC | 20:00 | |
inc0 | so lines_append add line -> it will append at the end of the file, lines_remove -> sed line to empty line, lines_override -> sed line to new line | 20:00 |
kfox1111 | fyi, I tried updating an entry in an ini file that had duplicate lines, but the line I wanted to edit was not duplicated. | 20:00 |
kfox1111 | crudini did not munge the file. | 20:00 |
kfox1111 | so we're fine using it for kolla-kubernetes. | 20:01 |
inc0 | so this will not be just for ini files...my main concern now are souces.list | 20:01 |
*** HyperJohnGraham has quit IRC | 20:01 | |
inc0 | and it will have to be bashy bashy as it'll be part of build | 20:01 |
*** ayoung has joined #openstack-kolla | 20:03 | |
kfox1111 | fun. so we're just using the uids of whatever package gets there first. | 20:03 |
kfox1111 | going to need a second init container I think... | 20:06 |
inc0 | issue is COPY in docker only works with local dir | 20:07 |
inc0 | so if somebody wants to add new sources.list they have to put it into kolla dir | 20:07 |
inc0 | which is crappy | 20:07 |
rhallisey | inc0, not getting kubespary working | 20:07 |
rhallisey | :/ | 20:07 |
rhallisey | gets stuck on a cert task | 20:07 |
kfox1111 | know offhand how to get k8s to override user? | 20:08 |
inc0 | rhallisey, try to run just this play https://github.com/kubespray/kargo/blob/master/cluster.yml | 20:08 |
sbezverk_ | kfox1111" would be goo to have crudini in base image | 20:09 |
rhallisey | inc0, am | 20:09 |
rhallisey | inc0, just changed to inventory | 20:09 |
kfox1111 | sbezverk_: yeah, that may be an option too. | 20:09 |
sbezverk_ | I need to do some prep which requires boot up with actual process image and crudini only installed in tool box | 20:09 |
kfox1111 | you can have more then one init container though, so I've been able to do it that way. | 20:09 |
sbezverk_ | otherwise I had to you sed which works but looks scary | 20:10 |
kfox1111 | yeah. | 20:10 |
sbezverk_ | have you tried namespace? | 20:10 |
kfox1111 | sbezverk_: I'm updating https://review.openstack.org/#/c/357557 to try and run it on my box, | 20:10 |
sbezverk_ | kfox1111 | 20:10 |
kfox1111 | using some of the stuff in your pastebin. | 20:10 |
sbezverk_ | cool, cannot wait to see the result | 20:10 |
sbezverk_ | I really hope it is just my cluster problem | 20:11 |
kfox1111 | the chown was throwing me though. as neutron uid is only in the neutron-base container, not the toolbox. | 20:11 |
kfox1111 | so adding a new init container now. | 20:11 |
kfox1111 | but the neutron container is running as netron user, so it cant chown. | 20:11 |
kfox1111 | know how to override? | 20:11 |
kfox1111 | sbezverk_: how did you get the chown to work? | 20:18 |
sbezverk_ | kfox1111: I had to force neutron ovs to use tcp socket | 20:20 |
kfox1111 | was talking about the init container: chown -R neutron:kolla /var/log/kolla; | 20:20 |
sbezverk_ | :-) ahhhh it is bacuse you use tool boc | 20:21 |
kfox1111 | when I run the l3-agent, its running as user 'neutron', and doesn't have permissions to do that. | 20:21 |
*** sdake has quit IRC | 20:21 | |
sbezverk_ | box neutron user is not defined in tool box | 20:21 |
kfox1111 | no, I'm running the chown in the neutron container. | 20:21 |
kfox1111 | but its switching to user neutron first. | 20:21 |
kfox1111 | had SecurityContextDeny in my admission-controler on api-server. | 20:23 |
sbezverk_ | kfox1111: because of all these incompatibilities I completely stopped using extended_start | 20:23 |
kfox1111 | yeah. I like the init containers better for that sort of thing. Ok. I have a solution. moving on.... | 20:24 |
rhallisey | sbezverk_, post your patch and document the kube issue. | 20:25 |
openstackgerrit | Eduardo Gonzalez proposed openstack/kolla: Ansible config for nova-serialproxy console https://review.openstack.org/358839 | 20:25 |
kfox1111 | failure applying iptables rules now... | 20:25 |
rhallisey | sbezverk_, what do you have for the extend_start changes | 20:25 |
rhallisey | sbezverk_, is it hard coded? | 20:25 |
rhallisey | inc0, ^ repo split dude | 20:25 |
sbezverk_ | I do not use it at all, I have command in the pod that start the process | 20:25 |
rhallisey | ok | 20:26 |
sbezverk_ | all preparation which used to be done in extended_start are done init container | 20:26 |
rhallisey | was there any changed on the kolla side? | 20:26 |
sbezverk_ | no, kolla does not need any changes | 20:26 |
rhallisey | perfect | 20:26 |
rhallisey | let's go with that for now | 20:26 |
sbezverk_ | it is even better to roll back to the original state | 20:27 |
rhallisey | it will add to our mound of tech debt, but it will work for now | 20:27 |
sbezverk_ | I mean for kolla | 20:27 |
sbezverk_ | to the point when we tried to optimize it for kube | 20:27 |
sbezverk_ | rhallisey: one thing which is not done is rendering of these new templates | 20:28 |
sbezverk_ | I will post templates, but will need somebody's help to get rendering around them. | 20:28 |
kfox1111 | almost there... | 20:29 |
kfox1111 | iptables is failing to apply now... | 20:29 |
*** egonzalez90 has quit IRC | 20:29 | |
sbezverk_ | kfox1111: hit that too | 20:29 |
*** salv-orlando has joined #openstack-kolla | 20:29 | |
sbezverk_ | you need to load ip6table | 20:29 |
sbezverk_ | you need to load ip6tables | 20:29 |
sbezverk_ | module | 20:30 |
kfox1111 | need it installed? | 20:30 |
kfox1111 | oh. | 20:30 |
kfox1111 | gotcha. | 20:30 |
sbezverk_ | mdules mount should fix it | 20:30 |
kfox1111 | it is loaded it seems. | 20:31 |
kfox1111 | still failing... | 20:31 |
sbezverk_ | kfox1111: how about hostIPC? | 20:32 |
sbezverk_ | It uses semaphors for iptables | 20:32 |
kfox1111 | yup got that. | 20:32 |
kfox1111 | neutron.agent.linux.iptables_manager RequiredOptError: value required for option: lock_path | 20:32 |
kfox1111 | maybe that's it. | 20:32 |
kfox1111 | I'm using my own neutron config files from the host. | 20:33 |
rhallisey | kfox1111, are you cool rebasing https://review.openstack.org/#/c/357557 after sbezverk_ patch lands? | 20:34 |
*** salv-orlando has quit IRC | 20:34 | |
rhallisey | would like to get a working neutron then we can optimize | 20:34 |
kfox1111 | which patch? | 20:34 |
rhallisey | https://review.openstack.org/#/c/357557 | 20:34 |
kfox1111 | no, which sbezverk_ patch? | 20:34 |
*** sdake has joined #openstack-kolla | 20:35 | |
kfox1111 | rhallisey: do you mean this: http://paste.openstack.org/show/562247/ ? | 20:36 |
*** jtriley has quit IRC | 20:36 | |
rhallisey | ya the one he is posting | 20:37 |
kfox1111 | when we talked earlier, he was just trying to get something to work. | 20:37 |
kfox1111 | I think we should merge both into 357557 in the end? | 20:37 |
*** david-lyle has quit IRC | 20:38 | |
kfox1111 | I've been pulling stuff in from the pastebin into review #357557. | 20:38 |
rhallisey | ya | 20:38 |
*** david-lyle has joined #openstack-kolla | 20:39 | |
sbezverk_ | kfox1111: If we can to first merge my containers and then add your labeling and placement but without changing how containers start | 20:39 |
kfox1111 | hmm... I think my config started more then one agent in the container, but it started this time. | 20:39 |
sbezverk_ | that is what I saw too when I was using exteneded_start | 20:40 |
sbezverk_ | second container will die eventually | 20:40 |
kfox1111 | I think there's a fair amount of difference in start aragement between the two, and I think mine might be closer to what we want in the end? | 20:40 |
sbezverk_ | kfox1111 if you make it working, sure | 20:40 |
kfox1111 | k. I can exec into the ip netns space.. | 20:40 |
kfox1111 | lets see if it was a leftover from l3agent running on the host though... | 20:41 |
kfox1111 | ok. deleted from host.... | 20:42 |
kfox1111 | restarting the container. | 20:42 |
kfox1111 | ok. too the point where I'm reproducting sbezverk_'s RTNETLINK answers: Invalid argument | 20:44 |
kfox1111 | on the host. | 20:44 |
kfox1111 | lets see... | 20:44 |
*** david-lyle has quit IRC | 20:44 | |
*** david-lyle_ has joined #openstack-kolla | 20:44 | |
sbezverk_ | :-( | 20:44 |
*** david-lyle_ has quit IRC | 20:44 | |
kfox1111 | sec. never did the docker share thing yet. | 20:44 |
kfox1111 | docs are wrong... fixing... | 20:45 |
*** daneyon has joined #openstack-kolla | 20:48 | |
rhallisey | man kubespray has my env all messed up | 20:50 |
rhallisey | :/ | 20:50 |
kfox1111 | all the propagation modes show up still as rslave still when docker inspecting the container. | 20:51 |
sbezverk_ | kfox1111: I mean it is not great, but it is not a show stopper fr us | 20:53 |
*** daneyon has quit IRC | 20:53 | |
sbezverk_ | as long as inside of a container the process can access namespace correctly it will work | 20:53 |
kfox1111 | nothing talks to the ip netns but that one container? | 20:53 |
sbezverk_ | what I also noticed that name space gets broken after a period of time | 20:53 |
sbezverk_ | the workaround I found which works was to remove namespace from the host | 20:54 |
kfox1111 | somethings cleaning it up? | 20:54 |
sbezverk_ | then it gets re-created and will work again for a while | 20:54 |
kfox1111 | In that case, maybe we just don't mount /run/netns? | 20:54 |
kfox1111 | it would be isolated then. | 20:55 |
sbezverk_ | kfox1111: do not know, all of a sudden I could not run ip netne exec even inside of container | 20:55 |
kfox1111 | if we're sure nothing else needs access to it, then yeah, lets just not mount /run/netns. it shoudl protect it from that I think. | 20:56 |
rhallisey | sbezverk_, as long as all the containers that share that volume can use the namespace then we're good | 20:58 |
kfox1111 | hmm.... user neutron can't enter it though. | 20:58 |
rhallisey | we can report the host issue as a kube bug | 20:58 |
sbezverk_ | rhallisey: we do not know if it is not by design | 20:58 |
rhallisey | indeed | 20:59 |
sbezverk_ | it would be great to get kube guy you met at summit to join irc to have a chat | 20:59 |
kfox1111 | ok. I"m able to ping the router from dhcp. | 20:59 |
sbezverk_ | and maybe even demonstration | 20:59 |
kfox1111 | so I think this update will work. | 20:59 |
openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP - l3/metadata DaemonSets + DVR, & labels https://review.openstack.org/357557 | 21:00 |
kfox1111 | sbezverk_: please give that a try when you get a sec. I think I merged in enough of your changes that it works. | 21:01 |
kfox1111 | got a meeting. bbiab | 21:01 |
*** dgonzalez has quit IRC | 21:01 | |
*** dgonzalez has joined #openstack-kolla | 21:03 | |
*** salv-orlando has joined #openstack-kolla | 21:04 | |
rhallisey | sbezverk_, did you do this to setup multinode? http://kubernetes.io/docs/getting-started-guides/docker-multinode/ | 21:05 |
rhallisey | looks neat | 21:05 |
rhallisey | super easy | 21:05 |
*** salv-orlando has quit IRC | 21:19 | |
*** salv-orlando has joined #openstack-kolla | 21:19 | |
*** david-lyle_ has joined #openstack-kolla | 21:19 | |
*** salv-orlando has quit IRC | 21:20 | |
*** salv-orlando has joined #openstack-kolla | 21:20 | |
*** JoseMello has quit IRC | 21:20 | |
*** imcsk8 has quit IRC | 21:25 | |
*** imcsk8 has joined #openstack-kolla | 21:25 | |
*** godleon has joined #openstack-kolla | 21:26 | |
*** adnavare has joined #openstack-kolla | 21:28 | |
adnavare | Hi Guys, I am installing docker for Kolla but I am getting following errors http://pastebin.com/hsEDv99p. I am using this http://pastebin.com/U68RjDgB playbook to install the packages to setup the host. Any inputs on this? Do I need some more extra packages before this? | 21:33 |
jrich523 | based on that it looks like docker isnt fully installed? im pretty sure docker-engine comes with docker, if you do docker --version whats it spew out? | 21:34 |
rhallisey | you can install it this way: curl -sSL https://get.docker.io | bash | 21:34 |
*** david-lyle_ has quit IRC | 21:34 | |
rhallisey | hmm haven't used that play before | 21:34 |
adnavare | jrich523: docker is not installed yet | 21:35 |
adnavare | so --version is saying docker not installed | 21:36 |
jrich523 | oh, well you've got to install it :) | 21:36 |
adnavare | I am trying to do that. The playbook does it | 21:36 |
adnavare | let me check again if it is present, but I believe so | 21:36 |
jrich523 | well, apparently it doesnt :) | 21:36 |
jrich523 | from what i see, it looks like it should | 21:37 |
adnavare | jrich523: so I am adding a repo using "deb https://apt.dockerproject.org/repo ubuntu-trusty main" and then installing docker-engine | 21:37 |
jrich523 | but i cant say i've done it from that repo or that way, i did the curl like rhallisey mentioned | 21:37 |
adnavare | jrich523: do i need first docker separately? and then docker engine | 21:38 |
jrich523 | well based on what you're doing, it should do it for you | 21:38 |
jrich523 | however thats a new playbook | 21:38 |
*** Mr_Broken has quit IRC | 21:38 | |
adnavare | jrich523: Okay, let me try with your curl way | 21:38 |
jrich523 | where did you get the playbook? | 21:38 |
adnavare | jrich523: I have done it for my setup | 21:39 |
jrich523 | nice | 21:39 |
jrich523 | so technically what you are doing should work | 21:39 |
jrich523 | but, it cant install docker, question is why | 21:39 |
adnavare | jrich523: hmm, yeah that's what I am held at | 21:39 |
jrich523 | since its all something you've created, it might be better to look for ansible help | 21:39 |
adnavare | and the packages it is recommending are never suggested | 21:39 |
adnavare | jrich523: so you did curl and then normal apt-get install? | 21:40 |
adnavare | is that the way you did? | 21:40 |
jrich523 | well, yum, im on centos, but yeah | 21:40 |
jrich523 | i used the quick start | 21:40 |
jrich523 | which is kinda off | 21:40 |
adnavare | Oh Okay | 21:40 |
jrich523 | but yeah, there are few things you need before starting the build, docker is one of those few :) | 21:41 |
jrich523 | and your 'set up packaging on target hosts' seems to have the rest of the dependencies | 21:41 |
adnavare | jrich523: the error seems weird not clearly stating what is going wrong | 21:41 |
adnavare | jrich523: hmm yeah :) | 21:42 |
jrich523 | well it looks like its trying to install the engine, and perhaps pass some params, which, could also be the problem? i dont know man, not awesome with ansible | 21:43 |
jrich523 | err what OS are you using? | 21:43 |
*** rhallisey has quit IRC | 21:44 | |
adnavare | jrich523: UBUNTU | 21:44 |
adnavare | 14.04 | 21:44 |
jrich523 | hmm | 21:44 |
mark-casey | adnavare: can you print the output of the previous task, the apt-get update? I wonder if it's skipping | 21:44 |
jrich523 | i see some references to OS issues, oddly worded tho | 21:44 |
jrich523 | ibdevmapper1.02.1 (>= 2:1.02.99) but 2:1.02.77-6ubuntu2 is to be installed | 21:45 |
adnavare | mark-casey: do you mean the error that I am getting? | 21:45 |
adnavare | jrich523: it is saying some broken packages | 21:46 |
jrich523 | a whole bunch, mostly based on new versions when looking for older? | 21:46 |
mark-casey | adnavare: yes. you have an apt task that just sets update_cache=yes. I'm wondering if Ansible is just returning OK and not running the apt-get update because there is not package listed to act on | 21:46 |
jrich523 | all steming from docker-engine it seems | 21:46 |
adnavare | mark-casey: oh okay let me copy that in pastebin | 21:47 |
mark-casey | granted I don't know why that would do this... :D | 21:48 |
openstackgerrit | Prithiv proposed openstack/kolla: Dockerfiles for collectd-ceilometer-plugin Container https://review.openstack.org/358632 | 21:48 |
adnavare | mark-casey: this is the output of play http://pastebin.com/gdSG1HsE | 21:48 |
adnavare | mark-casey: it seems to install packages | 21:49 |
adnavare | mark-casey: "item=[u'ntp', u'ntpdate', u'openssh-server', u'sudo', u'python-pip', u'python-dev', u'libffi-dev', u'libssl-dev', u'gcc', u'git', u'linux-image-generic-lts-wily']" | 21:49 |
adnavare | jrich523: what do you mean looking for older versions? | 21:50 |
jrich523 | adnavare: maybe newer, but, this: ibdevmapper1.02.1 (>= 2:1.02.99) but 2:1.02.77-6ubuntu2 is to be installed | 21:51 |
jrich523 | i assume thats saying greater than .99, but .77 to be installed (not new enough?) | 21:51 |
jrich523 | few version references in that error you posted | 21:51 |
adnavare | jrich523: Yeah seems something like that. Because even my curl is not working | 21:51 |
mark-casey | adnavare: after this fails you might "apt-get update" and "apt-get install docker-engine" manually. If that works then in your playbook move the update_cache:yes down to the docker install ansible task. If not then my guess was not it | 21:51 |
jrich523 | oh now thats weird | 21:51 |
adnavare | mark-casey: so shall i now run apt-get update? | 21:52 |
mark-casey | yeah. assuming docker still isn't installed right now | 21:52 |
adnavare | yeah docker not installed | 21:52 |
mark-casey | adnavare: I was wrong. this is on the docs for Ansible's apt module: "Run the equivalent of apt-get update before the operation. Can be run as part of the package installation or as a separate step." So what you had should have been ok | 21:55 |
mark-casey | 'or as a separate step' is ok | 21:55 |
adnavare | mark-casey: hmm but my apt-get update gave couple of warnings http://pastebin.com/LHr9iZ8k | 21:57 |
adnavare | rich523:seems i need certain packages basic version and then update those packages | 21:57 |
mark-casey | oh. yea. so the ansible just hasn't been able to give you that feedback as written. that could be it | 21:57 |
*** vhosakot has quit IRC | 21:58 | |
adnavare | mark-casey: i did not got you | 21:58 |
jrich523 | speaking of ansible/docker, if i setup a docker registry i have to configure that on each endpoint? im trying to use the multinode playbook and iv'e setup the registry which is ok on the controller but the remote end cant find it, and docker info doesnt list it as there | 21:58 |
jrich523 | i guess i was under the impression that would get passed over, but doesnt seem to be? | 21:58 |
mark-casey | adnavare: well the error is that apt needs one version of some packages and is installing another version. duplicate or wrong repos could, possibly, do that | 21:59 |
mark-casey | you could always do the curl method but I've seen docker commiters say they're looking to move away from that in the future | 21:59 |
*** matrohon has quit IRC | 22:01 | |
adnavare | mark-casey: yes, okay let me try to get "init-system-helpers 1.14, lsb-base 4.1Debian11ubuntu6, libdevmapper" | 22:03 |
adnavare | as these are the packages it is looking | 22:03 |
adnavare | mark-casey: i am just wondering these packages are never required, or atleast no where said to be installed | 22:04 |
*** caowei has joined #openstack-kolla | 22:14 | |
sbezverk_ | rhallisey: no, I have not used it.. | 22:19 |
*** srwilkers_ has joined #openstack-kolla | 22:27 | |
*** david-lyle_ has joined #openstack-kolla | 22:37 | |
*** daneyon has joined #openstack-kolla | 22:37 | |
*** daneyon has quit IRC | 22:41 | |
*** david-lyle has joined #openstack-kolla | 22:49 | |
*** david-lyle_ has quit IRC | 22:52 | |
openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: WIP introduces new approach in starting OVS in Kube https://review.openstack.org/358895 | 22:53 |
*** mark-casey has quit IRC | 22:53 | |
*** gfhellma has quit IRC | 22:55 | |
Pavo | evening everyone | 22:55 |
inc0 | hey Pavo | 22:55 |
Pavo | hows it be going? | 22:56 |
inc0 | slowly but steadily | 22:56 |
Pavo | nice | 22:56 |
Pavo | does kolla support pci-passthrough or sr-iov? | 22:57 |
Pavo | or should I say can it support? | 22:57 |
inc0 | I know a guy who ran sr-iov | 22:57 |
inc0 | but afair there was some issues with it | 22:57 |
inc0 | it is possible tho | 22:57 |
openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes: WIP introduces new approach in starting OVS in Kube https://review.openstack.org/358895 | 22:57 |
Pavo | ah, yeah been testing pci-passthough with just using custom nova.conf file but having issues also | 22:58 |
*** david-lyle has quit IRC | 23:02 | |
jemcevoy | inc0, I have a nVidia K2 Grid card in the kolla cluster I am trying to build... (stuck in neutron) that I will need to get pci-passthrough working on so I can help with the passthrough stuff. BUT I need to get neutron working first... | 23:15 |
inc0 | what seems to be an issue? | 23:15 |
jemcevoy | Pavo do you have all the kernel args and pci_stubs configured? | 23:15 |
jemcevoy | http://droplet.salemherbals.com/jmcevoy/20160821-kollaInstall.html | 23:16 |
*** sdake has quit IRC | 23:16 | |
jemcevoy | When I get to the red part I no longer have a connection to the public network.. Ping stops working | 23:16 |
*** michauds has quit IRC | 23:17 | |
inc0 | jemcevoy, ubuntu 14.04? | 23:17 |
jemcevoy | I need to use VLAN 10 to got off the my development rack to the rest of the network | 23:17 |
jemcevoy | CentOS 7.2 | 23:17 |
jemcevoy | and openvswitch | 23:18 |
jemcevoy | Should I change to linux bridge? | 23:18 |
inc0 | I don't think this is an issue | 23:18 |
inc0 | but tak a look at syslog | 23:18 |
inc0 | docker seems to be acting up | 23:19 |
jemcevoy | Do you mean /var/log/messages on the host system? cc1-d? | 23:19 |
inc0 | well, /var/log/syslog on ubuntu | 23:20 |
inc0 | equivalent | 23:20 |
jemcevoy | Just kickstarted so it will take 15-20 mints to fail again | 23:20 |
jemcevoy | syslog = messages,,, | 23:21 |
*** dave-mcc_ has quit IRC | 23:23 | |
*** zhiwei has joined #openstack-kolla | 23:27 | |
*** zhiwei has quit IRC | 23:31 | |
*** sdake has joined #openstack-kolla | 23:33 | |
*** adnavare has quit IRC | 23:42 | |
kbaegis | So with Centos7, installing it inside an openstack instance | 23:42 |
kbaegis | I can't seem to see any disks | 23:42 |
kbaegis | Does that have to be set up as a volume/qcow instead? | 23:43 |
inc0 | hmm, maybe selinux is not very permissive? | 23:43 |
*** bmace has quit IRC | 23:49 | |
*** bmace has joined #openstack-kolla | 23:49 | |
*** sdake has quit IRC | 23:55 | |
*** inc0 has quit IRC | 23:57 | |
*** britthouser has quit IRC | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!