| *** daneyon_ has joined #openstack-kolla | 00:14 | |
| *** sdake has quit IRC | 00:15 | |
| *** haplo37 has quit IRC | 00:15 | |
| *** g3ek has quit IRC | 00:16 | |
| *** haplo37 has joined #openstack-kolla | 00:16 | |
| *** diogogmt has quit IRC | 00:17 | |
| *** g3ek has joined #openstack-kolla | 00:17 | |
| *** daneyon_ has quit IRC | 00:18 | |
| *** dwalsh has joined #openstack-kolla | 00:42 | |
| *** aNupoisc has quit IRC | 00:42 | |
| *** zhiwei has joined #openstack-kolla | 00:53 | |
| *** Mr_Broken has quit IRC | 00:57 | |
| *** haplo37 has quit IRC | 00:58 | |
| *** aernhart has quit IRC | 01:06 | |
| *** haplo37 has joined #openstack-kolla | 01:07 | |
| *** Mr_Broken has joined #openstack-kolla | 01:07 | |
| *** Mr_Broken has quit IRC | 01:12 | |
| *** zhurong has joined #openstack-kolla | 01:12 | |
| *** Mr_Broken has joined #openstack-kolla | 01:14 | |
| *** zhurong has quit IRC | 01:16 | |
| *** jtriley has quit IRC | 01:18 | |
| *** dwalsh has quit IRC | 01:18 | |
| *** zhurong has joined #openstack-kolla | 01:19 | |
| *** yuanying has quit IRC | 01:22 | |
| *** williamcaban has joined #openstack-kolla | 01:25 | |
| *** yuanying has joined #openstack-kolla | 01:25 | |
| *** jtriley has joined #openstack-kolla | 01:29 | |
| *** banix has joined #openstack-kolla | 01:29 | |
| *** banix has quit IRC | 01:39 | |
| *** tfukushima has joined #openstack-kolla | 01:51 | |
| *** Mr_Broken has quit IRC | 01:54 | |
| *** Mr_Broken has joined #openstack-kolla | 01:55 | |
| *** sacharya has joined #openstack-kolla | 01:59 | |
| *** ssurana has joined #openstack-kolla | 02:01 | |
| openstackgerrit | Yoshifumi Sumida proposed openstack/kolla: Heat - F21 images not available for download https://review.openstack.org/334729 | 02:02 |
|---|---|---|
| *** daneyon_ has joined #openstack-kolla | 02:02 | |
| *** zhurong has quit IRC | 02:02 | |
| *** zhurong has joined #openstack-kolla | 02:03 | |
| openstackgerrit | Yoshifumi Sumida proposed openstack/kolla: Heat - F21 images not available for download https://review.openstack.org/334729 | 02:06 |
| *** Jeffrey4l__ has quit IRC | 02:07 | |
| *** daneyon_ has quit IRC | 02:07 | |
| *** jtriley has quit IRC | 02:08 | |
| *** jtriley has joined #openstack-kolla | 02:12 | |
| *** ssurana has quit IRC | 02:13 | |
| *** williamcaban has quit IRC | 02:16 | |
| *** jtriley has quit IRC | 02:17 | |
| *** fragatina has joined #openstack-kolla | 02:18 | |
| *** fragatina has quit IRC | 02:18 | |
| *** fragatina has joined #openstack-kolla | 02:19 | |
| *** sacharya_ has joined #openstack-kolla | 02:24 | |
| *** sacharya has quit IRC | 02:27 | |
| *** banix has joined #openstack-kolla | 02:28 | |
| *** mark-casey has joined #openstack-kolla | 02:34 | |
| *** beekhof has quit IRC | 02:37 | |
| *** g3ek has quit IRC | 02:37 | |
| *** g3ek has joined #openstack-kolla | 02:38 | |
| *** sdake has joined #openstack-kolla | 02:40 | |
| mark-casey | sdake: ping | 02:41 |
| *** beekhof has joined #openstack-kolla | 02:41 | |
| *** salv-orl_ has quit IRC | 02:41 | |
| *** jtriley has joined #openstack-kolla | 02:43 | |
| *** Mr_Broken has quit IRC | 02:44 | |
| *** yuanying has quit IRC | 02:50 | |
| *** klint has joined #openstack-kolla | 02:52 | |
| *** sdake has quit IRC | 02:56 | |
| *** Mr_Broken has joined #openstack-kolla | 03:02 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update kollaglue repo on docker to kolla https://review.openstack.org/333110 | 03:03 |
| *** banix has quit IRC | 03:06 | |
| *** sdake has joined #openstack-kolla | 03:08 | |
| *** Mr_Broken has quit IRC | 03:10 | |
| *** Mr_Broken has joined #openstack-kolla | 03:12 | |
| *** haplo37 has quit IRC | 03:21 | |
| openstackgerrit | Hui Kang proposed openstack/kolla: Add etcd Docker container https://review.openstack.org/298451 | 03:23 |
| *** jtriley has quit IRC | 03:27 | |
| *** sdake has quit IRC | 03:27 | |
| *** haplo37 has joined #openstack-kolla | 03:29 | |
| *** yuanying has joined #openstack-kolla | 03:48 | |
| *** salv-orlando has joined #openstack-kolla | 03:48 | |
| *** salv-orlando has quit IRC | 03:57 | |
| *** yuanying has quit IRC | 04:00 | |
| *** yuanying has joined #openstack-kolla | 04:01 | |
| *** stvnoyes has quit IRC | 04:02 | |
| *** stvnoyes1 has joined #openstack-kolla | 04:03 | |
| *** y-sumida has joined #openstack-kolla | 04:05 | |
| *** y-sumida has left #openstack-kolla | 04:10 | |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Fix Mariadb restarting issue https://review.openstack.org/333753 | 04:22 |
| *** salv-orlando has joined #openstack-kolla | 04:23 | |
| *** Jeffrey4l has joined #openstack-kolla | 04:31 | |
| *** salv-orlando has quit IRC | 04:32 | |
| *** fragatina has quit IRC | 04:39 | |
| *** Mr_Broken has quit IRC | 04:42 | |
| *** daneyon_ has joined #openstack-kolla | 04:45 | |
| *** yuanying has quit IRC | 04:49 | |
| *** daneyon_ has quit IRC | 04:49 | |
| *** Mr_Broken has joined #openstack-kolla | 04:53 | |
| *** g3ek has quit IRC | 04:56 | |
| *** haplo37 has quit IRC | 04:56 | |
| *** salv-orlando has joined #openstack-kolla | 04:58 | |
| *** salv-orl_ has joined #openstack-kolla | 04:59 | |
| *** salv-orlando has quit IRC | 05:03 | |
| *** g3ek has joined #openstack-kolla | 05:03 | |
| *** haplo37 has joined #openstack-kolla | 05:04 | |
| *** salv-orl_ has quit IRC | 05:04 | |
| *** tfukushima has quit IRC | 05:13 | |
| *** ayoung has quit IRC | 05:14 | |
| *** hanchao has quit IRC | 05:16 | |
| *** mewald has joined #openstack-kolla | 05:18 | |
| *** fragatina has joined #openstack-kolla | 05:20 | |
| *** mewald has quit IRC | 05:24 | |
| *** JianqingJiang has joined #openstack-kolla | 05:27 | |
| *** hanchao has joined #openstack-kolla | 05:28 | |
| *** salv-orlando has joined #openstack-kolla | 05:33 | |
| *** Mr_Broken has quit IRC | 05:40 | |
| *** Mr_Broken has joined #openstack-kolla | 05:41 | |
| *** salv-orlando has quit IRC | 05:41 | |
| *** haplo37 has quit IRC | 05:44 | |
| *** g3ek has quit IRC | 05:45 | |
| *** g3ek- has joined #openstack-kolla | 05:47 | |
| *** g3ek- is now known as g3ek | 05:47 | |
| *** haplo37 has joined #openstack-kolla | 05:47 | |
| *** tfukushima has joined #openstack-kolla | 05:51 | |
| *** Mr_Broken has quit IRC | 05:56 | |
| *** Mr_Broken has joined #openstack-kolla | 05:57 | |
| mark-casey | anyone have a log or paste of running kolla-ansible upgrade? I'm trying to see some order of events and don't have a lab set up right now. Anything from the last few months would be fine | 06:02 |
| *** yuanying has joined #openstack-kolla | 06:05 | |
| *** salv-orlando has joined #openstack-kolla | 06:05 | |
| *** yuanying has quit IRC | 06:11 | |
| *** yuanying has joined #openstack-kolla | 06:12 | |
| mark-casey | found one, nevermind | 06:12 |
| *** Mr_Broken has quit IRC | 06:14 | |
| *** JeroenBo has joined #openstack-kolla | 06:14 | |
| *** bootsha has joined #openstack-kolla | 06:21 | |
| *** y-sumida has joined #openstack-kolla | 06:23 | |
| *** daneyon_ has joined #openstack-kolla | 06:33 | |
| *** daneyon_ has quit IRC | 06:37 | |
| *** anbehl has quit IRC | 06:52 | |
| *** Mr_Broken has joined #openstack-kolla | 06:57 | |
| *** mewald has joined #openstack-kolla | 06:58 | |
| *** b_bezak has joined #openstack-kolla | 07:01 | |
| mewald | Can Kolla deploy Glance, Nova and Cinder with connection information to an existing Ceph cluster that wasn't deployed by Kolla? | 07:06 |
| *** haplo37 has quit IRC | 07:14 | |
| *** g3ek has quit IRC | 07:14 | |
| *** g3ek has joined #openstack-kolla | 07:23 | |
| *** haplo37 has joined #openstack-kolla | 07:23 | |
| openstackgerrit | Merged openstack/kolla: Create ansible home directory in kolla-toolbox container https://review.openstack.org/334237 | 07:29 |
| *** athomas has joined #openstack-kolla | 07:30 | |
| mdnadeem | mark-casey, I have done some 2 week ago, here is ansible.log http://pastebin.com/ywRyhb5w | 07:31 |
| mdnadeem | mark-casey, Let me know if you need any other log | 07:31 |
| mark-casey | thanks :) | 07:31 |
| openstackgerrit | Merged openstack/kolla: Add test for remove container, get state and get env https://review.openstack.org/331556 | 07:33 |
| *** yuanying has quit IRC | 07:34 | |
| *** yuanying has joined #openstack-kolla | 07:34 | |
| *** yuanying has quit IRC | 07:35 | |
| *** belmoreira has joined #openstack-kolla | 07:35 | |
| *** yuanying has joined #openstack-kolla | 07:36 | |
| *** salv-orlando has quit IRC | 07:41 | |
| *** Serlex has joined #openstack-kolla | 07:50 | |
| *** diogogmt has joined #openstack-kolla | 07:53 | |
| *** shardy_afk is now known as shardy | 07:55 | |
| *** y-sumida has quit IRC | 07:57 | |
| *** kproskurin has joined #openstack-kolla | 07:59 | |
| mewald | what is nova-ssh container? looks like it's just a ssh service nothing else. Why? | 08:00 |
| *** sacharya_ has quit IRC | 08:01 | |
| *** g3ek has quit IRC | 08:04 | |
| *** y-sumida has joined #openstack-kolla | 08:05 | |
| *** dmk0202 has joined #openstack-kolla | 08:06 | |
| *** haplo37 has quit IRC | 08:06 | |
| Jeffrey4l | mewald, the nova migrate/resize/rebuild need the ssh service. nova copy the image to another host by using rsync | 08:08 |
| *** g3ek has joined #openstack-kolla | 08:11 | |
| *** haplo37 has joined #openstack-kolla | 08:11 | |
| *** salv-orlando has joined #openstack-kolla | 08:13 | |
| *** gfidente has joined #openstack-kolla | 08:15 | |
| *** salv-orlando has quit IRC | 08:17 | |
| *** y-sumida has quit IRC | 08:20 | |
| *** Serlex has quit IRC | 08:21 | |
| *** daneyon_ has joined #openstack-kolla | 08:21 | |
| *** daneyon_ has quit IRC | 08:25 | |
| *** y-sumida has joined #openstack-kolla | 08:27 | |
| *** b_bezak has quit IRC | 08:29 | |
| *** b_bezak has joined #openstack-kolla | 08:32 | |
| *** b_bezak has quit IRC | 08:32 | |
| *** b_bezak has joined #openstack-kolla | 08:33 | |
| openstackgerrit | Merged openstack/kolla: Update kollaglue repo on docker to kolla https://review.openstack.org/333110 | 08:38 |
| *** b_bezak_ has joined #openstack-kolla | 08:38 | |
| *** b_bezak has quit IRC | 08:38 | |
| *** mbound has joined #openstack-kolla | 08:42 | |
| *** bootsha has quit IRC | 08:43 | |
| *** salv-orlando has joined #openstack-kolla | 08:48 | |
| *** g3ek has quit IRC | 08:53 | |
| *** haplo37 has quit IRC | 08:53 | |
| *** haplo37 has joined #openstack-kolla | 08:54 | |
| *** g3ek has joined #openstack-kolla | 08:55 | |
| *** salv-orlando has quit IRC | 08:56 | |
| *** Serlex has joined #openstack-kolla | 08:58 | |
| *** salv-orlando has joined #openstack-kolla | 09:00 | |
| openstackgerrit | Merged openstack/kolla: Be smarter about what to do when making a docker client https://review.openstack.org/330171 | 09:10 |
| *** bootsha has joined #openstack-kolla | 09:14 | |
| *** yuanying has quit IRC | 09:26 | |
| *** prithiv has joined #openstack-kolla | 09:28 | |
| *** mark-casey has quit IRC | 09:28 | |
| *** y-sumida has quit IRC | 09:32 | |
| *** dwalsh has joined #openstack-kolla | 09:34 | |
| *** bootsha has quit IRC | 09:38 | |
| *** prithiv has quit IRC | 09:40 | |
| *** david-lyle has quit IRC | 09:44 | |
| *** david-lyle has joined #openstack-kolla | 09:46 | |
| *** bootsha has joined #openstack-kolla | 09:49 | |
| *** hanchao has quit IRC | 09:58 | |
| *** zhurong has quit IRC | 10:01 | |
| *** JianqingJiang has quit IRC | 10:11 | |
| *** tfukushima has quit IRC | 10:27 | |
| *** dwalsh has quit IRC | 10:27 | |
| *** dwalsh has joined #openstack-kolla | 10:29 | |
| Mr_Broken | @mewald: did you deploy kolla to a existing Ceph cluster ? or did you just use kolla-ansible reconfigure to fiddle with configs ? | 10:34 |
| *** yuanying has joined #openstack-kolla | 10:47 | |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Added unit test for image operation https://review.openstack.org/334917 | 10:54 |
| pbourke | mewald: not currently | 10:57 |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Added unit test for image operation https://review.openstack.org/334917 | 10:58 |
| pbourke | mewald: though you may be able to do it with a combo of augment files and customised container json files to pick up the keyring files | 10:58 |
| Mr_Broken | pbourke: are you refering to deploying kolla with external ceph cluster ? | 10:58 |
| Mr_Broken | I got the same task to do soon ;] | 10:59 |
| *** salv-orl_ has joined #openstack-kolla | 10:59 | |
| mewald | Mr_Broken pbourke: currently I did nothing, we are going to have to deploy OpenStack with an existing Fujitsu CD10000 (Ceph). So that's going to be a problem :D | 11:00 |
| *** salv-orlando has quit IRC | 11:02 | |
| Mr_Broken | mewald: yeah i need to do similar thing - havent looked at the docs yet, but I assumed it will be a matter of changing some cinder configuration files ? | 11:09 |
| *** anbehl has joined #openstack-kolla | 11:10 | |
| *** ppowell_ has joined #openstack-kolla | 11:16 | |
| *** tyrola has joined #openstack-kolla | 11:19 | |
| *** g3ek has quit IRC | 11:19 | |
| *** haplo37 has quit IRC | 11:20 | |
| *** fragatin_ has joined #openstack-kolla | 11:22 | |
| *** fragatina has quit IRC | 11:22 | |
| sean-k-mooney | deploying with an external ceph cluser would be a very nic enchancment | 11:24 |
| sean-k-mooney | it is needed for kolla-kubernetes anyway so it would be nice to have in kolla-ansible | 11:24 |
| *** haplo37 has joined #openstack-kolla | 11:25 | |
| *** g3ek has joined #openstack-kolla | 11:25 | |
| *** fragatin_ has quit IRC | 11:30 | |
| *** mliima has joined #openstack-kolla | 11:34 | |
| *** rhallisey has joined #openstack-kolla | 11:38 | |
| *** zhurong has joined #openstack-kolla | 11:42 | |
| *** fragatina has joined #openstack-kolla | 11:54 | |
| *** sdake has joined #openstack-kolla | 11:58 | |
| *** sdake_ has joined #openstack-kolla | 12:00 | |
| sdake_ | hey folks | 12:00 |
| sdake_ | morning | 12:00 |
| sdake_ | why oh wy do i wake up at 5am every day | 12:01 |
| *** dwalsh has quit IRC | 12:01 | |
| *** dwalsh has joined #openstack-kolla | 12:01 | |
| *** neilus has quit IRC | 12:02 | |
| *** sdake has quit IRC | 12:03 | |
| *** neilus has joined #openstack-kolla | 12:03 | |
| *** neilus has quit IRC | 12:05 | |
| *** neilus has joined #openstack-kolla | 12:05 | |
| *** g3ek has quit IRC | 12:07 | |
| *** haplo37 has quit IRC | 12:08 | |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Add an alternative disk provisioning method for Ceph https://review.openstack.org/334970 | 12:08 |
| mewald | Mr_Broken: Yeah that's not all unfortunately. Sure, some configuration needs to go into the ini file (e.g. glance-api.ini) but we need to place a ceph.conf and keyring in /etc/ceph/. | 12:08 |
| mewald | At the moment, that information is fetched executing "docker exec ceph auth get-or-create …" but with an existing Ceph Cluster it's not going to work like that | 12:09 |
| *** haplo37 has joined #openstack-kolla | 12:13 | |
| *** g3ek has joined #openstack-kolla | 12:13 | |
| *** neilus1 has joined #openstack-kolla | 12:16 | |
| *** neilus1 has joined #openstack-kolla | 12:17 | |
| *** athomas has quit IRC | 12:18 | |
| *** neilus has quit IRC | 12:18 | |
| *** williamcaban has joined #openstack-kolla | 12:20 | |
| sdake_ | mewald might i inquire what your speaking about | 12:20 |
| sdake_ | mewald are you talking about using external ceph? | 12:21 |
| Mr_Broken | sdake: ytes | 12:21 |
| Mr_Broken | *yes | 12:21 |
| sdake_ | i'd love to see that work | 12:21 |
| Mr_Broken | kolla deploy, and connecting to external ceph cluster | 12:21 |
| *** diogogmt has quit IRC | 12:21 | |
| Mr_Broken | yeah, me 2 :] | 12:21 |
| Mr_Broken | since i need to set it up next week :] | 12:21 |
| sdake_ | i dodnt think its hard | 12:21 |
| sdake_ | but i dont know for sure | 12:21 |
| sdake_ | i dont have any idea how ceph works or i'd do the implementation already | 12:21 |
| Mr_Broken | hopefully I will have some insight next week ;] | 12:22 |
| *** neilus1 has quit IRC | 12:22 | |
| mewald | I will set up a lab today and begin working on what needs to be done | 12:22 |
| sdake_ | Mr_Broken it will probablly require changes | 12:22 |
| Mr_Broken | my friend is seting up a ceph cluster right now, we will try to connect it to kolla as soon as its up | 12:22 |
| mewald | I browsed the code a bit an I am pretty confident its not a lot | 12:22 |
| *** diogogmt has joined #openstack-kolla | 12:22 | |
| sdake_ | mewald ya i think its probably asmall sizzed task | 12:23 |
| *** athomas has joined #openstack-kolla | 12:23 | |
| Mr_Broken | mewald: any input appreciated :D | 12:23 |
| sdake_ | what would help is if someone that undersstood ceph did the work :) | 12:23 |
| sdake_ | it would take me weeks to learn the ins and outs of ceph | 12:23 |
| sdake_ | which Iguess I should do anyway | 12:23 |
| *** berendt has joined #openstack-kolla | 12:23 | |
| mewald | sdake_: yeah that's why I am trying to do it. I know Ceph pretty well, BUT don't know Kolla very well yet :D | 12:24 |
| sdake_ | mewald cool if your going ot take it on i can help you around with kolla | 12:24 |
| sdake_ | is there an existing blueprint? | 12:25 |
| mewald | where would I check? | 12:25 |
| sdake_ | launchpad.net/kolla | 12:26 |
| sdake_ | i'll take a look | 12:26 |
| Mr_Broken | @mewald: you said that using ceph auth (http://docs.ceph.com/docs/hammer/man/8/ceph-authtool/) is not working in docker ? | 12:26 |
| Mr_Broken | or what is the problem there ? | 12:26 |
| sdake_ | https://blueprints.launchpad.net/kolla/+spec/external-ceph | 12:26 |
| mewald | yeah just found that too :) | 12:27 |
| mewald | I need some time to be able to understand this Gerrit thingy :D I am not really a developer ^^ | 12:27 |
| sdake_ | looks like some progress on the blueprint | 12:28 |
| mewald | You gave it a down vote :D | 12:31 |
| sdake_ | gave what a down vote | 12:31 |
| sdake_ | a -1 vote only means more work needs to be done on the patch | 12:32 |
| sdake_ | a -2 vote is a "down vote" or veto of the technical work | 12:32 |
| mewald | ah | 12:33 |
| *** diogogmt has quit IRC | 12:34 | |
| sean-k-mooney | -2 are also sometimes adminsitrative. e.g. we are past a code freeze or where a spec is required but not approve the code may be given a -2 untill the spec is approved. | 12:34 |
| sean-k-mooney | kolla tends not to use specs much though | 12:34 |
| sdake_ | ya - at midcycle i think we are going to have a 40 minute discussion about specs and their future use | 12:35 |
| sdake_ | kolla is becoming too hard to manage | 12:36 |
| sdake_ | i tihnk its important to operate without specs for as long as possible | 12:36 |
| sean-k-mooney | well as it grows specs become more needed for large changes but specs do slow down the development of smaller features | 12:36 |
| sdake_ | yup | 12:37 |
| sdake_ | tradeoff | 12:37 |
| sean-k-mooney | kolla currently has a nice blance where specs are only used for large or controveral features | 12:37 |
| sdake_ | the problem we have as the core team is we have 87 blueprints | 12:37 |
| mewald | sdake_: how do we proceed to make this work? What needs to be done with that Commit so it would pass and be merged? | 12:38 |
| sean-k-mooney | actully 5 of them are mine but i tink that can be reduced to 3 | 12:38 |
| sdake_ | mewald which commit ? | 12:38 |
| sdake_ | sean-k-mooney if you have bps i should remove let me know their links | 12:38 |
| sdake_ | sean-k-mooney all for tryign to get rid of work :) | 12:38 |
| mewald | sdake_: the one that implements external ceph? | 12:39 |
| mewald | -?+. | 12:39 |
| sdake_ | mewald link please? | 12:39 |
| mewald | https://review.openstack.org/#/c/289462/ | 12:39 |
| patchbot | mewald: patch 289462 - kolla (stable/liberty) - Modify the conditional strategy for cinder and gla... (ABANDONED) | 12:39 |
| sean-k-mooney | well basically i think we can combine 2 of the smaller blueprints into the host prep playbook. | 12:39 |
| sean-k-mooney | sdake_: i will look at them later today and either close them or let you know the links to delete them | 12:40 |
| sdake_ | sean-k-mooney cool thanks | 12:40 |
| sdake_ | mewald that patch was not submitted against mater | 12:40 |
| sdake_ | that is why it was -2ed | 12:40 |
| mewald | what is mater? :D | 12:40 |
| sdake_ | masater | 12:40 |
| sdake_ | the tip of dev | 12:41 |
| sdake_ | everything goes in master first | 12:41 |
| sdake_ | then the core team handles backports | 12:41 |
| *** matrohon has joined #openstack-kolla | 12:41 | |
| mewald | ah ok | 12:42 |
| *** neilus has joined #openstack-kolla | 12:42 | |
| *** bootsha has quit IRC | 12:43 | |
| mewald | so we can just commit it to master then? Or should I test it before that against my cluster? | 12:43 |
| *** neilus has joined #openstack-kolla | 12:45 | |
| sdake_ | mewald it also needs a rebase | 12:47 |
| sdake_ | mewald i dont know what htose two patches do | 12:47 |
| sean-k-mooney | sdake_: it allow you to deploy kolla with an external ceph cluster but still have cinder... use ceph | 12:48 |
| sean-k-mooney | so basically it will remove the requirement for kolla to have deployed the ceph cluster before kolla can use it | 12:49 |
| sdake_ | cool looks like the patch needs alot of work | 12:49 |
| sdake_ | has alot of comments in it | 12:49 |
| sean-k-mooney | its also currently abandoned https://review.openstack.org/#/c/289462 | 12:50 |
| sean-k-mooney | this will be very usefull to have though | 12:50 |
| *** haplo37 has quit IRC | 12:50 | |
| Mr_Broken | quick question: what does kolla uses - priviledged or unpriviledged containers for ceph deployment ? | 12:50 |
| sean-k-mooney | it will be needed for k8s | 12:50 |
| *** rmart04 has joined #openstack-kolla | 12:51 | |
| sean-k-mooney | Mr_Broken: i think privalaged for the OSDs | 12:51 |
| sean-k-mooney | Mr_Broken: not sure though | 12:51 |
| Mr_Broken | kk thanks | 12:51 |
| sean-k-mooney | Mr_Broken: https://github.com/openstack/kolla/blob/master/ansible/roles/ceph/tasks/start_osds.yml#L41 | 12:52 |
| sean-k-mooney | the ceph mon and rados gateway are unprivalaged | 12:52 |
| Mr_Broken | kk cool | 12:53 |
| *** klint has quit IRC | 12:56 | |
| sdake_ | mewald - if you like I can rebase that patch to master - but it has a whole lot of comments that need addressing | 12:57 |
| *** haplo37 has joined #openstack-kolla | 13:00 | |
| mewald | sdake_: all this is very new to me :) I just clone the Gerrit repository of Kolla and used "git fetch ssh://mewald@review.openstack.org:29418/openstack/kolla refs/changes/62/289462/1 && git checkout FETCH_HEAD" to get to the commit that includes one of the patches. How would I proceed? | 13:00 |
| sdake_ | mewald did that fetch work properly? | 13:01 |
| mewald | I said something about detached head - so Im not sure :D | 13:01 |
| sdake_ | mewald if you have 15 mins to spend on webex, i can wlak you through how to rebase the patch to master | 13:01 |
| sdake_ | its a little complicated since its also in merge conflict | 13:02 |
| mewald | sure would be awesome! | 13:02 |
| *** Mr_Broken has quit IRC | 13:02 | |
| sdake_ | but you will learn how to rebase | 13:02 |
| sdake_ | do you have a mac or windows? | 13:02 |
| mewald | mac | 13:02 |
| sdake_ | have 15 misn now? | 13:02 |
| mewald | yep | 13:02 |
| sdake_ | http://cisco.webex.com/join/stdake | 13:03 |
| sdake_ | anyone is welcome to join and watch in | 13:03 |
| *** v1k0d3n has joined #openstack-kolla | 13:03 | |
| *** jtriley has joined #openstack-kolla | 13:03 | |
| sdake_ | ok i'm in | 13:04 |
| mewald | working on it :) | 13:04 |
| sdake_ | ya - not sure what its like f your ssystem isn't etup | 13:05 |
| sdake_ | we will need to screen share | 13:05 |
| sdake_ | so have to connect via mac | 13:05 |
| *** dwalsh has quit IRC | 13:07 | |
| *** zhurong has quit IRC | 13:08 | |
| *** diogogmt has joined #openstack-kolla | 13:15 | |
| pbourke | Jeffrey4l: its not valid to dismiss bugs on LTS distros as "you need to upgrade" | 13:18 |
| *** sdake_ has quit IRC | 13:23 | |
| *** banix has joined #openstack-kolla | 13:24 | |
| *** sdake has joined #openstack-kolla | 13:24 | |
| mewald | sdake: what happened? :D | 13:24 |
| sdake | lost power | 13:24 |
| sdake | sorry | 13:24 |
| sdake | you may have to rejoin | 13:24 |
| mewald | ok :D | 13:25 |
| sdake | laptop went to sleep because not plugged in | 13:25 |
| ccesario | sdake, https://git.centos.org/blob/rpms!device-mapper-multipath.git/12917067990de3a976e7a1a5abfbe28891216c77/SOURCES!0106-RHBZ-1212590-dont-use-var.patch;jsessionid=glu0ds254mfg1v056w85c8q81#L18 need to be 'bindmounted' ? | 13:27 |
| *** huikang has joined #openstack-kolla | 13:31 | |
| *** inc0 has joined #openstack-kolla | 13:31 | |
| inc0 | good morning | 13:32 |
| huikang | monring | 13:32 |
| sean-k-mooney | o/ inc0 | 13:32 |
| sean-k-mooney | hi huikang inc0 how are ye both | 13:33 |
| inc0 | pretty well, how are things out there on islands? | 13:33 |
| inc0 | I heard it's pretty tense now | 13:33 |
| sean-k-mooney | with the brexit stuff? | 13:33 |
| inc0 | yeah | 13:34 |
| huikang | or the Euro Cup | 13:34 |
| sean-k-mooney | ireland is still asessing what the impact will be | 13:34 |
| sean-k-mooney | well of course the euros are more important :P | 13:35 |
| inc0 | I've heard that Northern Ireland is thinking of separation from UK | 13:35 |
| inc0 | yeah, Poland stronk in this euro \o/ | 13:35 |
| sean-k-mooney | well scotland are thingink of having a second reforendum on leaving the uk to remain in the eu | 13:36 |
| sean-k-mooney | if they do its very unlikly that the north will not follow suite | 13:36 |
| sean-k-mooney | grabing some tea | 13:37 |
| sean-k-mooney | be back soon | 13:37 |
| huikang | WoW; this is real Democratic | 13:37 |
| inc0 | well, technically united kingdom is not a country on it's own | 13:37 |
| inc0 | Scotland, Ireland and such retain certain amount of independence | 13:38 |
| huikang | yup | 13:38 |
| openstackgerrit | Mathias proposed openstack/kolla: Add external ceph support for cinder and glance https://review.openstack.org/310208 | 13:40 |
| *** haplo37 has quit IRC | 13:40 | |
| openstackgerrit | Mathias proposed openstack/kolla: Add external ceph support for cinder and glance https://review.openstack.org/310208 | 13:45 |
| *** banix has quit IRC | 13:45 | |
| *** daneyon_ has joined #openstack-kolla | 13:46 | |
| pbourke | sean-k-mooney: you based in ireland?? | 13:46 |
| inc0 | a yeah guys, you can drink with each other | 13:46 |
| openstackgerrit | Mathias proposed openstack/kolla: Add external ceph support for cinder and glance https://review.openstack.org/310208 | 13:46 |
| inc0 | my fault, I should introduce you to each other;) | 13:46 |
| inc0 | Mathias, are you here? | 13:47 |
| *** haplo37 has joined #openstack-kolla | 13:48 | |
| sbezverk | wirehead_ bootstrapping keystone reports completed state 2/4 and log for last task shows a traceback, see here: http://paste.openstack.org/show/523777/ | 13:50 |
| *** Mr_Broken has joined #openstack-kolla | 13:50 | |
| *** daneyon_ has quit IRC | 13:50 | |
| sbezverk | wirehead_ have you seen it? | 13:52 |
| *** ayoung has joined #openstack-kolla | 13:52 | |
| *** rhallisey has quit IRC | 13:53 | |
| *** JeroenBo has quit IRC | 13:54 | |
| *** rhallisey has joined #openstack-kolla | 13:54 | |
| sdake | i think pbourke wins the drinking contest | 13:54 |
| sdake | i was three sheets to the wind and pbourke had 3 or 4 beers on me | 13:55 |
| pbourke | lol | 13:55 |
| sdake | nd he looked sober as normal | 13:55 |
| pbourke | that's funny though didn't know there was a fellow irish man here | 13:55 |
| pbourke | may have both been at openstack days ireland and didnt know | 13:55 |
| sdake | i wishthere was an openstack meetup in pheonix | 13:55 |
| *** Mr_Broken has quit IRC | 13:57 | |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Add an alternative disk provisioning method for Ceph https://review.openstack.org/335046 | 13:57 |
| sdake | never having been to a meetup i wouldn't really know where to begin organizing one :) | 13:57 |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Add an alternative disk provisioning method for Ceph https://review.openstack.org/334970 | 13:58 |
| sdake | new vim command mewald taught me to wrap lines - gq | 13:58 |
| pbourke | sdake: im assuming theres support there from the TC for organising them | 13:58 |
| sdake | pbourke ya i'm overloaded | 13:59 |
| sdake | maybe next cycle | 13:59 |
| pbourke | afaik Intel were the main organisers of the Irelan one | 13:59 |
| pbourke | *Ireland | 13:59 |
| sdake | how many people were at the meetup? | 13:59 |
| sdake | is it monthly or every 6 months or what? | 14:00 |
| pbourke | I think these are annual | 14:00 |
| pbourke | you could easily do something smaller and less official | 14:00 |
| sean-k-mooney | pbourke: hi back from tea and yes im from ireland | 14:00 |
| pbourke | sean-k-mooney: hi :) | 14:01 |
| pbourke | sean-k-mooney: were you at the openstack day ireland? | 14:01 |
| pbourke | sdake: https://www.openstack.org/community/events/openstackdays | 14:01 |
| sean-k-mooney | pbourke: so the rest of my team were but no i did not attend | 14:02 |
| pbourke | ah | 14:02 |
| sean-k-mooney | pbourke: one of our team members was leaving so i wanted to send time with him before he did | 14:02 |
| *** zhurong has joined #openstack-kolla | 14:04 | |
| *** Mr_Broken has joined #openstack-kolla | 14:06 | |
| sean-k-mooney | my manager was one of the people organising the openstack ireland day this year. next year it will likely still have 1 confernece track but also have some breakout sessions | 14:07 |
| *** b_bezak has joined #openstack-kolla | 14:08 | |
| sean-k-mooney | by the way the kolla midcycle will be soon correct https://wiki.openstack.org/wiki/Sprints/KollaNewtonSprint | 14:09 |
| Jeffrey4l | pbourke, just curiosity that, the docker require kernel 3.10 minimum. is there really someone to install docker/kolla in RHEL 4? | 14:09 |
| sean-k-mooney | RHEL6 is below 3.10 | 14:10 |
| *** banix has joined #openstack-kolla | 14:10 | |
| sean-k-mooney | rhel 7 uses 3.10 with backports form 4.x | 14:10 |
| *** b_bezak_ has quit IRC | 14:11 | |
| Jeffrey4l | sean-k-mooney, if so. at least, it is bad idea to install docker/kolla in rhel 6, right? | 14:11 |
| Jeffrey4l | pbourke, ^ | 14:11 |
| pbourke | oracle VM has a backported kernel | 14:13 |
| pbourke | I assume similar is possible in RHEL | 14:13 |
| sean-k-mooney | how was the webex for the last mid cycle out of interest? i wont be able to attend in person my might try to join remotely | 14:13 |
| sean-k-mooney | Jeffrey4l: am docker 1.10 should be ok | 14:13 |
| Jeffrey4l | sean-k-mooney, what? | 14:13 |
| pbourke | these distros are making use of docker specifically to allow newer software on older user space libs | 14:13 |
| sean-k-mooney | jeffery docker 1.11.x was the first to use feature that actully needed 3.10 | 14:14 |
| pbourke | im not saying kolla has to 100% abide by older distros but should at least accept changes that helps them | 14:14 |
| Jeffrey4l | how the docker 1.10 ? kolla need docker >= 1.10 actually. | 14:14 |
| sdake | sean-k-mooney yup midcycle is 12/13 | 14:15 |
| sdake | sean-k-mooney will you be making it | 14:15 |
| Jeffrey4l | OK. backport kernel should work. | 14:15 |
| Jeffrey4l | another question, what happen to gpt? | 14:15 |
| sean-k-mooney | sdake: ill try and join remotely | 14:15 |
| Jeffrey4l | what not use gpt without udev?( if udev is bust) | 14:16 |
| pbourke | Jeffrey4l: how do you parse the disk labels in that case | 14:16 |
| pbourke | other than screen scraping parted or the like which requires a full rewrite of find_disks | 14:16 |
| sean-k-mooney | Jeffrey4l: so ya to summeries docker offically only support kernel 3.10+ but before the 1.11 release of docker docker did not use any feaures that prevented older kernels from working | 14:16 |
| Jeffrey4l | pbourke, the label is write in the disk. you can read the from the raw disk. | 14:17 |
| Jeffrey4l | pbourke, here is a example https://github.com/ceph/simplegpt/blob/master/simplegpt.py | 14:17 |
| Jeffrey4l | sean-k-mooney, ok. that's fine | 14:18 |
| pbourke | Jeffrey4l: I came across that code, its pretty low level though. Do we really want to be dealing with changes in GPT partition structures | 14:20 |
| pbourke | Jeffrey4l: I wouldnt be opposed to a rewrite of find_disks to use the above code but what I have works | 14:21 |
| sean-k-mooney | it would be nice if gpt labels were not reuired | 14:21 |
| Jeffrey4l | pbourke, is there any tools can read the gpt table? like parse the gparted output. it should be more stable. | 14:22 |
| sean-k-mooney | not sure if there is a cleaner way to do it generically | 14:22 |
| pbourke | sean-k-mooney: my patch does that | 14:23 |
| mewald | sdake: do you have any link that explains the steps we took with Git and Gerrit? I would like to have a reference that helps me with this a bit :) | 14:23 |
| sean-k-mooney | pbourke: oh cool do you have a link | 14:23 |
| sean-k-mooney | scrooling back to find it | 14:23 |
| pbourke | sean-k-mooney: yes Id appreciate a look https://review.openstack.org/#/c/334970/ | 14:24 |
| patchbot | pbourke: patch 334970 - kolla - Add an alternative disk provisioning method for Ceph | 14:24 |
| Jeffrey4l | pbourke, msdos disk has some issue i think. how to handle that: how to handle the external journal? | 14:25 |
| sean-k-mooney | msdos partition can have lable on primary partitions | 14:26 |
| Jeffrey4l | right. | 14:26 |
| sean-k-mooney | you can have up to 4 primary partition so that should be ok | 14:26 |
| Jeffrey4l | that's the point. | 14:26 |
| sean-k-mooney | the main issue with msdos partition tabels is i think the lables we use currently are too long | 14:26 |
| sdake | mewald http://docs.openstack.org/infra/manual/developers.html | 14:27 |
| sdake | so we can't use msdos partitions | 14:27 |
| sdake | just a limitation of the technology | 14:27 |
| sdake | the world has moved to eufi and GPT anyway | 14:27 |
| sean-k-mooney | i think the bootstrap lable works | 14:27 |
| *** rhallisey has quit IRC | 14:27 | |
| sdake | UEFI that is | 14:28 |
| sean-k-mooney | but if you want to use a cache partion with an external journal the current lable we use is too long for the msdos limit | 14:28 |
| *** rhallisey has joined #openstack-kolla | 14:28 | |
| sdake | Jeffrey4l i fixed the gates to build and test images correctly :) | 14:28 |
| *** g3ek has quit IRC | 14:28 | |
| pbourke | Are there concerns with this proposed approach. If not, its solving a specific problem - there's many ways to skin a cat unfortunately I dont have time to write them | 14:29 |
| Jeffrey4l | sdake, cool | 14:29 |
| sdake | pbourke which approach | 14:29 |
| sean-k-mooney | sdake: ubuntu,centos and rhel still default to creating msdos partions on install | 14:29 |
| *** haplo37 has quit IRC | 14:29 | |
| pbourke | sdake: just discussing https://review.openstack.org/#/c/334970/ | 14:29 |
| patchbot | pbourke: patch 334970 - kolla - Add an alternative disk provisioning method for Ceph | 14:29 |
| sdake | pbourke first question - what if different hardware has different disk identifiers | 14:31 |
| *** Mr_Broken has quit IRC | 14:31 | |
| pbourke | sdake: can you give an example? | 14:31 |
| *** haplo37 has joined #openstack-kolla | 14:31 | |
| sdake | which i think we cn expect to be the case | 14:31 |
| sdake | in my work lab | 14:31 |
| sdake | I have 3 storage nodes, they have 20 spinning disks | 14:31 |
| sdake | my compute nodes have 3-5 spinning disks | 14:31 |
| *** g3ek has joined #openstack-kolla | 14:31 | |
| *** diogogmt has quit IRC | 14:32 | |
| sean-k-mooney | well the main issue with using /dev/sdb,/dev/sdc is that i can change between reboot correct? | 14:32 |
| sdake | sean-k-mooney its not supposed to change | 14:32 |
| sean-k-mooney | e.g. if you add or remove another disk | 14:32 |
| sdake | a better way would beto use disk labels or wwns tho | 14:32 |
| pbourke | well it mounts them using UUIDs | 14:32 |
| *** diogogmt has joined #openstack-kolla | 14:32 | |
| sean-k-mooney | ok cool so udev maps the uuid to sdX | 14:32 |
| sdake | sean-k-mooney god i hope so by this point :) | 14:33 |
| sdake | pbourke it looks pretty good - i just worry about storage nodes with different storage setups | 14:33 |
| sdake | it would be nice if we could specify per node | 14:34 |
| sdake | is that possible? | 14:34 |
| pbourke | sdake: yes, you can set it using ansible host vars / group vars | 14:34 |
| sdake | per node? | 14:34 |
| Jeffrey4l | the bad point is: you restrict to use /dev/sda1 as data, and /dev/sda2 as journal. and never use /dev/sda3+ | 14:34 |
| Jeffrey4l | pbourke, | 14:34 |
| pbourke | Jeffrey4l: that's also the case with find_disks though? | 14:35 |
| Jeffrey4l | the more use case is: use ssd as journal and sata as data disk. | 14:35 |
| Jeffrey4l | pbourke, no. find_disks is more power than that. | 14:35 |
| sdake | ssds should also be used as cache tiers | 14:35 |
| Jeffrey4l | find_disk can use external journal. | 14:35 |
| sdake | well i'm glad someone is making ceph more usable in kolla | 14:35 |
| *** Mr_Broken has joined #openstack-kolla | 14:35 | |
| sdake | pbourke keep it up ;) | 14:36 |
| Jeffrey4l | in my env. I split one ssd disk into 4 partition, one partition for on sas disk. | 14:36 |
| sdake | Jeffrey4l the last time i used ceph it would only work on the entire disk | 14:36 |
| sdake | has that changed? | 14:36 |
| Jeffrey4l | sdake, as far as i know, cache tiers is not good. | 14:36 |
| Jeffrey4l | sdake, no. it support this all the time. | 14:36 |
| *** kproskurin has quit IRC | 14:37 | |
| sean-k-mooney | Jeffrey4l: i do someting similar on my home ceph server. 6 partitions on one ssd. (os + 4 journals+ cache osd partition) + 4 spinning disks for data | 14:37 |
| Jeffrey4l | sean-k-mooney, yes. this is more common use case. | 14:37 |
| sdake | Jeffrey4l yes but does kolla operate in that way? | 14:38 |
| sdake | Jeffrey4l last i tried, you had to format the entire disk as a label | 14:38 |
| Jeffrey4l | sdake, there is a more hack way. | 14:38 |
| sdake | i.e. one disk = one partition | 14:38 |
| sean-k-mooney | unfortunetly kolla didnt support that when i set it up so im looking forward to mewald's external ceph support | 14:38 |
| Jeffrey4l | sean-k-mooney, kolla support that, indeed. | 14:39 |
| sean-k-mooney | sdake: kolla now has partition support | 14:39 |
| Jeffrey4l | let me show u | 14:39 |
| sdake | sweet | 14:39 |
| sdake | sam said it was impossible ;) | 14:39 |
| sdake | can someone document it? | 14:39 |
| sean-k-mooney | sdake: its documented in his blog but not in our ceph guide | 14:39 |
| sdake | or tell me how and i'll document it | 14:39 |
| sean-k-mooney | sdake: we need to port it | 14:40 |
| Jeffrey4l | sdake, it is possible. but it lack of some doc. | 14:40 |
| Jeffrey4l | sam wrote a article about this. | 14:40 |
| sdake | sean-k-mooney did he have code changes | 14:40 |
| sean-k-mooney | sdake: no code chages its merged in master/mitaka | 14:40 |
| sean-k-mooney | sdake: https://yaple.net/2016/03/20/deploying-openstack-mitaka-with-kolla-docker-and-ansible/ | 14:40 |
| sean-k-mooney | is sams site down for others or is it just slow loading? | 14:41 |
| sdake | down | 14:41 |
| sean-k-mooney | oh it just loaded for me | 14:41 |
| mewald | takes ages | 14:41 |
| sdake | oh slow :) | 14:42 |
| Jeffrey4l | i opened it. | 14:42 |
| mewald | browser is still "resolving host" :D | 14:42 |
| Jeffrey4l | it is this article. | 14:42 |
| pbourke | try the google cache | 14:42 |
| sean-k-mooney | so ya you bassicaly add _J for the journal to the lables | 14:42 |
| Jeffrey4l | write. | 14:42 |
| sean-k-mooney | KOLLA_CEPH_OSD_BOOTSTRAP_1 first data partion KOLLA_CEPH_OSD_BOOTSTRAP_1_J for journal partion | 14:42 |
| sean-k-mooney | it also should work for cache partions | 14:43 |
| Jeffrey4l | correct. this require make the labels manually as expected. | 14:43 |
| Jeffrey4l | sean-k-mooney, no. in my env, i never used the cache partition. | 14:43 |
| sdake | all that works in master? | 14:43 |
| Jeffrey4l | sdake, all the branches i think | 14:43 |
| sdake | sweet | 14:43 |
| sean-k-mooney | sdake: it should but i dont think we test it so its probably broken | 14:43 |
| Jeffrey4l | ( we are using mitaka deploy liberty) so all | 14:43 |
| Jeffrey4l | sean-k-mooney, I use this. really. | 14:44 |
| Jeffrey4l | sean-k-mooney, in several env. it works very well. | 14:44 |
| sean-k-mooney | Jeffrey4l: well i was more commenting on the fact that if its not tested in openstack its usually broken | 14:44 |
| sean-k-mooney | we should have a gate job that tests it | 14:45 |
| pbourke | Jeffrey4l: let me review the journal stuff and get back to you | 14:45 |
| Jeffrey4l | sean-k-mooney, yes. we should. but now, kolla even has not ceph gate. | 14:45 |
| *** zhiwei has quit IRC | 14:45 | |
| sean-k-mooney | Jeffrey4l: fair point :) | 14:45 |
| sean-k-mooney | it would be nice to have the ablitiy to use files for osd in kolla to make gate testing of ceph simpler but we should be able to use loop back devices to test this in the gate | 14:46 |
| sdake | loopback is what we use | 14:46 |
| Jeffrey4l | sdake, pbourke sean-k-mooney see my env to use the external journal. http://paste.openstack.org/show/523782/ | 14:47 |
| Jeffrey4l | i even split the ssd into multi data partition to improve the performance. | 14:47 |
| sean-k-mooney | 20GB journals are quite big | 14:48 |
| Jeffrey4l | sean-k-mooney, what value are u using.. | 14:48 |
| sean-k-mooney | thats what i was planning on using too by from talking to sam he said they could be smaller | 14:48 |
| sdake | pbourke might I ask what problem your tyring to solve - is that gpt isn't in use by distros? | 14:48 |
| Jeffrey4l | sdake, i think the root cause it udev rather than the gpt.. | 14:49 |
| pbourke | sdake: the problem is udev is too old on some distros to work with kolla | 14:49 |
| pbourke | sdake: RHEL 6 for example | 14:49 |
| sean-k-mooney | ceph deply uses 5GB i think on a 2TB drive if you let it deploy ceph | 14:49 |
| sdake | which distros in particular | 14:49 |
| *** thumpba has joined #openstack-kolla | 14:49 | |
| Jeffrey4l | sean-k-mooney, hmm.. i am head that 5G is more small for that. | 14:49 |
| sdake | kolla is not meant for rhel6 fwiw :) | 14:49 |
| sdake | can we have a quck improptu discussion about 16.04 | 14:50 |
| pbourke | sdake: yes but there's no reason we can't do these distros | 14:50 |
| sdake | i really would like to see 16.04 implemented in newton | 14:50 |
| sdake | since 14.04 is byebye | 14:50 |
| sean-k-mooney | Jeffrey4l: as far as i know there is no performance hit to larger journals so using 20GB journals should be fine | 14:50 |
| pbourke | sdake: RHEL 6 is LTS till 2020 | 14:50 |
| sdake | anyone now the status of thi work? | 14:50 |
| sdake | pbourke yes I know, but rdo for example I think doesnt support rhel6 | 14:50 |
| Jeffrey4l | sdake, pbourke is talking about the host machine . not the docker image. | 14:50 |
| sdake | i understand but ideally they should match | 14:51 |
| pbourke | i diagree they should match | 14:51 |
| sean-k-mooney | sdake: well 14.04 is not eol for 2.5 years but it wont support openstack byond mitaka. | 14:51 |
| pbourke | *disagree | 14:51 |
| sean-k-mooney | sdake: i would like to see 16.04 though | 14:51 |
| sdake | well its eol for openstack then :) | 14:51 |
| sdake | pbourke cn you expand | 14:52 |
| *** tyrola has quit IRC | 14:52 | |
| sean-k-mooney | for what its worth 14.04 docker images on a 16.04 host seams to work fine as it is but we have to update the quickstart somewhat | 14:52 |
| *** tyrola has joined #openstack-kolla | 14:52 | |
| pbourke | sdake: big selling point of containers is the whole "write once run anywhere" thing. Its not true for every case but there's no need to enforce the container base to match the host | 14:53 |
| sean-k-mooney | basically the same systemd changes that you make on centos are need on the host for Ubuntu 16.04 hosts | 14:53 |
| sean-k-mooney | some filepaths are different though | 14:53 |
| sdake | pbourke write once run anywhere is a fallcy | 14:53 |
| sdake | even docker inc has admitted as such | 14:53 |
| sdake | but ya we dont have to enforce it | 14:53 |
| pbourke | sdake: i know there's issues with say centos containers on ubuntu | 14:54 |
| sean-k-mooney | so you can have a different host os to container os but you need to carful on the host to configure selinux/apparmor so that it works correctly | 14:54 |
| sdake | you can run some ubuntu containers on centos as well :) | 14:54 |
| pbourke | but saying you have to have centos 7 containers to run on centos 7 is bad | 14:54 |
| Jeffrey4l | i think we need set up some rule. like does the kolla need support rhel6? | 14:54 |
| Jeffrey4l | if so, we need have gate to test this. | 14:54 |
| sdake | rhel6 isnt' in the upstream gates | 14:54 |
| sdake | so we cant gate on rhel6 | 14:54 |
| Jeffrey4l | so i can say, we do not support rhel6 . | 14:55 |
| pbourke | i dont suggest we officially "support" these OSes | 14:55 |
| Jeffrey4l | because it is never tested. | 14:55 |
| sdake | i'm not sure how long 14.04 will be in the usptream gates either | 14:55 |
| pbourke | but should not reject patches that help their case | 14:55 |
| *** zhurong has quit IRC | 14:55 | |
| sdake | pbourke ++ | 14:55 |
| sdake | pbourke that said people do come looking to us for help | 14:56 |
| sdake | and I don't have any centos6 setup | 14:56 |
| sean-k-mooney | is kolla only intended to support the latest lts of any release. so rhel/centos 7 and ubuntu 16.04 would be what i gues we would support | 14:56 |
| sean-k-mooney | for what is worth i think infra where planning on swapping the upstream gate to 16.04 by mid july | 14:57 |
| sean-k-mooney | i know they had the devstack gate job working at the summit | 14:57 |
| Jeffrey4l | sean-k-mooney, at least, we test the kolla on rhel/cento 7 ubuntu 14.04 now. we have no other os gate to ensure that. | 14:57 |
| sean-k-mooney | sdake: you were refering to the container os though right. not host os | 14:58 |
| Jeffrey4l | but, at the udev bug issue point, add such fix make the kolla more complicated. :( | 14:59 |
| sdake | actually rhel isn't in the gates either | 14:59 |
| sean-k-mooney | so for newton you would like to see the ubuntu contaier base os move to ubuntu 16.04 | 14:59 |
| sdake | here is my take | 14:59 |
| sdake | : | 14:59 |
| sdake | vendors need some things like rhel6 implementation and 14.04 implemeentaitons, and we shouldn't try to stop that | 14:59 |
| sdake | that said, we shoudl have a big fat warning in our documentation stating "these are the distros kolla are tested with, these are the ones known tow ork with - the ones that work with your on your own" | 15:00 |
| sdake | (in better english:) | 15:00 |
| britthouser | Reading the scrollback regarding disk order re-arranging. | 15:00 |
| britthouser | We've seen storage controllers re-arrange the disk orders | 15:00 |
| sean-k-mooney | since cannonical will only be supporting Newton on 16.04/16.10 i think we do need to transition form 14.04 this cycle though. | 15:01 |
| britthouser | but /dev/disk/by-path has always been consistent for us | 15:01 |
| *** Mr_Broken has quit IRC | 15:01 | |
| pbourke | i guess the issue is I've seen udev fail even on centos 7 | 15:02 |
| pbourke | steve hindle also has a bug open related to this | 15:03 |
| pbourke | so its not 100% related to older distros | 15:03 |
| Jeffrey4l | pbourke, i am using centos 7 + kolla + ceph. never hit any issue. | 15:03 |
| pbourke | i get that | 15:04 |
| pbourke | Jeffrey4l: are you running bare metal? | 15:04 |
| Jeffrey4l | pbourke, right. a prod env. | 15:05 |
| pbourke | someone mentioned the issue can arise in vms | 15:05 |
| pbourke | *VMs | 15:05 |
| inc0 | Jeffrey4l, there can be lots of variations | 15:05 |
| inc0 | fact that it never happened to you may mean that you're just lucky | 15:05 |
| Jeffrey4l | pbourke, vms.. really? | 15:05 |
| pbourke | Jeffrey4l: yes | 15:06 |
| sdake | britthouser that is the wwn addressing right? | 15:06 |
| Jeffrey4l | so it must related to the kvm. because it can not provide a better *physical* env. | 15:06 |
| britthouser | no, sdake, /dev/disk/by-id is wwn address which is consistent also | 15:06 |
| britthouser | but /dev/disk/by-path lets gives you consistent name which you can also narrow down by controller | 15:07 |
| sdake | ok, well i dont see objection to adding things to make kolla work with rhel6 | 15:07 |
| britthouser | e.g. on UCS you can see that drives on this controller are flexflash, drives on another controller are the boot SSD, and drives on a different controller are attached to the RAID card. | 15:07 |
| sdake | (I mean technical objection) | 15:07 |
| sdake | other then it complicates things | 15:08 |
| *** Mr_Broken has joined #openstack-kolla | 15:08 | |
| Jeffrey4l | britthouser, kolla need label to record which disk the osd container need. | 15:08 |
| *** sacharya has joined #openstack-kolla | 15:09 | |
| pbourke | britthouser: that's good info to know | 15:09 |
| pbourke | britthouser: sounds like will need to use something other than /dev/xxx | 15:10 |
| britthouser | Yeah I think a /dev/disk/by-(whatever) would be more consistent, but would require some user education | 15:10 |
| *** sacharya_ has joined #openstack-kolla | 15:12 | |
| *** Mr_Broken has quit IRC | 15:12 | |
| britthouser | lol...I see you already addd that info to the review pbourke. =) | 15:12 |
| pbourke | britthouser: made sure to get it in there as is important! | 15:12 |
| pbourke | last thing we want is to wipe the wrong disk | 15:12 |
| britthouser | I'm just slow this morning | 15:12 |
| *** sacharya has quit IRC | 15:14 | |
| Jeffrey4l | sean-k-mooney, here is how to calculate the journal size: journal size = (2 * (expected bandwidth throughput * filestore max sync interval)) | 15:16 |
| Jeffrey4l | For a well optimised 10GB network: 12,5GB = (2 * (1250 * 5)) | 15:16 |
| *** g3ek has quit IRC | 15:18 | |
| *** Mr_Broken has joined #openstack-kolla | 15:18 | |
| *** b_bezak has quit IRC | 15:19 | |
| *** haplo37 has quit IRC | 15:19 | |
| *** b_bezak has joined #openstack-kolla | 15:19 | |
| *** haplo37 has joined #openstack-kolla | 15:19 | |
| *** g3ek has joined #openstack-kolla | 15:20 | |
| *** b_bezak has quit IRC | 15:24 | |
| *** Mr_Broken has quit IRC | 15:27 | |
| *** Mr_Broken has joined #openstack-kolla | 15:30 | |
| *** belmoreira has quit IRC | 15:31 | |
| sdake | can someone that is a ceph rcoket scientist translate sam's blog post into the ceph user guide? | 15:33 |
| *** Mr_Broken has quit IRC | 15:34 | |
| pbourke | imagine if all blog posts were docs | 15:37 |
| *** y-sumida has joined #openstack-kolla | 15:37 | |
| pbourke | i'll try if I get time | 15:38 |
| pbourke | need to see if my patch will work with what he proposes first | 15:38 |
| sean-k-mooney | Jeffrey4l: yes that sounds like what sam told me at the summit | 15:40 |
| Jeffrey4l | i have 20G network. so 20G journal is find. sean-k-mooney :D | 15:40 |
| *** dmk0202 has quit IRC | 15:42 | |
| sean-k-mooney | i still only have 1G at home and no ceph at work :( yet :) | 15:43 |
| *** pbourke has quit IRC | 15:43 | |
| *** pbourke has joined #openstack-kolla | 15:43 | |
| *** jmccarthy has joined #openstack-kolla | 15:44 | |
| *** rhallisey has quit IRC | 15:45 | |
| *** rhallisey has joined #openstack-kolla | 15:46 | |
| *** jmccarthy has left #openstack-kolla | 15:47 | |
| openstackgerrit | Joshua Harlow proposed openstack/kolla: Consistently use sys.exit https://review.openstack.org/334690 | 15:49 |
| openstackgerrit | Joshua Harlow proposed openstack/kolla: Consistently use sys.exit https://review.openstack.org/334690 | 15:50 |
| *** y-sumida has quit IRC | 15:53 | |
| *** v1k0d3n has quit IRC | 15:53 | |
| *** sdake_ has joined #openstack-kolla | 15:54 | |
| *** Jeffrey4l has quit IRC | 15:55 | |
| *** sdake has quit IRC | 15:57 | |
| *** haplo37 has quit IRC | 16:04 | |
| *** g3ek has quit IRC | 16:04 | |
| *** haplo37 has joined #openstack-kolla | 16:06 | |
| *** g3ek has joined #openstack-kolla | 16:07 | |
| *** rmart04 has quit IRC | 16:09 | |
| *** daneyon_ has joined #openstack-kolla | 16:28 | |
| *** vhosakot has joined #openstack-kolla | 16:32 | |
| *** ravig has joined #openstack-kolla | 16:32 | |
| *** daneyon_ has quit IRC | 16:33 | |
| *** salv-orl_ has quit IRC | 16:38 | |
| *** neilus has quit IRC | 16:40 | |
| *** dwalsh has joined #openstack-kolla | 16:47 | |
| *** g3ek has quit IRC | 16:52 | |
| *** haplo37 has quit IRC | 16:52 | |
| *** ravig has quit IRC | 16:53 | |
| *** ravig has joined #openstack-kolla | 16:54 | |
| *** haplo37 has joined #openstack-kolla | 16:55 | |
| *** g3ek has joined #openstack-kolla | 16:55 | |
| *** zhurong has joined #openstack-kolla | 16:56 | |
| *** Serlex has quit IRC | 16:56 | |
| *** matrohon has quit IRC | 17:04 | |
| *** dwalsh has quit IRC | 17:05 | |
| *** mbound has quit IRC | 17:17 | |
| *** salv-orlando has joined #openstack-kolla | 17:18 | |
| *** mewald has quit IRC | 17:18 | |
| *** ravig has quit IRC | 17:20 | |
| *** aNupoisc has joined #openstack-kolla | 17:24 | |
| *** vhosakot has quit IRC | 17:30 | |
| *** vhosakot has joined #openstack-kolla | 17:36 | |
| *** mark-casey has joined #openstack-kolla | 17:39 | |
| *** vhosakot has quit IRC | 17:39 | |
| *** haplo37 has quit IRC | 17:41 | |
| *** haplo37 has joined #openstack-kolla | 17:46 | |
| *** vhosakot has joined #openstack-kolla | 17:47 | |
| *** ravig has joined #openstack-kolla | 17:51 | |
| *** ravig has quit IRC | 17:55 | |
| *** fragatin_ has joined #openstack-kolla | 17:55 | |
| *** fragatina has quit IRC | 17:55 | |
| *** diogogmt has quit IRC | 17:58 | |
| *** banix has quit IRC | 17:59 | |
| *** diogogmt has joined #openstack-kolla | 18:00 | |
| *** zhurong has quit IRC | 18:01 | |
| *** ravig has joined #openstack-kolla | 18:01 | |
| *** ravig has quit IRC | 18:03 | |
| *** banix has joined #openstack-kolla | 18:03 | |
| *** ravig has joined #openstack-kolla | 18:03 | |
| *** huikang_ has joined #openstack-kolla | 18:06 | |
| *** salv-orlando has quit IRC | 18:13 | |
| *** diogogmt has quit IRC | 18:16 | |
| *** mbound has joined #openstack-kolla | 18:18 | |
| *** mewald has joined #openstack-kolla | 18:19 | |
| *** mewald has left #openstack-kolla | 18:22 | |
| *** mewald has joined #openstack-kolla | 18:23 | |
| *** mark-casey has quit IRC | 18:23 | |
| *** mbound has quit IRC | 18:24 | |
| *** mark-casey has joined #openstack-kolla | 18:24 | |
| mark-casey | sdake_: ping | 18:28 |
| *** diogogmt has joined #openstack-kolla | 18:28 | |
| sdake_ | mark-casey shoot | 18:28 |
| mark-casey | Was reading eavesdrop yesterday and I wrote a blurb on what xtrabackup is doing for us; if that is still an issue. Pre-wrote it in an attempt at being succinct but not sure it worked. sorry for minor wall of text | 18:30 |
| mark-casey | When a galera node fails it needs a state transfer from a healthy node **when it is restarting**. This is essentially just making a backup on a healthy node and sending it to the node that has failed and is recovering. The TL;DR is that without xtrabackup the "donor node" will become read-only for the duration of the state transfer. If two out of three nodes fail and then restart, the entire db will be unavailable until one of th | 18:30 |
| mark-casey | the issue is in getting a consistent backup from a node while the cluster is running, as you obviously do not want to restore the failed node with tables from two (or more) different points in time. Xtrabackup uses InnoDB crash logs to get a consistent state w/out blocking writes, while switching to rsync for SST will block writes | 18:33 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Liberty is now on 1.1.2 prerelease https://review.openstack.org/334637 | 18:33 |
| openstackgerrit | Merged openstack/kolla-kubernetes: Glance works without host network https://review.openstack.org/333022 | 18:33 |
| sdake_ | mark-casey so if Iprocessed that correctly we could offer both modes | 18:34 |
| sdake_ | im glad to finally hear what why xtradbbackup is needed | 18:34 |
| inc0 | sdake_, mark-casey that makes sense | 18:35 |
| inc0 | but I'd say we need xtrabackup | 18:35 |
| mark-casey | :D yes - could offer both. you'd need the mariadb config template to switch the value of wsrep_sst_method and possibly wsrep_sst_donor_rejects_queries, and then tell the script that informs haproxy who is healthy to exclude donors | 18:35 |
| inc0 | as having read-only master node effectively causes downtime | 18:35 |
| inc0 | and having both modes may be hard to maintain | 18:36 |
| mark-casey | you only go read-only if you get to a single healthy node left, but are still -1 on HA magic for all failure scenarios | 18:36 |
| sdake_ | mark-casey i worked on ha tech for 10+ years | 18:37 |
| sdake_ | more magic is not necessarily a good thing :) | 18:37 |
| inc0 | well, if healthy node goes read only it will have to sync itself later on too | 18:37 |
| inc0 | sdake_, is it critical for you to get rid of it? | 18:38 |
| *** ravig has quit IRC | 18:38 | |
| sdake_ | nah i just dont like deps | 18:38 |
| mark-casey | inc0: IIRC background replication continues, but I personally still favor keeping read/write on donors | 18:38 |
| inc0 | sdake_, I'd keep it in then. For the sake of maintainability | 18:39 |
| inc0 | less logic == good | 18:39 |
| sdake_ | ya i can deal with that on my own if its as simple as changing a few flags | 18:39 |
| inc0 | not really | 18:39 |
| inc0 | it will change way DB servers will behave | 18:40 |
| sdake_ | mark-casey can you send me an email to stdake@cisco.com with what you suggest i change to get it operational | 18:40 |
| sdake_ | inc0 i mean for my internal stuff | 18:40 |
| inc0 | sdake_, oh, ok | 18:40 |
| mark-casey | sdake_: sure | 18:40 |
| inc0 | mark-casey, cc michaljastrzebski@intel.com please | 18:40 |
| inc0 | michal.jastrzebski@intel.com | 18:41 |
| mark-casey | inc0: will do | 18:41 |
| inc0 | I should know my email address... | 18:41 |
| inc0 | thanks | 18:41 |
| *** athomas has quit IRC | 18:41 | |
| *** dwalsh has joined #openstack-kolla | 18:46 | |
| *** banix has quit IRC | 18:49 | |
| *** banix has joined #openstack-kolla | 18:55 | |
| *** huikang_ has quit IRC | 18:59 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Fix upgrades fail to upgrade https://review.openstack.org/334644 | 19:00 |
| *** bootsha has joined #openstack-kolla | 19:01 | |
| *** salv-orlando has joined #openstack-kolla | 19:01 | |
| *** huikang_ has joined #openstack-kolla | 19:01 | |
| *** sdake has joined #openstack-kolla | 19:02 | |
| *** sdake_ has quit IRC | 19:03 | |
| *** huikang_ has quit IRC | 19:06 | |
| *** daneyon_ has joined #openstack-kolla | 19:10 | |
| *** daneyon_ has quit IRC | 19:15 | |
| *** sdake_ has joined #openstack-kolla | 19:16 | |
| *** tyrola_ has joined #openstack-kolla | 19:17 | |
| *** athomas has joined #openstack-kolla | 19:19 | |
| *** sdake has quit IRC | 19:20 | |
| *** tyrola has quit IRC | 19:21 | |
| *** tyrola_ has quit IRC | 19:21 | |
| *** mummer has joined #openstack-kolla | 19:25 | |
| *** dwalsh has quit IRC | 19:25 | |
| sdake_ | anyone have any idea what this means: GATHERING FACTS *************************************************************** | 19:26 |
| sdake_ | failed: [minime-02] => {"cmd": "/usr/local/sbin -a", "failed": true, "rc": 13} | 19:26 |
| sdake_ | msg: [Errno 13] Permission denied | 19:26 |
| sean-k-mooney | it looks like a variable whas not defined | 19:28 |
| sean-k-mooney | e.g. it should /usr/local/sbin/somthing -a | 19:28 |
| sdake_ | stable/lbierty multinode eploy | 19:28 |
| sdake_ | aio works | 19:29 |
| sdake_ | only change is to multinode | 19:29 |
| *** aNupoisc has quit IRC | 19:30 | |
| sean-k-mooney | i assume that minime-02 is in you /etc/hosts or dns and the ansible user can ssh to it? | 19:30 |
| *** ravig has joined #openstack-kolla | 19:30 | |
| sdake_ | yup and resolving via resolver library properly | 19:30 |
| sdake_ | 1 and 3 work | 19:31 |
| sean-k-mooney | thats odd :) | 19:31 |
| sean-k-mooney | sorry could not resist | 19:31 |
| sdake_ | can ssh in as root to minime-02 | 19:31 |
| sdake_ | and the other nodes | 19:31 |
| sdake_ | it is odd ;) | 19:31 |
| sdake_ | i just caught that | 19:31 |
| sdake_ | slow today :) | 19:31 |
| *** ravig has quit IRC | 19:32 | |
| sdake_ | all machines have selinux disabled and sameetc hosts file | 19:32 |
| *** ravig has joined #openstack-kolla | 19:33 | |
| sean-k-mooney | i have not seen that before. does the /etc/kolla directory exist. im guessing its failing right at the start in the common role? | 19:33 |
| sdake_ | first task | 19:34 |
| sdake_ | aio works though | 19:34 |
| sdake_ | which is wier | 19:34 |
| sdake_ | d | 19:34 |
| sdake_ | do you know which task does that gathering facts thing | 19:34 |
| sean-k-mooney | aio works in minime-02 | 19:34 |
| sdake_ | i have not tried that | 19:34 |
| sean-k-mooney | or on the deployment host | 19:34 |
| sdake_ | on minime3 and 1 aio works | 19:34 |
| sean-k-mooney | maybe run the precheck on minime2 if you havent | 19:34 |
| *** ayoung has quit IRC | 19:35 | |
| mark-casey | sdake_: pretty sure it's setup module - it is implicit if gather_facts: false is not set on a play | 19:35 |
| mark-casey | http://docs.ansible.com/ansible/setup_module.html | 19:35 |
| *** fragatin_ has quit IRC | 19:36 | |
| sdake_ | what binary is it trying to run | 19:36 |
| sdake_ | how would I find that out? | 19:36 |
| *** bootsha has quit IRC | 19:37 | |
| sdake_ | nm i'm a dummy | 19:37 |
| sdake_ | i was trying t o run as non-root | 19:37 |
| sdake_ | which i knkow works | 19:37 |
| sdake_ | but requires some magic incantations which are undocumented | 19:37 |
| sdake_ | inc0 re that upgrade problem | 19:38 |
| inc0 | yeah? | 19:38 |
| sdake_ | inc0 you said you didn't want the error to silenty as - is there a way to give a warning instead | 19:38 |
| sdake_ | because as is what we have is busted | 19:38 |
| inc0 | that's actually a good question | 19:38 |
| inc0 | let me get back to you, I'll need to dig into ansible | 19:38 |
| sdake_ | TASK: [mariadb | Copying over galera.cnf] ************************************* | 19:39 |
| sdake_ | fatal: [minime-03] => {'msg': "One or more undefined variables: 'dict object' has no attribute u'ansible_p1p1'", 'failed': True} | 19:39 |
| sdake_ | fatal: [minime-03] => {'msg': 'One or more items failed.', 'failed': True, 'changed': False, 'results': [{'msg': "One or more undefined variables: 'dict object' has no attribute u'ansible_p1p1'", 'failed': True}]} | 19:39 |
| sdake_ | any ieas? | 19:39 |
| sdake_ | guess i should have pasted that | 19:39 |
| sdake_ | sorry | 19:39 |
| sdake_ | p1p1 = network interface | 19:40 |
| sean-k-mooney | sdake_: yes your network interface in your global.yml is wronge | 19:40 |
| sean-k-mooney | sdake_: or on that node its named something else | 19:40 |
| sdake_ | unlikely | 19:41 |
| sdake_ | all machines have p1p1 as their primary network 10gig interface | 19:41 |
| *** rmart04 has joined #openstack-kolla | 19:41 | |
| sdake_ | all are up and running | 19:41 |
| *** ravig has quit IRC | 19:42 | |
| sean-k-mooney | and i take it network_interface: is p1p1. | 19:42 |
| *** ravig has joined #openstack-kolla | 19:42 | |
| sdake_ | network_interface: "p1p1" | 19:42 |
| sean-k-mooney | strange i have only seen that fail if i set it wrong | 19:42 |
| sdake_ | ya this is stable liberty | 19:42 |
| sdake_ | i haven't seen it fail in a long long time like this | 19:43 |
| sdake_ | docker engine configured properly on all nodes | 19:43 |
| sean-k-mooney | dose that interface have an ip? | 19:44 |
| sean-k-mooney | on minime03 | 19:44 |
| sdake_ | yes as it should on all 3 nnodes | 19:44 |
| *** rmart04 has quit IRC | 19:44 | |
| sdake_ | all 3 nodes have the correct ip adddress | 19:44 |
| sean-k-mooney | ok if i remember correctly that is what that variabel used as a key to retrive | 19:47 |
| sean-k-mooney | https://github.com/openstack/kolla/blob/stable/liberty/ansible/roles/mariadb/templates/galera.cnf.j2#L3 | 19:47 |
| *** mliima has quit IRC | 19:47 | |
| *** v1k0d3n has joined #openstack-kolla | 19:47 | |
| *** fragatina has joined #openstack-kolla | 19:48 | |
| sdake_ | nah it definately doesnt like soething about my minime-02 node | 19:49 |
| sdake_ | There was a problem importing one of the Python modules | 19:50 |
| sdake_ | required to run yum. The error leading to this problem was: | 19:50 |
| sdake_ | No module named yum | 19:50 |
| sdake_ | groan | 19:50 |
| sean-k-mooney | wow where did you find that error? | 19:51 |
| sdake_ | any python operation on minime-02... | 19:51 |
| sdake_ | tared /usr/lib/ptyhon still doesn't find the yum module | 19:52 |
| * sdake_ hates reinstalling | 19:52 | |
| sean-k-mooney | are you on centos/rhel? | 19:53 |
| sdake_ | yup | 19:53 |
| sdake_ | yum is in /usr/lib/python2.7/site-packages | 19:53 |
| *** zhiwei has joined #openstack-kolla | 19:54 | |
| sean-k-mooney | ok fedora 23 has a nasty packaging change where they made python depend on pip so devstack would remove python when it unistalled pip. | 19:54 |
| sean-k-mooney | i think on centos yum depends on yum-fastmirros-plugin also | 19:55 |
| sdake_ | i don ot run devstack on these machines | 19:55 |
| sean-k-mooney | oh i know my point was i wonder i you unitsalled something that actidentally removed the python module. | 19:55 |
| sdake_ | reinstalling yum does not fix it | 19:57 |
| sdake_ | python-2.7.5-34.el7.x86_64 | 19:57 |
| *** gfidente has quit IRC | 19:57 | |
| sdake_ | python interepeter runs | 19:58 |
| *** zhiwei has quit IRC | 19:58 | |
| *** haplo37 has quit IRC | 20:01 | |
| sean-k-mooney | if you do which python is it /usr/bin/python | 20:01 |
| sdake_ | [sdake@minime-02 bin]$ pip show | 20:01 |
| sdake_ | Traceback (most recent call last): | 20:01 |
| sdake_ | File "/usr/bin/pip", line 5, in <module> | 20:01 |
| sdake_ | from pkg_resources import load_entry_point | 20:01 |
| sdake_ | ImportError: No module named pkg_resources | 20:01 |
| sean-k-mooney | googleing on line seams to point to a broke python instaces | 20:02 |
| *** g3ek has quit IRC | 20:02 | |
| *** huikang_ has joined #openstack-kolla | 20:02 | |
| *** athomas has quit IRC | 20:02 | |
| *** g3ek has joined #openstack-kolla | 20:03 | |
| *** haplo37 has joined #openstack-kolla | 20:04 | |
| *** v1k0d3n has quit IRC | 20:06 | |
| *** huikang_ has quit IRC | 20:07 | |
| sdake_ | fixed it | 20:08 |
| sdake_ | it is lucky the namespace change didn't break upgrades | 20:11 |
| sdake_ | completely random | 20:12 |
| sean-k-mooney | so what was the fix in the end? resinatll python? or someting else | 20:12 |
| sdake_ | tarred /usr/lib and /usr/lib/64/python* | 20:14 |
| sdake_ | rather /usr/lib/python* | 20:14 |
| sdake_ | and untarred on busted machine | 20:14 |
| sdake_ | TASK: [mariadb | Waiting for MariaDB service to be ready] ********************* | 20:14 |
| sdake_ | failed: [minime-02] => {"elapsed": 60, "failed": true} | 20:14 |
| sdake_ | msg: Timeout when waiting for search string MariaDB in 192.168.1.102:3306 | 20:14 |
| * sdake_ growls | 20:14 | |
| *** ppowell_ has quit IRC | 20:16 | |
| sdake_ | connection refused on 3306 yet mariadb running | 20:16 |
| *** bootsha has joined #openstack-kolla | 20:17 | |
| sean-k-mooney | are you doing a fresh install or upgrade by the way | 20:19 |
| sdake_ | fresh install atm | 20:19 |
| sdake_ | but the machines have been installed before | 20:19 |
| sdake_ | http://paste.fedoraproject.org/386044/45207146/ -> http://paste.fedoraproject.org/386044/45207146 | 20:20 |
| sdake_ | mariadb output | 20:20 |
| sdake_ | data looks correct to me | 20:20 |
| sdake_ | yet the fact remains i cannot connect to 103 | 20:20 |
| sdake_ | 3306 | 20:20 |
| sdake_ | flushing ip tables doesn't help | 20:22 |
| mark-casey | sdake_: WSREP: no nodes coming from prim view, prim not possible | 20:27 |
| sdake_ | what does that mean | 20:28 |
| mark-casey | likely all started nodes have that in their log. there is no quorum/all nodes think they are part of a partition | 20:28 |
| sdake_ | i took all the log messages out o corosync, got tired of people asking me about them :) | 20:28 |
| mark-casey | this is the case you'd run lights out for, but if I'm understanding you there shouldn't have been a need to | 20:29 |
| mark-casey | :D | 20:29 |
| sdake_ | well whatever the case is, a cleanup fixed it | 20:29 |
| sdake_ | cleanup in mster pretty good | 20:29 |
| sdake_ | kind of hacky implementation | 20:29 |
| *** ayoung has joined #openstack-kolla | 20:29 | |
| sdake_ | i think we need a way to shut off cleanup | 20:31 |
| openstackgerrit | David Wang proposed openstack/kolla-kubernetes: POC (do no merge): Kolla-Kubernetes multi-node persistence pattern https://review.openstack.org/335215 | 20:32 |
| *** v1k0d3n has joined #openstack-kolla | 20:33 | |
| *** jtriley_ has joined #openstack-kolla | 20:35 | |
| *** v1k0d3n has quit IRC | 20:37 | |
| *** huikang_ has joined #openstack-kolla | 20:38 | |
| *** jtriley has quit IRC | 20:38 | |
| *** jtriley_ has quit IRC | 20:39 | |
| *** haplo37 has quit IRC | 20:44 | |
| *** thumpba has quit IRC | 20:45 | |
| *** aernhart has joined #openstack-kolla | 20:46 | |
| *** g3ek has quit IRC | 20:48 | |
| *** g3ek has joined #openstack-kolla | 20:50 | |
| *** aernhart has quit IRC | 20:52 | |
| *** haplo37 has joined #openstack-kolla | 20:54 | |
| *** fragatina has quit IRC | 20:54 | |
| *** fragatina has joined #openstack-kolla | 20:54 | |
| *** aNupoisc has joined #openstack-kolla | 20:55 | |
| *** ravig has quit IRC | 20:56 | |
| sdake_ | 7 minutes to deploy 3 node ha cluster | 20:56 |
| sdake_ | full control plane | 20:56 |
| sdake_ | bet computeis probably 20 seconds or a less a node | 20:56 |
| inc0 | little over 6 minutes for 10 node + ceph | 20:58 |
| inc0 | you're slow sdake | 20:58 |
| sdake_ | ssh did eem to block often | 20:58 |
| *** huikang_ has quit IRC | 20:58 | |
| sdake_ | anyway no sense optimzing that anymore | 20:59 |
| sdake_ | our big chunk of deploy tie will be bare metal itself | 20:59 |
| *** ravig has joined #openstack-kolla | 20:59 | |
| *** sdake has joined #openstack-kolla | 21:01 | |
| sean-k-mooney | hopefully ironic will be up to the task | 21:02 |
| *** ravig has quit IRC | 21:03 | |
| sean-k-mooney | sdake_: one think i noticed about the cleanup script on master is if you put your invetory file in /etc/kolla it will happily delete it | 21:04 |
| *** sdake_ has quit IRC | 21:04 | |
| sdake | ya i know wierd etc deletion | 21:04 |
| sdake | it also deletes certificates | 21:04 |
| sdake | i think cleanup is a bad bad misnomer | 21:04 |
| sdake | that is getting renamed | 21:05 |
| sdake | it shoudl be kolla-ansible remove-permanently | 21:05 |
| sean-k-mooney | id does not delet the images or the gobal.yml/kolla-build.conf which is also strange | 21:06 |
| *** ravig has joined #openstack-kolla | 21:08 | |
| *** dmk0202 has joined #openstack-kolla | 21:11 | |
| *** ravig has quit IRC | 21:12 | |
| *** bootsha has quit IRC | 21:13 | |
| *** rhallisey has quit IRC | 21:13 | |
| *** harlowja has quit IRC | 21:19 | |
| *** stvnoyes1 has quit IRC | 21:22 | |
| *** v1k0d3n has joined #openstack-kolla | 21:27 | |
| *** inc0 has quit IRC | 21:27 | |
| *** williamcaban has quit IRC | 21:28 | |
| *** haplo37 has quit IRC | 21:31 | |
| *** v1k0d3n has quit IRC | 21:34 | |
| *** vhosakot has quit IRC | 21:35 | |
| *** vhosakot has joined #openstack-kolla | 21:36 | |
| *** g3ek has quit IRC | 21:36 | |
| *** g3ek has joined #openstack-kolla | 21:37 | |
| *** banix has quit IRC | 21:39 | |
| *** sdake has quit IRC | 21:39 | |
| *** stvnoyes has joined #openstack-kolla | 21:39 | |
| *** haplo37 has joined #openstack-kolla | 21:40 | |
| *** dmk0202 has quit IRC | 21:42 | |
| *** daneyon_ has joined #openstack-kolla | 21:53 | |
| *** daneyon_ has quit IRC | 21:58 | |
| *** huikang_ has joined #openstack-kolla | 21:59 | |
| *** huikang_ has quit IRC | 22:04 | |
| *** huikang has quit IRC | 22:06 | |
| *** zhiwei has joined #openstack-kolla | 22:14 | |
| *** aNupoisc has quit IRC | 22:15 | |
| *** harlowja has joined #openstack-kolla | 22:16 | |
| *** papacz has quit IRC | 22:18 | |
| *** zhiwei has quit IRC | 22:18 | |
| *** dwalsh has joined #openstack-kolla | 22:21 | |
| *** harlowja has quit IRC | 22:23 | |
| *** g3ek has quit IRC | 22:24 | |
| *** haplo37 has quit IRC | 22:24 | |
| *** g3ek has joined #openstack-kolla | 22:29 | |
| *** mewald has quit IRC | 22:29 | |
| *** haplo37 has joined #openstack-kolla | 22:29 | |
| *** mbound_ has joined #openstack-kolla | 22:33 | |
| *** aNupoisc has joined #openstack-kolla | 22:35 | |
| *** mark-casey has quit IRC | 22:45 | |
| *** ravig has joined #openstack-kolla | 22:47 | |
| *** salv-orl_ has joined #openstack-kolla | 22:59 | |
| *** salv-orlando has quit IRC | 23:02 | |
| *** g3ek has quit IRC | 23:06 | |
| *** g3ek has joined #openstack-kolla | 23:11 | |
| *** harlowja has joined #openstack-kolla | 23:12 | |
| *** dcwangmit01_ has quit IRC | 23:17 | |
| *** dcwangmit01 has joined #openstack-kolla | 23:17 | |
| *** dwalsh has quit IRC | 23:38 | |
| *** mbound_ has quit IRC | 23:44 | |
| *** adn has joined #openstack-kolla | 23:44 | |
| *** aNupoisc has quit IRC | 23:45 | |
| *** aNupoisc has joined #openstack-kolla | 23:46 | |
| *** adn has quit IRC | 23:49 | |
| *** vhosakot has quit IRC | 23:55 | |
| harlowja | qq, why in rhel and such are all the repos turned on even if u have no desire to build said components? | 23:56 |
| *** aNupoisc has quit IRC | 23:57 | |
| harlowja | ie kibana, mariadb, elasticearch | 23:57 |
| harlowja | why aren't those just turned on if/when u are building those components | 23:58 |
| *** kbyrne has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!