| *** fatdragon has quit IRC | 00:52 | |
| *** gougounel has quit IRC | 01:14 | |
| *** mpjetta has quit IRC | 01:17 | |
| *** mpjetta has joined #openstack-ansible | 01:19 | |
| *** mattt has quit IRC | 01:30 | |
| *** mattt has joined #openstack-ansible | 01:30 | |
| *** smatzek has joined #openstack-ansible | 01:53 | |
| *** threestrands has joined #openstack-ansible | 01:55 | |
| *** smatzek has quit IRC | 01:57 | |
| *** mattt has quit IRC | 02:20 | |
| *** masber has joined #openstack-ansible | 02:53 | |
| masber | good afternoon, I am wondering if this is a typo https://docs.openstack.org/project-deploy-guide/openstack-ansible/pike/overview-requirements.html says VLAN offloading, which increases performance by adding and removing VLAN tags in hardware, rather than in the server’s main CPU... | 02:54 |
|---|---|---|
| masber | I have searching on internet and I can't find the way of doing VLAN offloading in my nics so I was wondering whether this really means VXLAN offloading? | 02:54 |
| ANKITA | hi | 03:01 |
| ANKITA | few of myconmtainers have stopped | 03:02 |
| ANKITA | on starting them they give an err | 03:02 |
| ANKITA | lxc-start: start.c: __lxc_start: 1358 Failed to spawn container | 03:02 |
| ANKITA | FAILED - RETRYING: TASK: pip_install : Install pip packages (fall back mode) (3 retries left). FAILED - RETRYING: TASK: pip_install : Install pip packages (fall back mode) (2 retries left). FAILED - RETRYING: TASK: pip_install : Install pip packages (fall back mode) (1 retries left). fatal: [infra1_galera_container-24d80c82]: FAILED! => {"attempts": 5, "changed": false, "cmd": "/usr/local/bin/pip2 install -U --isolated --c | 03:43 |
| *** pmannidi has joined #openstack-ansible | 03:45 | |
| ANKITA | ok: [infra3_repo_container-82be5c9d -> 10.40.100.23] fatal: [infra2_neutron_agents_container-40a749a7 -> 10.40.100.22]: FAILED! => {"changed": false, "error": "Failed to start container [ infra2_neutron_agents_container-40a749a7 ]", "failed": true, "lxc_container": {"init_pid": -1, "interfaces": [], "ips": [], "name": "infra2_neutron_agents_container-40a749a7", "state": "stopped"}, "msg": "The container [ infra2_neutron_ag | 04:03 |
| ANKITA | fatal: [infra3_glance_container-c88158c5 -> 10.40.100.23]: FAILED! => {"changed": false, "error": "Failed to start container [ infra3_glance_container-c88158c5 ]", "failed": true, "lxc_container": {"init_pid": -1, "interfaces": [], "ips": [], "name": "infra3_glance_container-c88158c5", "state": "stopped"}, "msg": "The container [ infra3_glance_container-c88158c5 ] failed to start. Check to lxc is available and that the con | 04:03 |
| *** lbragstad has quit IRC | 04:11 | |
| *** udesale has joined #openstack-ansible | 04:37 | |
| *** taseer1 has quit IRC | 04:39 | |
| *** eumel8 has joined #openstack-ansible | 05:50 | |
| ANKITA | lxc-attach --clear-env --name infra2_glance_container-dd55fdab -- su - root -c lxc-attach: attach.c: lxc_attach: 742 Failed to get init pid. | 06:04 |
| ANKITA | can i delete the container and rerun the playbook | 06:05 |
| *** armaan has quit IRC | 06:21 | |
| cloudnull | masber: no typo. with ethtool you should be able to enable or disable rxvlan and txvlan | 06:38 |
| cloudnull | a lot server of nic come with this already enabled. | 06:39 |
| ANKITA | is there any other cmd to start a stopped container lxc-start -n -L infra2_cinder_api_container-90624f0b --foreground | 06:42 |
| ANKITA | ?? | 06:42 |
| *** ThomasS_ has quit IRC | 06:58 | |
| *** threestrands has quit IRC | 07:25 | |
| *** pcaruana has joined #openstack-ansible | 07:32 | |
| evrardjp | ANKITA: lxc-start -n infra2_cinder_api_container-90624f0b is good enough | 07:36 |
| evrardjp | and in background, add -d | 07:36 |
| evrardjp | you have a mix of stuff messed up ANKITA | 07:37 |
| evrardjp | it looks like you can't install pip packages (which I assume a problem of connectivity to your repo server node) | 07:37 |
| evrardjp | but then you have problem with neutron containers (that's something I guess problematic in your env) | 07:38 |
| evrardjp | ANKITA: yesterday I suggested you to find the root cause of your issues, what was it? | 07:38 |
| evrardjp | you seem to be a little further, but the issue seem to be there still | 07:38 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: [DNM] debug why on metal fails on gates and not on my machine. https://review.openstack.org/524954 | 07:50 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: [DNM] debug why on metal fails on gates and not on my machine. https://review.openstack.org/524954 | 07:50 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: [WIP] Build everything on metal https://review.openstack.org/504224 | 07:52 |
| ANKITA | fatal: [infra2_neutron_agents_container-40a749a7 -> 10.40.100.22]: FAILED! => {"changed": false, "error": "Failed to start container [ infra2_neutron_agents_container-40a749a7 ]", "failed": true, "lxc_container": {"init_pid": -1, "interfaces": [], "ips": [], "name": "infra2_neutron_agents_container-40a749a7", "state": "stopped"}, "msg": "The container [ infra2_neutron_agents_container-40a749a7 ] failed to start. Check to l | 07:53 |
| *** markvoelker has quit IRC | 07:55 | |
| ANKITA | how can i check whether env is not fine | 07:55 |
| ANKITA | thr repo conncetivity does breaks a couple of time | 07:56 |
| *** shardy has joined #openstack-ansible | 08:16 | |
| evrardjp | ANKITA: it should never break | 08:23 |
| evrardjp | it should work out of the box, and be redundant | 08:23 |
| evrardjp | maybe worth cleaning your nodes, and start from scratch, and tell us what you do in the full process, maybe there is something you're missing from our docs, or something is missing in our docs | 08:24 |
| evrardjp | this way we can help you, step by step | 08:25 |
| evrardjp | because you're stuck for a while now, and that's not normal | 08:25 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: [DNM] debug why on metal fails on gates and not on my machine. https://review.openstack.org/524954 | 08:29 |
| ANKITA | hi was able to fix 2 containers | 08:29 |
| ANKITA | out of 6 | 08:30 |
| ANKITA | evrardjp@shall i format all the servers? | 08:30 |
| evrardjp | ANKITA: yes I would. This is cloud after all. It breaks, it goes often faster to reroll the node :p | 08:31 |
| evrardjp | ANKITA: I would also cleanup the deploy node. Only keep the openstack_user_config and conf.d files | 08:33 |
| evrardjp | maybe you can show us those files | 08:33 |
| Guest89461 | hi evrardjp, if anyone need cloudkitty pike role, I wrote it. anyone can use it. it is working successfully :) | 08:44 |
| Guest89461 | https://github.com/b3lab/openstack-ansible-os_cloudkitty/tree/stable/pike | 08:44 |
| *** Guest89461 is now known as gokhan | 08:44 | |
| gokhan | Guest89461 is known gokhan | 08:44 |
| *** gokhan is now known as Guest65919 | 08:45 | |
| *** Guest65919 is now known as gokhan_ | 08:45 | |
| *** sxc731 has joined #openstack-ansible | 08:47 | |
| *** hamza21 has joined #openstack-ansible | 08:51 | |
| *** sxc731 has quit IRC | 08:57 | |
| *** ThomasSch has joined #openstack-ansible | 09:13 | |
| ThomasSch | hi | 09:13 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: [DNM] debug why on metal fails on gates and not on my machine. https://review.openstack.org/524954 | 09:14 |
| *** sxc731 has joined #openstack-ansible | 09:22 | |
| *** cshen has quit IRC | 09:30 | |
| evrardjp | hi ThomasSch | 09:35 |
| evrardjp | hello gokhan_ | 09:36 |
| *** hamza21 has quit IRC | 09:46 | |
| *** cshen has joined #openstack-ansible | 09:53 | |
| *** markvoelker has joined #openstack-ansible | 09:56 | |
| *** hamza21 has joined #openstack-ansible | 09:58 | |
| *** hamza21 has quit IRC | 10:15 | |
| *** hamza21 has joined #openstack-ansible | 10:18 | |
| *** pbandark has joined #openstack-ansible | 10:26 | |
| *** markvoelker has quit IRC | 10:30 | |
| *** hamza21 has quit IRC | 10:32 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Cleanup environment before bootstrap-ansible in upgrades https://review.openstack.org/531123 | 10:54 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: [DNM] debug why on metal fails on gates and not on my machine. https://review.openstack.org/524954 | 11:00 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Define GATE_EXIT_RUN_DSTAT before use https://review.openstack.org/531131 | 11:13 |
| -openstackstatus- NOTICE: zuul seems to have gotten stuck and will probably need a restart, please be patient | 11:26 | |
| *** markvoelker has joined #openstack-ansible | 11:27 | |
| *** openstackstatus has quit IRC | 11:28 | |
| *** openstack has quit IRC | 11:28 | |
| *** openstack has joined #openstack-ansible | 13:10 | |
| *** ChanServ sets mode: +o openstack | 13:10 | |
| *** Smeared_Beard has joined #openstack-ansible | 13:12 | |
| *** SmearedBeard has quit IRC | 13:14 | |
| evrardjp | foutatoro: generally swift is with local things, we do not do ceph + a swift gateway IIRC. | 13:17 |
| evrardjp | maybe worth checking what you can do with ceph-ansible, because that's what we are using to deploy ceph, behind the scenes | 13:18 |
| evrardjp | our swift role is doing everything locally if I am not mistaken. andymccr could help you more there | 13:18 |
| foutatoro | evrardjp: since ceph has an object storage feature I wondered if this could be used by swift to avoid having to types of storage nodes (swift servers + ceph servers) | 13:22 |
| evrardjp | foutatoro: yeah. Using rados directly. Like I said, it's maybe possible using ceph-ansible | 13:25 |
| evrardjp | I don't know ceph-ansible enough | 13:25 |
| evrardjp | I guess it would make sense to configure a "middleware" in ceph that speaks swift and stores blobs into rados | 13:26 |
| ArchiFleKs | Hi, in recent master changes (or maybe before but i hadn't notice) did all nova service have been back to only one container (aio1_nova_api_container for example) ? | 13:27 |
| *** foutatoro has quit IRC | 13:31 | |
| *** sxc731 has joined #openstack-ansible | 13:37 | |
| openstackgerrit | Periyasamy Palanisamy proposed openstack/openstack-ansible-os_neutron master: add ml2 config for networking bgpvpn https://review.openstack.org/522598 | 13:38 |
| *** savvas` has quit IRC | 13:39 | |
| openstackgerrit | Periyasamy Palanisamy proposed openstack/openstack-ansible-os_neutron master: add ml2 config for networking bgpvpn https://review.openstack.org/522598 | 13:41 |
| *** sxc731 has quit IRC | 13:47 | |
| *** Miouge has quit IRC | 13:53 | |
| *** Miouge has joined #openstack-ansible | 13:56 | |
| *** aludwar has joined #openstack-ansible | 14:02 | |
| *** pbandark1 has joined #openstack-ansible | 14:04 | |
| *** pbandark has quit IRC | 14:04 | |
| *** pbandark1 is now known as pbandark | 14:04 | |
| *** savvas` has joined #openstack-ansible | 14:20 | |
| *** sxc731 has joined #openstack-ansible | 14:23 | |
| *** wspthr has quit IRC | 14:24 | |
| *** mrtenio has joined #openstack-ansible | 14:25 | |
| *** smatzek has quit IRC | 14:25 | |
| *** wspthr has joined #openstack-ansible | 14:27 | |
| *** sxc731 has quit IRC | 14:28 | |
| *** esberglu has joined #openstack-ansible | 14:39 | |
| *** sxc731 has joined #openstack-ansible | 14:40 | |
| *** lbragstad has joined #openstack-ansible | 14:42 | |
| *** yolanda has quit IRC | 14:45 | |
| *** hamza21 has quit IRC | 14:45 | |
| -openstackstatus- NOTICE: zuul has been restarted, all queues have been reset. please recheck your patches when appropriate | 14:46 | |
| evrardjp | ArchiFleKs: not all, but yes we are reducing the amount of containers | 14:53 |
| evrardjp | ArchiFleKs: have you noticed an issue? | 14:57 |
| evrardjp | upgrades aren't automatically migrated | 14:57 |
| *** esberglu has quit IRC | 14:57 | |
| *** vnogin has joined #openstack-ansible | 14:58 | |
| *** smatzek has joined #openstack-ansible | 15:01 | |
| *** smatzek has quit IRC | 15:02 | |
| *** smatzek has joined #openstack-ansible | 15:02 | |
| cloudnull | mornings | 15:18 |
| *** vnogin has quit IRC | 15:21 | |
| *** smatzek has quit IRC | 15:24 | |
| *** smatzek has joined #openstack-ansible | 15:26 | |
| *** smatzek_ has joined #openstack-ansible | 15:27 | |
| *** smatzek_ has quit IRC | 15:28 | |
| *** smatzek_ has joined #openstack-ansible | 15:28 | |
| *** smatzek has quit IRC | 15:31 | |
| *** smatzek_ has quit IRC | 15:33 | |
| *** smatzek has joined #openstack-ansible | 15:36 | |
| *** rromans has quit IRC | 15:38 | |
| *** smatzek has quit IRC | 15:38 | |
| *** rromans has joined #openstack-ansible | 15:42 | |
| MikeW | Hey this isn't directly related to OSA, but you guys are all very very experienced... and I'm a dumbie a lot of the time... anyway can anyone point me to a guide that shows me how to push logs to graylog without having duplicates on host? Everything I've found just rotates locally into zipped files and pushes them to graylog | 15:44 |
| jafeha | MikeW, we have set the rsyslog_client_user_defined_targets within the osa deployment and ship logs to graylog via rsyslog. i cannot remember how graylogs side is configured, but we have no duplicates so far. | 15:47 |
| *** smatzek has joined #openstack-ansible | 15:51 | |
| ArchiFleKs | evrardjp: nope, no issue, but I just did some cleanup on my own | 15:52 |
| spotz | evrardjp: I'm thinking abandon this https://review.openstack.org/#/c/530146/2 and move it to the other file in jamesdenton's comment. thoughts? | 15:54 |
| *** smatzek has quit IRC | 15:55 | |
| *** sxc731 has quit IRC | 15:57 | |
| *** smatzek has joined #openstack-ansible | 15:57 | |
| *** smatzek_ has joined #openstack-ansible | 16:00 | |
| *** udesale has joined #openstack-ansible | 16:01 | |
| *** jafeha has quit IRC | 16:01 | |
| *** smatzek has quit IRC | 16:01 | |
| *** ivve has quit IRC | 16:04 | |
| *** smatzek_ has quit IRC | 16:05 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: [DNM] debug why on metal fails on gates and not on my machine. https://review.openstack.org/524954 | 16:05 |
| *** smatzek has joined #openstack-ansible | 16:06 | |
| *** smatzek has quit IRC | 16:07 | |
| *** smatzek has joined #openstack-ansible | 16:08 | |
| evrardjp | spotz: I will have a look | 16:09 |
| evrardjp | spotz: answered! | 16:10 |
| MikeW | jafeha Can you show me some of your configs? I think mine are off, because I ship off logs but also have duplicates. | 16:12 |
| evrardjp | MikeW: jafeha seems disconnected | 16:13 |
| evrardjp | MikeW: but I think it would be good if you both document this | 16:13 |
| *** udesale has quit IRC | 16:14 | |
| evrardjp | if you document this together I mean | 16:14 |
| spotz | evrardjp: Ok abandoned this oone I'll get the other one going in a sec | 16:14 |
| evrardjp | thanks spotz | 16:14 |
| *** smatzek has quit IRC | 16:15 | |
| *** smatzek_ has joined #openstack-ansible | 16:15 | |
| *** smatzek_ has quit IRC | 16:16 | |
| *** cshen has quit IRC | 16:17 | |
| *** RandomTech has joined #openstack-ansible | 16:17 | |
| RandomTech | Hey everyone, After running the playbooks to setup openstack we are able to access openstack but we are getting the following error with ceph: http://paste.openstack.org/show/638101/ | 16:19 |
| RandomTech | Anyone have any idea what has happened? | 16:19 |
| *** vnogin has joined #openstack-ansible | 16:21 | |
| *** smatzek has joined #openstack-ansible | 16:22 | |
| *** smatzek has quit IRC | 16:27 | |
| *** smatzek has joined #openstack-ansible | 16:30 | |
| openstackgerrit | Amy Marrich (spotz) proposed openstack/openstack-ansible master: [DOCS] Add limit parameter to adding a new compute section https://review.openstack.org/531193 | 16:31 |
| *** sxc731 has joined #openstack-ansible | 16:33 | |
| *** smatzek_ has joined #openstack-ansible | 16:34 | |
| *** smatzek has quit IRC | 16:35 | |
| *** ChanServ sets mode: -r | 16:46 | |
| *** esberglu has joined #openstack-ansible | 16:47 | |
| *** pcaruana has quit IRC | 16:52 | |
| *** smatzek_ has quit IRC | 16:53 | |
| *** smatzek_ has joined #openstack-ansible | 16:56 | |
| *** jafeha has joined #openstack-ansible | 16:57 | |
| *** sxc731 has left #openstack-ansible | 16:58 | |
| jafeha | re | 17:00 |
| *** smatzek_ has quit IRC | 17:00 | |
| jafeha | MikeW, evrardjp all we used is well documented within the rsyslog_client repo. we could definitely push this to an own documentation page, thats a no brainer, but i don't know where to put this exactly. https://github.com/openstack/openstack-ansible-rsyslog_client/blob/master/defaults/main.yml | 17:03 |
| *** esberglu has quit IRC | 17:03 | |
| *** esberglu has joined #openstack-ansible | 17:03 | |
| *** hamza21 has joined #openstack-ansible | 17:09 | |
| *** vnogin has quit IRC | 17:15 | |
| *** smatzek has joined #openstack-ansible | 17:18 | |
| *** epalper has quit IRC | 17:21 | |
| *** smatzek has quit IRC | 17:21 | |
| *** smatzek has joined #openstack-ansible | 17:21 | |
| jafeha | MikeW, evrardjp i just rechecked the rsyslog client docs. the config block is documented there. i don't see that we should document how to configure graylog or any other syslog server / siem itself. https://docs.openstack.org/openstack-ansible-rsyslog_client/pike/ | 17:24 |
| evrardjp | great then | 17:24 |
| evrardjp | jafeha: thanks for having a look! | 17:24 |
| jafeha | no problem | 17:24 |
| jafeha | evrardjp, mbe we can have another chat about ovs scenario tomorrow. i have no idea how much work it actually is, i'd love to help | 17:26 |
| evrardjp | that would be great | 17:27 |
| evrardjp | If you know how to configure OVS that should be fast. | 17:27 |
| evrardjp | I will help you through it, but I have to go right now, someone is waiting for me | 17:27 |
| jafeha | hope so. but we'll see. | 17:27 |
| jafeha | same here. lets talk tomorrow | 17:27 |
| evrardjp | good | 17:28 |
| *** fatdragon has joined #openstack-ansible | 17:28 | |
| evrardjp | cya all! | 17:28 |
| *** woodard has joined #openstack-ansible | 17:31 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: [DNM] debug why on metal fails on gates and not on my machine. https://review.openstack.org/524954 | 17:32 |
| evrardjp | ok now I am really gone! | 17:33 |
| *** fatdragon has quit IRC | 17:33 | |
| *** vnogin has joined #openstack-ansible | 17:35 | |
| *** fatdragon has joined #openstack-ansible | 17:35 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Add nspawn container driver https://review.openstack.org/477017 | 17:42 |
| MikeW | jafeha Thanks this is perfect! | 17:54 |
| *** vnogin has quit IRC | 18:12 | |
| RandomTech | Hey everyone, After running the playbooks to setup openstack we are able to access openstack but we are getting the following error with ceph: http://paste.openstack.org/show/638101/ | 18:16 |
| RandomTech | Anyone have any idea what has happened? | 18:17 |
| *** gkadam has joined #openstack-ansible | 18:23 | |
| RandomTech | cloudnull: you have any ideas? | 18:25 |
| *** shardy has quit IRC | 18:25 | |
| * cloudnull reading back | 18:37 | |
| cloudnull | RandomTech: no clue off the top of my head | 18:38 |
| * cloudnull needs to get better with ceph | 18:38 | |
| cloudnull | andymccr: ^ | 18:38 |
| *** vnogin has joined #openstack-ansible | 18:40 | |
| *** poul_ has joined #openstack-ansible | 18:49 | |
| *** ivve has joined #openstack-ansible | 18:52 | |
| *** vnogin has quit IRC | 18:52 | |
| *** poopcat has joined #openstack-ansible | 18:59 | |
| ivve | yo guys, is there any swift to ceph playbooks ? | 19:03 |
| ivve | im pretty sure i have seen some, can't find em now though | 19:03 |
| ivve | or rather keystone to rgw? | 19:04 |
| *** poopcat1 has joined #openstack-ansible | 19:04 | |
| *** poul_ has quit IRC | 19:04 | |
| *** poopcat has quit IRC | 19:05 | |
| RandomTech | hey andymccr you avaible for a second? | 19:13 |
| *** hamza21 has quit IRC | 19:13 | |
| *** hamza21 has joined #openstack-ansible | 19:14 | |
| cloudnull | ivve: nothing that we have to migrate from swift to rgw (ceph). | 19:15 |
| cloudnull | the ceph-ansible playbooks should be able to stand up rgw | 19:16 |
| cloudnull | which you can integrate into openstack | 19:16 |
| ivve | aye manually | 19:16 |
| cloudnull | but im not aware of anything we have that would do it automagically | 19:16 |
| ivve | i was thinking like with cinder | 19:16 |
| cloudnull | all you should have to do with that is define the cinder backend in the openstack_user_config.yml which points at ceph. | 19:17 |
| ivve | integrate a working ceph cluster with keystone/swift endpoints | 19:17 |
| cloudnull | like so https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.example#L673-L690 | 19:17 |
| ivve | aye that i know of | 19:17 |
| cloudnull | ok. | 19:17 |
| cloudnull | beyond that I'm not sure. | 19:18 |
| ivve | i was thinking about something similar to that, but for keystone/swift endpoints for ceph rgws | 19:18 |
| cloudnull | ^ mattt logan- andymccr ? | 19:18 |
| RandomTech | cloudnull: when we had that block in it was throwing errors, any idea why | 19:18 |
| ivve | i was under the impression that existed in osa.. that i had seen it before | 19:18 |
| cloudnull | ivve: I can look around , but i don't recall any rgw integration. | 19:19 |
| cloudnull | RandomTech: which block ? | 19:19 |
| ivve | ait, maybe its just a deja vu :) | 19:19 |
| RandomTech | the one you have highlighted | 19:20 |
| cloudnull | ivve: cloud be :D | 19:20 |
| cloudnull | **could | 19:20 |
| ivve | :D | 19:20 |
| * cloudnull still still with holiday brain | 19:20 | |
| ivve | hah i thought it was with an intended pun :P | 19:20 |
| cloudnull | I'm not that clever | 19:21 |
| cloudnull | RandomTech: that should be all that's needed when integrating ceph with block storage | 19:21 |
| cloudnull | but truth be told I've not worked much with ceph | 19:21 |
| RandomTech | and ive noticed that block isnt in the ceph example: https://docs.openstack.org/project-deploy-guide/openstack-ansible/pike/app-config-prod-ceph.html | 19:22 |
| ivve | i've worked with ceph, perhaps i can help | 19:22 |
| RandomTech | that would be great, weve been pounding or head against the wall. would you like to look at our error and configs? | 19:23 |
| ivve | is it deployment with cinder+ceph? | 19:23 |
| ivve | shoot | 19:23 |
| RandomTech | we tried to deploy it using opesntack ansible | 19:23 |
| ivve | should be easy | 19:23 |
| RandomTech | it hasnt been :/ | 19:24 |
| RandomTech | mind taking a look at our stuff? | 19:24 |
| ivve | if you could paste the "ceph auth list" + openstack_user_config.yml as well as user_variables.yml | 19:24 |
| RandomTech | ceph auth list? im guessing thats a command? | 19:25 |
| ivve | oh yea and a "ceph osd pool ls detail" | 19:26 |
| ivve | aye, a ceph command | 19:26 |
| RandomTech | one moment while i compile those | 19:27 |
| ivve | cool | 19:27 |
| RandomTech | im guessing ceph wil throw an error but ill put that in the paste if it does | 19:27 |
| ivve | was ceph deployed with osa or a separate deployment with ansible or ceph-deploy? | 19:28 |
| *** hamza21 has quit IRC | 19:28 | |
| *** hamza21 has joined #openstack-ansible | 19:28 | |
| RandomTech | it was deployed with osa | 19:28 |
| ivve | alright | 19:29 |
| ivve | i haven't tried that yet. i like my ceph deployment done "manually" as there is much to be tweaked imo :) | 19:30 |
| ivve | but ill have a go and see what i can do | 19:30 |
| RandomTech | okay heres ceph commands: ansible@R01U05BC:~$ ceph auth list 2018-01-04 14:33:05.297333 7fa8bb8fb700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: (2) No such file or directory 2018-01-04 14:33:05.298981 7fa8bb8fb700 -1 monclient: authenticate NOTE: no keyring found; disabled cephx authentication 2018-01-04 14:33:05.299001 7fa8bb | 19:34 |
| RandomTech | oops wrong thing | 19:34 |
| RandomTech | heres ceph commands: http://paste.openstack.org/show/638288/ | 19:34 |
| ivve | ok so you have a ceph error | 19:34 |
| RandomTech | heres openstack_user_config.yml: http://paste.openstack.org/show/638289/ | 19:34 |
| RandomTech | heres user_variables.yml: http://paste.openstack.org/show/638290/ | 19:34 |
| RandomTech | yes | 19:35 |
| RandomTech | we think it may have been a problem with the setup though | 19:35 |
| RandomTech | because we have been getting there error since it was set up | 19:35 |
| ivve | basically ceph keyrings are not distributed | 19:36 |
| RandomTech | im not sure why | 19:36 |
| RandomTech | but i guess so | 19:36 |
| ivve | ill see if i can find the task in the playbook, im guessing its not using ceph-deploy | 19:37 |
| RandomTech | i do not know much about ceph. im a college student that was asked to setup openstack | 19:37 |
| RandomTech | thanks for trying to help us | 19:37 |
| ivve | :) | 19:38 |
| hamza21 | RandomTech, you need to bind the ceph services in group_bind section of your openstack_user_config.yml, something like this: | 19:38 |
| ivve | check if /etc/ceph contains anything | 19:39 |
| spotz | RandomTech: Yoouo might want to try asking in the morning andymccr is on UK time | 19:39 |
| *** smatzek has quit IRC | 19:39 | |
| jrosser | heres a ceph/swift integration worked example https://keithtenzer.com/2017/03/30/openstack-swift-integration-with-ceph/ | 19:39 |
| jrosser | seems like quite low hanging fruit for adding to OSA | 19:39 |
| hamza21 | http://paste.openstack.org/show/638295/ | 19:40 |
| ivve | jrosser: aye thats why i thought it was already in | 19:40 |
| *** poul has joined #openstack-ansible | 19:40 | |
| jrosser | yes it surprises me it isnt given how many OSA+ceph users there seem to be | 19:40 |
| ivve | RandomTech: to start with ceph_osd_hosts is only 2x hosts | 19:40 |
| RandomTech | ivve: this is what is in there: http://paste.openstack.org/show/638296/ | 19:40 |
| ivve | should be 3 assuming its using default size = 3, min_size = 2 | 19:41 |
| RandomTech | we only have 2 because we currently have 5 blades 3 of which need to be infrastructure/controll. | 19:41 |
| *** DimGR has joined #openstack-ansible | 19:42 | |
| RandomTech | as far as ive understood anyway | 19:42 |
| ivve | i suggest you put 3 of 5 as infra and 5 of 5 as compute if you can.. | 19:42 |
| ivve | colocate | 19:42 |
| RandomTech | make them all act as compute? | 19:43 |
| ivve | aye | 19:43 |
| RandomTech | it wont cause any problems with three of them doing all the infrastructure? | 19:43 |
| ivve | but not sure how to config that with osa (for ceph). not sure how it identifies block devices for ceph | 19:44 |
| ivve | well i have 3 nodes here right now that acts everything | 19:44 |
| ivve | both ceph, compute and controller :) | 19:44 |
| ivve | they all contain mon processes and osds | 19:44 |
| ivve | i guess im not the best man for the job to assist with osa-ceph deployment.. im more osa+existing ceph :) | 19:45 |
| RandomTech | would it help if i explain what we have to work with some. | 19:45 |
| ivve | sure | 19:45 |
| DimGR | hi | 19:46 |
| RandomTech | we have 5 nodes for the cluster and one for deployment/monitoring, these 6 nodes have a 10Gb connection on an isolated network, and a 1Gb connection on a public network of a university | 19:47 |
| ivve | basically it seems like you are missing ceph.client.admin.keyring | 19:47 |
| ivve | it should be somewhere as it is created early on | 19:47 |
| ivve | if that is the only issue then its quite easy | 19:47 |
| RandomTech | could that be because ceph was not mentioned in group binds? | 19:48 |
| ivve | but i suspect its not | 19:48 |
| RandomTech | ya only the client keyring is in there | 19:48 |
| ivve | on the deployment node "find / -iname "ceph.client.admin.keyring" :P | 19:49 |
| ivve | im guessing its there, possibly in /etc/openstack_ansible/* ? | 19:49 |
| *** gkadam has quit IRC | 19:49 | |
| ivve | i see, using ubuntu or centos? | 19:51 |
| RandomTech | were using ubuntu 16 and hers the output of that command: http://paste.openstack.org/show/638303/ | 19:51 |
| ivve | good guess there :P | 19:52 |
| RandomTech | hamza21:could that be what caused the keyring to not get distributed | 19:52 |
| ivve | for i in blade1 blade2 blade3 blade4 blade5; do scp /etc/openstack_deploy/ceph-fetch/db5a73f2-56f1-4e55-8265-6489bb133b95/etc/ceph/ceph.client.admin.keyring root@${i}:/etc/ceph/ ; done | 19:53 |
| ivve | replace blade with actual names | 19:53 |
| ivve | or ips | 19:54 |
| ivve | and input whatever passwordrequests come up for root user assuming thats allowed | 19:54 |
| ivve | lots of assumptions here :P | 19:54 |
| RandomTech | there is no root password and we need to fix the setup so that we can reinstall stuff | 19:55 |
| RandomTech | and have a nice clean install | 19:55 |
| ivve | ait | 19:55 |
| ivve | anyways, getting those keys to /etc/ceph on each node might help a bit but most probably not all the way | 19:58 |
| hamza21 | RandomTech, not sure if group_binds could impact keyring, but clearly you need to add the ceph service to that group, | 19:58 |
| ivve | on the deployment node you could always check if ceph-deploy package is installed | 19:58 |
| hamza21 | i deployed ceph-osa 8 months ago let me review my conf.... | 19:59 |
| RandomTech | i just added ceph-osd to the group to match the example here: https://docs.openstack.org/project-deploy-guide/openstack-ansible/pike/app-config-prod-ceph.html | 20:00 |
| RandomTech | thank you hamza21 | 20:02 |
| ivve | RandomTech: remember to add 3 hosts for osd | 20:02 |
| ivve | 2 wont suffice | 20:03 |
| RandomTech | will adding the infrastructure host as osd host as well cause issues? | 20:03 |
| ivve | hard to tell as i have never tried, but shouldn't be an issue | 20:04 |
| ivve | they have a lot of host in the example | 20:04 |
| RandomTech | ya :/ | 20:04 |
| RandomTech | i wish there was a list of requirements with it | 20:04 |
| hamza21 | RandomTech: we integrated ceph and rgws with osa, we used https://github.com/ceph/ceph-ansible , here is our working openstack_user_config.yml : | 20:05 |
| *** chhavi__ has joined #openstack-ansible | 20:05 | |
| ivve | if you could supply 5 clean ubuntu hosts and a deployment node which has access to those i could probably deploy it quite quickly. can't do it until saturday though. CET timezone here | 20:06 |
| ivve | however i would deploy ceph with ceph-deploy and then osa for openstack | 20:06 |
| hamza21 | RandomeTech : http://paste.openstack.org/show/638325/ | 20:07 |
| hamza21 | we also installed rgws and keyston in the same container, like this : | 20:08 |
| hamza21 | http://paste.openstack.org/show/638327/ | 20:09 |
| hamza21 | here are the ceph variables that we used : | 20:10 |
| hamza21 | http://paste.openstack.org/show/638331/ | 20:12 |
| hamza21 | Hope this helps you :) , it is been working fine for us | 20:12 |
| RandomTech | rgw and keystone? | 20:14 |
| RandomTech | and i dont believe we have that option ivve | 20:14 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: [DNM] debug why on metal fails on gates and not on my machine. https://review.openstack.org/524954 | 20:15 |
| RandomTech | whats the copy_admin_key line do? | 20:16 |
| hamza21 | rgw the cepg getway you need it for the object storage, we defined it as a new group in /opt/openstack-ansible/playbooks/inventory/env.d/keystone.yml | 20:16 |
| RandomTech | im not seeing that anywhere in the example? | 20:18 |
| *** poul has quit IRC | 20:19 | |
| RandomTech | it looks like you overwrote alot more than the example if im looking at it right | 20:19 |
| hamza21 | yes, you can install rgw separately, but for us we thought it could be good to install rgw along side keystone as they need each other | 20:20 |
| hamza21 | copy_admin_key: is variable belongs to ceph-ansible to copy the admin key, you can refer to ceph-ansible roles for more details | 20:22 |
| RandomTech | i think i may pick this back up later to try again | 20:24 |
| *** poul has joined #openstack-ansible | 20:24 | |
| RandomTech | 1 quick question though. which of the networks is suppposed to be the external one for floating ip's to use? | 20:25 |
| *** chyka has joined #openstack-ansible | 20:25 | |
| hamza21 | in our setup we used OVS, we configured br-vlan to be the external and tenats netwotks, let me share my user_variable.yml : | 20:27 |
| *** poul has quit IRC | 20:29 | |
| hamza21 | user_variables.yml, br-external is just used by haproxy: | 20:30 |
| hamza21 | http://paste.openstack.org/show/638348/ | 20:31 |
| hamza21 | if you are using Linux bridge it will be different of course.... | 20:31 |
| RandomTech | i am | 20:32 |
| RandomTech | ill figure it out eventually | 20:32 |
| RandomTech | i have to get going | 20:32 |
| RandomTech | thanks everyone for trying to help | 20:32 |
| hamza21 | no problem, good luck | 20:33 |
| *** RandomTech has quit IRC | 20:37 | |
| *** poul has joined #openstack-ansible | 20:41 | |
| *** poul has quit IRC | 20:45 | |
| *** smatzek has joined #openstack-ansible | 20:50 | |
| *** vnogin has joined #openstack-ansible | 20:53 | |
| *** poul has joined #openstack-ansible | 20:58 | |
| *** vnogin has quit IRC | 20:58 | |
| *** poul has quit IRC | 21:02 | |
| *** smatzek has quit IRC | 21:05 | |
| *** smatzek has joined #openstack-ansible | 21:06 | |
| *** smatzek has quit IRC | 21:10 | |
| *** poul has joined #openstack-ansible | 21:14 | |
| *** eumel8 has quit IRC | 21:19 | |
| *** poul has quit IRC | 21:19 | |
| *** poul has joined #openstack-ansible | 21:24 | |
| *** poul has quit IRC | 21:28 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Add scafolding for multiple container techs https://review.openstack.org/527749 | 21:28 |
| *** threestrands has joined #openstack-ansible | 21:35 | |
| *** threestrands has quit IRC | 21:35 | |
| *** threestrands has joined #openstack-ansible | 21:35 | |
| *** chyka has quit IRC | 21:43 | |
| *** chyka has joined #openstack-ansible | 21:44 | |
| *** esberglu has quit IRC | 21:54 | |
| *** smatzek has joined #openstack-ansible | 21:57 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: [DNM] debug why on metal fails on gates and not on my machine. https://review.openstack.org/524954 | 21:58 |
| *** hamza21 has quit IRC | 21:58 | |
| *** jose-phillips has quit IRC | 22:05 | |
| *** esberglu has joined #openstack-ansible | 22:06 | |
| *** jose-phillips has joined #openstack-ansible | 22:08 | |
| *** smatzek has quit IRC | 22:15 | |
| *** pedja has quit IRC | 22:20 | |
| *** smatzek has joined #openstack-ansible | 22:21 | |
| *** jose-phillips has quit IRC | 22:22 | |
| *** smatzek has quit IRC | 22:22 | |
| *** poul has joined #openstack-ansible | 22:22 | |
| *** smatzek has joined #openstack-ansible | 22:23 | |
| *** smatzek has quit IRC | 22:27 | |
| *** poul has quit IRC | 22:27 | |
| *** jose-phillips has joined #openstack-ansible | 22:29 | |
| *** poul has joined #openstack-ansible | 22:32 | |
| *** poul has quit IRC | 22:37 | |
| *** pedja has joined #openstack-ansible | 22:41 | |
| *** woodard has quit IRC | 22:44 | |
| *** savvas` has quit IRC | 22:48 | |
| *** poul has joined #openstack-ansible | 22:49 | |
| *** jose-phillips has quit IRC | 22:51 | |
| *** jose-phillips has joined #openstack-ansible | 22:53 | |
| *** poul has quit IRC | 22:53 | |
| *** smatzek has joined #openstack-ansible | 23:04 | |
| *** poul has joined #openstack-ansible | 23:05 | |
| *** gutter has joined #openstack-ansible | 23:06 | |
| *** dave-mccowan has joined #openstack-ansible | 23:06 | |
| *** poul has quit IRC | 23:10 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: [DNM] debug why on metal fails on gates and not on my machine. https://review.openstack.org/524954 | 23:12 |
| *** gutter has quit IRC | 23:14 | |
| *** smatzek has quit IRC | 23:19 | |
| *** smatzek has joined #openstack-ansible | 23:19 | |
| *** smatzek has quit IRC | 23:21 | |
| *** smatzek has joined #openstack-ansible | 23:21 | |
| *** poul has joined #openstack-ansible | 23:22 | |
| *** smatzek has quit IRC | 23:26 | |
| *** poul has quit IRC | 23:27 | |
| *** poul has joined #openstack-ansible | 23:31 | |
| *** dave-mccowan has quit IRC | 23:31 | |
| *** poul has quit IRC | 23:35 | |
| *** poul has joined #openstack-ansible | 23:48 | |
| *** poul has quit IRC | 23:53 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: [DNM] debug why on metal fails on gates and not on my machine. https://review.openstack.org/524954 | 23:53 |
| *** gunix has quit IRC | 23:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!